Issue using @Transactional with Camunda batch

Community Community Extensions
,
1

We found an issue when trying to use @Transactional along with a batch operation in Camunda. For a reason that we don’t understand, the Camunda operation is being launched too late, AFTER the code wrapped by the transaction is executed, as opposed to be executed inside that code in the place where it is invoked.

An entire a complete project where the issue can be reproduced is available in

camunda-example-spring-boot-transactional

In order to run the application, you just need to execute

mvn spring-boot:run

The problematic method is transactionBatch() in the class CalculateBatchImpl.java. The method creates a fake list of data and calls the batch execution

logger.info("Create new Batch");
final List<String> simpleStringList = IntStream.range(0, 20)
      .mapToObj(i -> "SomeRandomBatchData_" + 
            UUID.randomUUID()).collect(toList());

// Execute a custom camunda batch
Batch batch = CustomBatchBuilder.of(simpleStringList)
      .configuration(this.processEngineFactoryBean.
            getProcessEngineConfiguration())
            .jobHandler(batchJobHandler).create();

Then, we implemented a mechanism to wait until the asynchronous process finishes, using an await library:

BatchQuery batchQuery = 
      processEngineFactoryBean.getProcessEngineConfiguration().
            getManagementService()
                  .createBatchQuery().batchId(batch.getId());

// await to know when the camunda batch has finished
await().atMost(5, TimeUnit.MINUTES).until(
      () -> 0 == batchQuery.list().size());

logger.info("Finish new Batch");

We put some log information inside the job handler to check when the Camunda batch is being executed. This code is inside the execute() method of the handler:

	@Override
	public void execute(List<String> data, CommandContext commandContext) {

		data.forEach(entry -> {
			logger.info("Working on data: {}", entry);
			runtimeService.startProcessInstanceByKey("loanApproval");
		});
	}

So, when executing the application, we expect to see in the log something like:

Create new Batch

Working on data: SomeRandomBatchData_0876151d-96dd-45ca-ae6e-3c2dbea8bfe0
Working on data: SomeRandomBatchData_3bcaf3a9-4e22-4894-be5b-afe3dd736944
Working on data: SomeRandomBatchData_6a3a1c55-da83-413b-b1fb-5130e2214c50

Finish new Batch

Unfortunately, what we see is as if the method finishes first and the Camunda batch operation happens after. Something like this:

Create new Batch

Finish new Batch  <- This shouldn’t happen yet!!

Working on data: SomeRandomBatchData_0876151d-96dd-45ca-ae6e-3c2dbea8bfe0
Working on data: SomeRandomBatchData_3bcaf3a9-4e22-4894-be5b-afe3dd736944
Working on data: SomeRandomBatchData_6a3a1c55-da83-413b-b1fb-5130e2214c50

Actually we see that the Camunda operations happen after the run() method in the main class.

If we remove the @Transactional annotation, the application work as expected, executing the batch operation inside the scope of the method.

@Transactional is mandatory for us because we need to do some local database operations AFTER the Camunda batch operation is finished and we need those operations to meet an ACID transaction along with the batch.

Thanks in advance for helping us to understand this behavior.

Juan.

2

Hi @jppazmin ,

the Problem is that when you use the @Transactional in the method where you create the batch, then camunda will use your Transaction. And the batch will be created after Finishing the Transaction. This is no camunda (batch) issue, this is a normal transactional behaviour.

So if you want to use the @Transactional at your service, you have to move out the batch query. Or you have to ensure that Camunda uses it’s own transaction, there are several ways to approve this. (E.g. by using the CommandExecutor from Camunda. On the ProcessEngineConfigurationImpl you can get the getCommandExecutorTxRequiresNew. You just have to wrap your call for creating a batch into a Camunda command.

But I do not understand why do you have to check directly after creating the batch if it is finished? A batch is completely async an could last a long time to finish. (Depending of the amount) And await is a test utility, which is also blocking your thread, this should not really be used in prod code.

Maybe if you really need to know about an end of a batch, a good solution to solve this would maybe to use a simple Camunda process? In the first delegate you start the batch, and after it you have a little loop with a second delegate and a timer where you check every x seconds it the batch is finished.

Cheers,
Patrick

1 Like
3

Hi @patrick.wunderlich

Thank you for your answer!

You are right this is normal transactional behaviour, we going to adjust the code to moving moving out the batch query.

Cheers

Juan.