CallActivity and multi-threading in Java

Camunda 7 Topics Discussion & Questions
1

Hi Everyone,

My question is similar to this, but it seems having different issue that related to multi-instance settings.

I have a CallActivity in my process. A collection of data comes from uperstream, each item in the collection triggers the CallActivity once.

As the CallActivity has IO operations, I want it to run concurrently (with java multi-threading).

First try, I have set a TaskExecutor with corePoolsize=10, and expect all 5 threads to start/finish in concurrency.

From logs, I see the tasks are executed one by one, even when they are distributed to different threads.

16:49:04.048 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 0
16:49:04.151 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 1
16:49:04.254 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 2
16:49:04.356 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 3
16:49:04.458 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 4
16:49:04.728 [pool-2-thread-1] INFO com.test.MyTask - Task 2 count 0
16:49:04.830 [pool-2-thread-1] INFO com.test.MyTask - Task 2 count 1
16:49:04.933 [pool-2-thread-1] INFO com.test.MyTask - Task 2 count 2
16:49:05.036 [pool-2-thread-1] INFO com.test.MyTask - Task 2 count 3
16:49:05.139 [pool-2-thread-1] INFO com.test.MyTask - Task 2 count 4
16:49:05.387 [pool-2-thread-1] INFO com.test.MyTask - Task 3 count 0
16:49:05.489 [pool-2-thread-1] INFO com.test.MyTask - Task 3 count 1
16:49:05.592 [pool-2-thread-1] INFO com.test.MyTask - Task 3 count 2
16:49:05.694 [pool-2-thread-1] INFO com.test.MyTask - Task 3 count 3
16:49:05.797 [pool-2-thread-1] INFO com.test.MyTask - Task 3 count 4
16:49:06.586 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 0
16:49:06.689 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 1
16:49:06.791 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 2
16:49:06.894 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 3
16:49:06.997 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 4
16:49:07.246 [pool-2-thread-2] INFO com.test.MyTask - Task 5 count 0
16:49:07.348 [pool-2-thread-2] INFO com.test.MyTask - Task 5 count 1
16:49:07.451 [pool-2-thread-2] INFO com.test.MyTask - Task 5 count 2
16:49:07.553 [pool-2-thread-2] INFO com.test.MyTask - Task 5 count 3
16:49:07.655 [pool-2-thread-2] INFO com.test.MyTask - Task 5 count 4

Then I tried to uncheck “Multi Instance Exclusive”, Tasks are running concurrently as expected, but getting OptimisticLockingException when camunda save the results.

19:02:52.868 [pool-2-thread-2] INFO com.test.MyTask - Task 2 count 0
19:02:52.970 [pool-2-thread-2] INFO com.test.MyTask - Task 2 count 1
19:02:52.995 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 0
19:02:53.073 [pool-2-thread-2] INFO com.test.MyTask - Task 2 count 2
19:02:53.096 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 1
19:02:53.136 [pool-2-thread-3] INFO com.test.MyTask - Task 3 count 0
19:02:53.175 [pool-2-thread-2] INFO com.test.MyTask - Task 2 count 3
19:02:53.198 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 2
19:02:53.238 [pool-2-thread-3] INFO com.test.MyTask - Task 3 count 1
19:02:53.278 [pool-2-thread-2] INFO com.test.MyTask - Task 2 count 4
19:02:53.301 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 3
19:02:53.341 [pool-2-thread-3] INFO com.test.MyTask - Task 3 count 2
19:02:53.403 [pool-2-thread-1] INFO com.test.MyTask - Task 1 count 4
19:02:53.443 [pool-2-thread-3] INFO com.test.MyTask - Task 3 count 3
19:02:53.545 [pool-2-thread-3] INFO com.test.MyTask - Task 3 count 4
19:02:53.625 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 0
19:02:53.711 [pool-2-thread-1] INFO com.test.MyTask - Task 5 count 0
19:02:53.728 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 1
19:02:53.814 [pool-2-thread-1] INFO com.test.MyTask - Task 5 count 1
19:02:53.830 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 2
19:02:53.916 [pool-2-thread-1] INFO com.test.MyTask - Task 5 count 2
19:02:53.932 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 3
19:02:54.019 [pool-2-thread-1] INFO com.test.MyTask - Task 5 count 3
19:02:54.034 [pool-2-thread-2] INFO com.test.MyTask - Task 4 count 4
19:02:54.121 [pool-2-thread-1] INFO com.test.MyTask - Task 5 count 4
19:02:54.354 [pool-2-thread-2] WARN org.camunda.bpm.engine.jobexecutor - ENGINE-14006 Exception while executing job 12979:
org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of ‘UPDATE ExecutionEntity[12922]’ failed. Entity was updated by another transaction concurrently.

From documentation optimistic-locking, I see OptimisticLockingException is expected for multi-threading.
But in my case,
(1) I observed OptimisticLockingException, but the logs does not show concurrent threads, neither any “retry”, is it expected?
(2) If I understand correctly, OptimisticLockingException happens at the end of a task execution. If there are N threads concurrently executing the same task, there will be N-1 wasting their time because only one can have the optimistic lock. As my real-case CallActivity is long, I’m trying to assign only one (pooled) thread on one task, is it doable with configuration? There might be different solution for my case, please kindly advise.

Thank you in advance

main_process.bpmn (3.4 KB) my_activity.bpmn (2.3 KB)
DataProvider.java.txt (450 Bytes) MyTask.java.txt (718 Bytes)

2

To have the activities executed in different threads, you have to correctly set the “async before” attribute, so that each iteration is executed in its own job.

1 Like
3

As @fml2 said. Make sure to check the “multi instance asynchronous before” box, so that each iteration of the multi instance is executed asynchronously, allowing your threadpool to be used to its full extent for the work created. That’s assuming you already configured the multi instance to be in parallel, not sequential, of course.

For even more parallelism, but slightly more overhead, you can consider to make the activities inside the process that’s started from the call activity asynchronous before.

2 Likes
4

I think you have to additionally uncheck the “Multi Instance Exclusive” option.
When the exclusive option is checked, the engine will use the same thread for all tasks (to avoid optimistic locking exceptions)

2 Likes
5

Thank you for response @fml2 @tiesebarrell and @KeyKon
I have followed your suggestions and the processes are running in concurrency as expected.
The only concern was the OptimisticLockingException.
After some research, I noticed the OptimisticLockingException happens on nrOfCompletedInstances, a Camunda managed variable, and it has expected value at the end of execution.
Given my tasks (business logic) are completed correctly, and in concurrency as expected.
As mentioned here, OptimisticLockingException is expected and handled correctly by Camunda. The exception does not cause business logic retry as long as there is no other shared parameter between CallActivities.

Thank you for having provided suggestion.