Delay in completing the processes when multiple process instances are started concurrently

Process Model:-

System specification: -
OS : Windows
CPU : 8 cores
Processor : Intel(R)Xeon(R) CPU E5-2686 v4 @ 2.30GHz 2.30 GHz
RAM : 32.0 GB

Camunda configuration in spring boot:-
camunda.bpm.job-execution.core-pool-size=8
camunda.bpm.job-execution.max-pool-size=50
camunda.bpm.job-execution.max-jobs-per-acquisition=20
camunda.bpm.job-execution.queue-capacity=20

We are using Spring Boot with embedded camunda 7.14.0 In this process we included five service task(Java).

For single execution taking 1426 milli second.
If we execute 8 concurrent execution, it is taking around 4 seconds

If we execute 20 concurrent execution, it is taking around 10 seconds

As we increase the concurrent requests for process execution, the time taken to complete all the processes is increasing accordingly.

Is this expected behavior with the mentioned configuration?

Should not the total time taken for execution be same for all the requests?

Do you have any service task or other element marked as Async before or Async after?
When you mark your task as async before/after it will act as a commit point, like other wait states (user tasks, timers, message receiver, etc).
Job executor will commit the transaction to the database and create a new job for the next activity of this process instance, so it will go to the end of the job queue.

In this scenario, its normal that when you have new jobs starting, the queue gets bigger and the next steps take longer to start.

If you want your process to run from the start to the end in a single transaction/thread, you cant have those commit points… but maybe you will have problems when any delegate throws some error and you get a full rollback as you didnt commit anything.

EDIT: I see now that you are using parallel gateway in your process definition and i think that this is enough to make your process not run in a single thread, thus it will commit a concurrent job to the database and wait at the end of the queue for a job acquisition thread to take it.

1 Like

Hi @Jean_Robert_Alves

Thanks for clarifying the behavior. It will be very useful if you could also help us with the links to understand more about the internal working of the jobs and its execution

I think this link will help you understand how it works when you have parallel jobs, and the exclusive execution of them:

https://docs.camunda.org/manual/7.16/user-guide/process-engine/the-job-executor/#concurrent-job-execution

In the same page you’ll find much more information about how job acquisition and job execution works.

Thanks @Jean_Robert_Alves