maxJobsPerAcquisition

Camunda 7 Topics Discussion & Questions
1

Good day

The default value for maxJobsPerAcquisition is 3. If I understand correctly this means that even if there are hundreds of process instances only 3 jobs (e.g. services tasks) would be executing at a maximum at any given moment. My questions now is this:

  1. Is it common practice in production environments to increase the value for maxJobsPerAcquisition?
  2. Also, what is the best way to determine the optimum value for maxJobsPerAcquisition?

Thank you.
Giovanni

2

There are a lot of reasons you might be messing around maxJobsPerAcquisition but usually it’s got something to do with the number of process instances you expect to start in any given second.

Generally speaking if you’re starting less than 10 instances per second. 3 is perfectly fine. But you you’ve got a bunch of nodes and maybe 100 or so instances per second you might start seeing performance issues and in that scenario you could increase the jobs per acquisition to keep nodes busy for longer.

3

Thank you for the reply. It is still not 100% clear in my mind. My doubt is with this scenario:

If I have 100 process instances of a simple process model with just one service task. This service task however take exactly 5 minutes to run (for the execution to return) and the service is synchronous.

  1. Does this mean that (roughly) for 5 minutes at a time only 3 job would be running and only as these jobs finishes new jobs would be running?
  2. Is increasing the maxJobsPerAcquisition a acceptable way to solve this?
4

It doesn’t refer to the number of threads doing jobs - it’s the number of jobs that are acquired each time the engine asks for work to do.

5

So, it means that the jobexecutor will take up 3 jobs, then run them until done and only take up new jobs if a previous one is completed?
So, if the service tasks are long running, like in the scenario above, I have to increase maxJobsPerAcquisition to have more then 3 job being executed at any time?

6

Hi Gio,

Not quite - check out this [1] diagram. Job acquisition and job execution will typically run in separate threads. Thus as per the diagram, the job acquisition thread will lock up to three jobs at a time and queue them for execution. If there are execution threads free in the execution pool, these jobs will be de-queued and executed.

Thus if the max size of the thread pool was say 100 threads, and the jobs are relatively long running, you could have 100 concurrent jobs running even though only three were fetched at a time. To use a crude analogy, the job acquisition thread is a bit like the man shovelling coal into a steam engine. Each shovel full may contain 3 lumps of coal, but he can keep shovelling into a large burner…

The trick is to find a balance between acquisition and execution such that the pipeline is tuned…

regards

Rob

[1] https://docs.camunda.org/manual/7.6/user-guide/process-engine/the-job-executor/

3 Likes
7

Hi Rob

Very well explained, Rob!
Thanks a lot all for the feedback.

Ciao
Giovanni

8