Multiple Workflow's Parallel Processing

Camunda 7 Topics Discussion & Questions
1

Hi,

I’m facing an issue where I have two workflows and their respective instances are not getting processed parallelly by camunda. I have two workflows WF1.bpmn and WF2.bpmn, for load testing I started 50k-50k instances on each of the workflow at the same time. Once all 1 lakh instance reached to their respective timer event, I updated the DUEDATE_ column of ACT_RU_JOB for 1 lakh active instances to previous day so that the testing can be continued without waiting for timer to complete.

Query to update timer for 1 lakh instances in one go:

update act_ru_job set DUEDATE_ = sysdate-1 where process_def_id in (‘WF1:1:32a795d7-90b4-11ed-8f91-0a580a801275’, ‘WF2:1:32a9dfdf-90b4-11ed-8f91-0a580a801275’) ;

But then I observed that instances on WF1 were moved to next event first and when all the 50k instances got cleared then only WF2 instances waiting on timer started to move to next events. Sample workflows have been attached at the bottom.

In the middle of this testing when almost 10-12k instances of WF1 moved from timer event to next event, I thought defining more priority to WF2 timer event and restarting Camunda service may make a difference and their instances may started moving to next event. But nothing happened and I had to wait for WF1 timer event to complete.

Query to update priority

update act_ru_jobdef set job_priority_ = 5 where act_id_ = ‘MyTimerActId’ and proc_def_key_= ‘WF2’;
update act_ru_jobdef set job_priority_ = 4 where act_id_ = ‘MyTimerActId’ and proc_def_key_= ‘WF1’;

Above explained situation is not acceptable because journey of two customers running on two different workflows are completely independent and camunda should process them parallelly for better customer experience.

Can anyone please help me out to solve this problem, is there any configuration that needs to be done to make sure multiple workflows with their respective instances are processed parallelly.

Configurations:

Spring Boot Version: 2.5.4
JDK: 1.8
Database: Oracle SQL
CPU: 500 millicore(0.5core)/pod
Memory: 1GB/pod
Pods: 20(Running Parallelly to scale performance)

image
image1936×416 20.3 KB

image
image1936×416 20.4 KB

Thanks,
Rohit

2

Manipulating the Camunda DB directly is rarely a good idea. It will cause you many issues.

My personal guess on this is that it’s because you used a non-deterministic value for what you set the DUEDATE_ value to. If instead of using sysdate-1, you had used ‘2023-08-27 00:00:00’ then all 100K (lakh is a country specific unit of measure that many people don’t understand…) instances would have the exact same DUEDATE_, which also likely would break your engine, as it would bog down heavily. However, it would likely result in both the WF1 and WF2 being actioned at the same time.

3

Hi @rohit74,

By default the job executor does not impose an order in which acquirable jobs are acquired

the job acquisition query can be controlled by the process engine configuration properties. Currently, three options are supported:

  • jobExecutorAcquireByPriority. If set to true, the job executor will acquire the jobs with the highest priorities.
  • jobExecutorPreferTimerJobs. If set to true, the job executor will acquire all acquirable timer jobs before other job types. This doesn’t specify an order within types of jobs which are acquired.
  • jobExecutorAcquireByDueDate. If set to true, the job executor will acquire jobs by ascending due date. An asynchronous continuation receives its creation date as due date, so it is immediately executable.

All of these options are set to false by default and should only be activated if required by the use case. The options alter the used job acquisition query and may affect its performance. That’s why we also advise to add an index on the corresponding column(s) of the ACT_RU_JOB table.

Below docs might be of help to you.

4

@GotnOGuts, thanks for your response, but can you tell me how I can make sure that all instances of all workflows are executed parallelly when there are two or more events present with higher priority?