Our company is currently unable to upgrade one of our primary customers to the newer version of our software that utilizes Camunda 7. Previously we used the YAWL engine, which is what they’re using now.
They have a requirement of being able to have multiple Camunda engines / multiple installations of our software that we can query the number of running jobs on, and programmatically choose the resource with the fewest running jobs to execute the .bpmn file on.
However, many of our .bpmn files are “asynchronous”, which causes the Camunda API call to " /process-instance/count" to always return 0, even though there infact are jobs running on these machines.
Also, async process instances do not appear in the Camunda cockpit. We only see process instances for synchronous ones.
How can we make our workflows stay asynchronous but return non-zero values when we call " /process-instance/count"??
I think your approach is clever and I would like to point you to Camunda 8 and Zeebe directly as it is distributed by architecture and handles your issue already.
Your approach can also work, but I think the measurement api endpoint you are actually looking for is this one:
Instead of measuring process instances, you can measure jobs. Jobs are the async work the job executor needs to do and the count of jobs gives you an idea how much load a process engine faces.
If you are using external tasks, you might also like to add the count of open tasks as well:
Update… The reason we weren’t seeing process instances appear for some .bpmn files in the Camunda Cockpit and for /process-instance/count is because we simply needed to check these boxes for all service tasks for all of our .bpmn files.
alright, I am glad to hear this. The difference between a job and a process instance:
A process instance is the instance of a process definition, and a process instance lives from when the process instance is instantiated (by triggering a start event) until it is finished (by having all tokens consumed in end events in most cases). A job is a piece of work that needs to be done in the background and spans one database transaction.
By ticking the box you showed, you tell the engine to save the state before invocation of the service task to the database, commit the transaction and create a job that takes over from there.
The job will be done asynchronously by the job executor. The amount of due/overdue jobs is a very good measurement for workload on the process engine. The more open jobs, the more delay in the execution.
This is why I would preferably measure jobs instead of process instances.
Thanks for your reply. Unfortunately we are facing a different issue now, it’s not waiting for the tasks to finish after checking the boxes in the image I sent.
Hello, I’ve tried /job/count, /external-task/count, and /process-instance/count APIs and we are still not seeing correct counts - on the machine with synchronous workflows, we keep seeing a static count of 0 for all 3 APIs, even when I can see tasks actively running in Process Explorer. When I run the same API on the other machine with asynchronous tasks & a visible process instance in the camunda cockpit, the /job/count API returns a value of 7. After checking the act_ru_job table there are entries in there from several days ago even though the jobs are not currently running. We discovered this API is querying this table. This tells me the /job/count and /external-task/count APIs cannot be relied on for synchronous workflows.
Additionally, we’ve received some complaints from several members on the team (including my boss) that it’s not helpful that Camunda shows a different count in the Cockpit than the API is giving us.
It seems we need synchronous tasks to appear in the cockpit without making them asynchronous / without checking the boxes, and need the /process-instance/count API to return non-zero values for synchronous workflows. That way we can count the process instances since the other APIs seem to not return values that reflect the correct number of actively running tasks.
did you also try to look at the jobs you queried for under /job? This could give you an insight on how query variables might help you.
The reason why synchronous process instances do not appear in any metrics here is that they do not leave any database entries. It will run transactional and build statements, but on commit, they will all go straight to the history table.
Tasks are something different again. Tasks are user tasks. They are also saved to the database and do not affect performance of the engine.
For more reading on performance indicators (and maybe also performance tuning), please read here:
The page in general contains Best Practices about Camunda Products, the section specifically handles your issue. These are findings from many years of Consulting about our products with our costumers in the field.
And if you want to use Prometheus for measuring performance and load, there is a nice plugin:
sorry about that