We are using zeebe version 8.2.12 deployed on kubernetes through helm chart. The deployment has 1 gateway and 1 broker instance. There are 2 exporters configured - elasticsearch and hazelcast. We are using the spring-zeebe client to register a job worker for a custom type “http”.
We have observed that two process instances do not execute in parallel despite updating the ZEEBE_CLIENT_WORKER_THREADS environment variable to 5 on the zeebe client. Only when the first process completes, the other process instance executes.
This is causing a major performance bottleneck. Can the experts on this group share insights into what could be causing this blocking behaviour?
Hi @jgeek1 - I believe your issue isn’t with your job worker but with your single broker instance. If you add an additional broker do you experience the same issue? Our recommended minimum configuration is typically 3 brokers.
@nathan.loding - Digging further in the code I observed that increasing the number of job workers for a given job type increases the parallel execution of jobs. We did this on zeebe 8.2.12 and it worked.
From the logs we could see if there is only one job worker for a job type, the job poller doesn’t poll for new jobs. The reason could be because there is 1:1 relationship between the job poller and job worker. So spawning new job workers as a workaround worked.
It possibly looked like a bug hence I have created an issue on github here. If you could confirm this behaviour and help expedite the fix for this it would be helpful.