Hi,
I’m using Camunda 8 to automate a complex process. Currently, I’m relying heavily on Service Tasks, each with its corresponding worker. However, I’m concerned about performance because workers use a pulling mechanism to fetch and process messages.
I am considering the use of RabbitMQ as middleware, with workers subscribe to RabbitMQ to process instead of directly pulling from Camunda. After processing, workers call Camunda to complete task to move to next step. In order to do that, I need to replicate the necessary information such as JobKey, ProcessInstanceKey and Variables from Service Task to RabbitMQ.
How can I implement the above approach, or do you have any idea?
Thanks, in advance.
Introducing RabbitMQ will never improve performance. It will increase latency and complexity. It makes sense when integrating with other systems and when you have that infrastructure in place.
But to get any job out of the engine, you must poll it out with a gRPC API call. That’s the only way that it can get to RabbitMQ.
So, all you are doing is adding RabbitMQ to the other side of a polling worker. So the system you describe will be “Zeebe Gateway → Zeebe Worker → RabbitMQ → Service Task Worker → Zeebe Gateway”.
It is sometimes referred to as the “decoupled worker completion pattern”. It breaks up the encapsulated functionality of the job worker (polling, activation, calling business logic, completing job) so that you can distribute it throughout a system. This can be useful when the business logic is some other system (like a mainframe) that you need to interface with asynchronously to complete a task.
But, as I said at the outset, it increases latency and complexity. It doesn’t sound like you have the problem for which this architecture is the solution.