Transfer activated job to a specific target pod in a kubernetes cluster.

Hi Camunda Team,

We are trying to do a POC to build a synchronous request response model with Camunda ZeeBe.

We have:

  1. A java based micro-service in a Kubernetes cluster
  2. ZeeBe Gateway (along with ZeeBe Brokers) hosted in a separate Kubernetes cluster.
  3. The micro-service also contains a User-Task Job Worker that polls ZeeBe Gateway for Activated jobs.

The micro-service starts a workflow by initiating a call to ZeeBe gateway and waits till the UserTask wait state is reached and job is activated. User Task Job Worker then receives the activated job asynchronously and joins the original waiting thread to close loop.

Now the problem is this works well only if you have a single pod in your micro-service Kubernetes cluster. If we have multiple pods, then user job worker from any random pod gets the job and is not able to close the original thread which could be waiting in a different pod.

Though results look better with single pod, it will not be able to scale. This acts as a major hindrance to make use of ZeeBe scaling capabilities.

Please suggest if there are any ways to poll/route jobs of a specific process instance to a target pod/job-worker?

This POC could also be an anti-pattern for ZeeBe architecture, which promotes event based asynchronous communication pattern. But do you know if there is another way to address large scale use-cases that expect synchronous request-response from ZeeBe workflow engine?

Thanks,
Hirdesh Tomar