Ordering of messages or ConsistentHashing for Workers

Hey,
So I have hit a wall with that and cannot find a right solution.
What is the design pattern when you have to maintain some sort of ordering between process instances?

My usecase is that I receive data on Kafka topic, my consumer takes it all and spawns process instance in Zeebe for each of those requests. Then the flow goes on for all items orchestrated by Zeebe. But some items depends in some way on each other.
For one id we can have multiple versions. All the versions are sent to one kafka partition in order. How can we maintain ordering for those items? Is it possible at all?
Best would be to have some sort of ConsistentHashing for Workers, so that for designated key jobs would be assigned to very same worker always, guaranteeing order of processing of those items.

We found in docs that when you have a message start event process instance, you can add correlationKey for that and it guarantees that only one processInstance will exists in time for that correlationKey.
Docs: Messages | Camunda Platform 8

But it has a drawback it stops the world for the items in the queue. And I just wish it to process simultaneously but in order of receving (so the flow progress for all items in the meantime, and not wait for the very end of the oldest).

Is there any design pattern for that case?

Hi @jt89,

that is an interesting question.

The idea with the message start event sounds valid but has its downsides with the delay.

Another idea would be an orchestration process to coordinate the work.

In general, the job workers can’t control the order of the jobs. Usually, a worker picks the jobs in the order as they are created. But it doesn’t work if there are more than one job worker for a job type. Or, a job worker fail a job, or a job times out.

Best regards,
Philipp