How to synchronise different workflows on same correlation key

[ankit_joinwal]

Hi ,

We have a use case of entity management(example blog post created/updated) wherein

• When an entity is created in an upstream service, we trigger a workflow (named CreateEntityWorkflow) which does a series of enrichments to the entity by orchestrating micro-services that implement enrichment steps.

• When an entity is updated, we trigger another workflow(named UpdatedEntityWorkflow) which does 80% similar enrichment as CreateEntityWorkflow and remaining 20% is business logic specific to update case.

Since the two workflows are independent, in a scenario where a CreateEntityWorkflow is triggered for a CreateEntity event and is in-progress (maybe paused for some external task to complete) and meanwhile an update entity event occurs which results in trigger of UpdatedEntityWorkflow. Thus the processing result of CreateEntityWorkflow might be overridden by tasks in UpdatedEntityWorkflow and would result in unpredictable results as both processes operate on the same record in database.

There can be multiple ways to handle this use case eg. locking on the entity until one workflow is completed so that the workflows are synchronised.

I wanted to know if this use case can be handled with Zeebe without requiring external locking solution. Any suggestions?

Thanks

[korthout]

Hi @ankit_joinwal

That’s a really interesting question.

One idea is to use the single instance message pattern. To use this in your case you’d have to extract the common part of the processes that should not be run multiple times at the same time into a separate process. Then use a send task to publish a message from either your CreateEntityWorkflow or UpdatedEntityWorkflow with a TTL=0 and using the same correlation key to enable message cardinality. Sadly this approach does not fully work for you, because the second process instance that runs does not actually update the entity anymore.

So instead of just a single instance and throwing away the subsequent instance we want to harness the power of message buffering as well. You can do that by increasing the TTL. The message will then be buffered for up to that time. When the existing instance completes, the buffered message will be correlated to the start event. If the TTL expires, the message will be deleted.

If you want to continue your original process instances you can use messages again to wait and continue. For example:

[ankit_joinwal]

Hi @korthout

Thank you for the suggestion. I had a follow up question - in the above solution can we call EnrichEntityProcess as a Call Activity from the two outer processes? I ask this because as per the doc Call activities | Camunda Cloud Docs call activity only supports none type start events and not message start events.

Thanks

[korthout]

In the above solution can we call EnrichEntityProcess as a Call Activity from the two outer processes?

Sure, just add a none start event to the called process as an additional start event. So then it has 2 start events, 1 none start event and 1 message start event.