How to avoid duplicate BussinessKey

Hello Everyone,

We are developing an order management workflow that sends out the notification/message as and when we receive the order. We are facing the problem that producing duplicate notifications, reason behind the duplicate notifications is that we are seeing duplicate requests are being pushed to workflow within a fraction of seconds (like 4-5 ms). At the time of 2nd record is getting committed in the DB, it is getting rejected (we have added DB constraint), by the time both processes/requests would have sent the notification already.

We have added unique constraints on DB also but the issue is still there as the complete process/workflow is in a single transaction. If there is a failure in notification we will throw out the error for workflow creation itself.

We are thinking about introducing one more table just to avoid duplicates, we are gonna introduce 2 step transaction processes wherein, before triggering the workflow we will make an entry for the order number if that doesn’t exist in the table and commit the transaction and then we will create a workflow, so in case of duplicate orders, database check will fail and won’t create a duplicate workflow.

Is there any other better approach to avoid the duplicate business key?

business key - order number.
Postgresql DB scripts, constraint we used :

Runtime:
alter table ACT_RU_EXECUTION add constraint ACT_UNIQ_RU_BUS_KEY UNIQUE (PROC_DEF_ID_, BUSINESS_KEY_);

History:
alter table ACT_HI_PROCINST add constraint ACT_UNIQ_HI_BUS_KEY UNIQUE (PROC_DEF_ID_, BUSINESS_KEY_);

Workflow snapshot:

Hey :wave:

An idea would be to implement a listener on startup.

Before the process will be executed, the listener gets the business key and lookups in camundas datatables maybe by the Process Engine API with the business key and the process id or s.th. like that.

Should be the result of the query not null, the key already exists and the process trigger will be rejected else the process can be started.

Could be a solution if the API can handle such queries!?

We have attached one listener at the start event of the process, by the time the listener checks in the DB, if we get another request within a few milliseconds, both processes/requests would have already executed or started executing the send task, and would have sent the notification.

Maybe you can block other requests like the function „lock and fetch“ of the external task pattern or you can implement a queue (like a rabbit mq) in front of the process to buffer the trigger.

We already discussed this option to implement the queue either using rabbit mq or DB (like the way external task works). But still, we are looking for any other option before proceeding.

Hi @Tushar_Kapadnis,

one way to avoid duplicate business key is this process model: grafik

If you use message correlation with a business key to start the process instances, additional process instances will be absorbed in the non interrupting event sub processes.

Hope this helps, Ingo

1 Like

Hey @Ingo_Richtsmeier, Thanks for the reply, We did the changes as per your comments, but still, I could see 2 instances are getting created for the same business key. We triggered 2 requests with differences of 3-4 ms. and could see 2 instances getting created. Please find the attached Modeller file screenshot (as I am not able to attach the BPMN file) and screenshot for your Reference.

Is this means you are first sending the notification then writing in the DB? If so, why? You can first write in the DB and then make notification synchronously. In this case if, DB writing fails because of duplicate key process will resul in error and will not send duplicate notification.

Hey @shamil,
Here is the thing, if the notification failed we don’t want to create a workflow and inform the client that there is a failure so that they can take care of the failure from their side. That is the reason we are not using Asynch before/after feature to commit in DB.

Hi @Tushar_Kapadnis,

the duplicate check happens only after the first (running) process is commited into the database.

If you set the async before flag on the first service task, the process instance will be saved directly after the start.

This setting should minimize the amount of time, where duplicate business keys will be created.

But it isn’t guaranteed, as two requests can be handled in the web server REST API in parallel.

It make it bullet proof, you have check before you start a process instance, if another one exists.

Hope this helps, Ingo

Thanks a lot…We will try something which will persist 1st and start the process.