How to Start ProcessInstance idempotently?

Camunda 7 Topics Discussion & Questions
1

my expectation is as follow：

however times you call
POST /process-definition/aProcessDefinitionId/start ， with the same businessKey ,
should return the same ProcessInstance.

but now each time I call this interface with the same businessKey, I got a new ProcessInstance .

Why i want to do this ? → the REST interface may timeout, when got exception, I wanna to retry the action.

1 Like
2

Hi,

Could you please rephrase your question? I don’t get the connection between “Start ProcessInstance with retry safely” and “businessKey”.

Cheers,
Christian

3

Hi,

I believe I understand the issue you are dealing with. We have had a similar issue…

We use an integration layer to start process instances using the REST API. On occasion, comms may drop out. Hence the integration layer may have a lost response and thus doesn’t know if the process instance started or not. Thus the integration layer must try again, however the start process API is not idempotent resulting in potentially multiple instances for the same business key. By default, the engine does not enforce uniqueness on the business key. See [1] for additional detail.

Hence you probably have four options;

  1. Turn on the unique business key constraint (Im not that familiar with this option)
  2. At the start of your process, use the engine API to find an existing process with the same business key and if found, abandon this instance.
  3. Use an external business key lock, eg external database etc.
  4. Query the engine API for a process with the business key before trying to start one.

regards

Rob

[1] https://docs.camunda.org/manual/7.4/user-guide/process-engine/database/#business-key

2 Likes
4

you mean add unique key on database table of processInstance？

5

Hi,

looking at the guide, it looks like its added to the Execution entity. Note I should point out, none of the proposed solutions actually make the start API idempotent. They either fail ‘silently’, eg a duplicate instance which self exits, or they explicitly throw an error. If you wanted true idempotency, you would probably have to write an API wrapper which performs the checks and responds appropriately.

R

2 Likes
6

Still another problem：

if you start a ProcessInstance that finish fast，then the execution records had been moved to the history tables ( act_ru_* -> act_hi_* )；

So we may need to call Runtime & History REST api together to confirm the same business-key not exists.

7

Note that Rob’s proposed solutions 2 and 4 may not work when two process instances with the same business key are started in parallel. In such a case, you may still end up with two persisted process instances with the same business key. If you need uniqueness, also for processes that are never persisted to the runtime tables, I recommend going with Rob’s solutions 1 or 3. For 1, given that you use the history tables, it should be sufficient to define uniqueness on ACT_HI_PROCINST.BUSINESS_KEY_.

Cheers,
Thorben

2 Likes
8

@thorben, hello and thanks for your remarks. We use synchronization by lock (one per businessKey), but again rarely duplicated process instances can be created :frowning:
Can it be so,

.createProcessInstanceQuery().processInstanceBusinessKey(businessKey).active().singleResult()

doesn’t return process, that was already started?

Maybe because there wasn’t wait state in a .bpmn, for DB commit of created process instance

9

Hi
You are correct that the engine must flush state to the DB for these kinds of approaches to work. Thus prevention of a duplicate can be difficult. An alternate approach is to detect duplicates and design the duplicates to gracefully end…

This video presentation discusses some paterns with examples on how this can be achieved…

regards

Rob

1 Like
10