Service task gets retried every ~6 Min on different Camunda threads, even if 1st instance execution is in a running state.
It’s like if a service task takes more than ~6 Min to execute then Camunda launches a new instance of the same task. which can lead to idempotency issues…
Is there any default config in Camunda that we can turn off to prevent this.?
Has anybody faced this before? I do not find any reference to this kind of issue on the internet.
Looking forward to getting help from the community.
I see that this behavior might seem odd, but arguably one does not want a process to get stuck because a single job does not complete, either. So, restarting a job prevents this issue.
Can you provide us some more information?
- How do you implement your services? As java delegates or as external task workers?
- Which version of Camunda are you running?
If you use the external task worker, jobs are only locked for a predefined amount of time. If you exceed this time, the job is offered again and may restart. You can extend an existing lock or increase the default. You can read more here:
If you use java delegates, this behavior may be caused by the internal job worker: it locks a job and removes it after successful completion. However, if completion takes too long, the same job may be picked up again. In this case, you may split up the job or change the configuration of the job executer. You can read more here:
can you pinpoint a particular config from here that can help here?
While I would prefer splitting up the task, you can set the lock time (default 5 min). This is done via the property camunda.bpm.job-execution.lock-time-in-millis. You can find more properties here.
→ Is there any way to continue executing the longer task without launching a new instance of the same task?
→ e.g. not splittable task like, polling from an external service on a regular interval until it gets desired output from the external service),
→ in such cases, retry shouldn’t happen even if 1st execution exceeds the lock-time-in-millis time in executing state.
I believe the short answer is no: if the lock-time-in-millis is exceeded, the job can be picked up by the job executor again, which leads to a new instance of the same task.
The long answer is: there are many ways of handling this. Let’s take the example that you gave: polling from an external service on a regular interval until it gets desired output from the external service. Without claiming that the following list is complete, here are some options:
Option 1: Model the Task as Technical Process Model
You can make the repeated polling and the interval part of your models. Make the task a call activity and link a process model such as the following:
Option 2: Continuations
Continuations allow that one job picks up the work where the previous job stopped. In Camunda, you can do this by setting and retrieving process variables in your long-running jobs. This way, multiple task instances are created, but they may know about each other.
Option 3: External Task Worker / Rest API
You can use external task workers / the REST API instead of java delegates. External task workers can explicitly extend the lock. This would effectively prevent multiple task instances.
1 Like
Hi @Vivek_Korat,
You have to extend the lock-duration-in-millis. Some background information: if the lock time is exceeded, the engine assumes that the job has died and needs to be restarted.
If your service execution takes longer by purpose, you have to aoounce this to the process engine with the configuration. It is applied to all jobs.
Hope this helps, Ingo
1 Like
