Fetching external tasks creates Optimistic Locking Exception. How to fetch only not locked tasks?

Camunda 7 Topics Discussion & Questions
1

Hello,

we have a problem when fetching external tasks asynchronously:

  • We fetch external tasks periodically
  • Asynchronously we execute them and we complete them

The problem is that eventual calls to fetchAndLock that will occur before we complete the task, will make completion fail with an optimistic locking exception. At the moment we solved that by keeping track of running tasks and avoid to call fetchAndLock if there are tasks that are in process, however this will typically reduce our capacity to parallelize tasks.

In production, we can also make our polling frequency > average time to complete tasks, but this is not a solution either.

Can we fetch only ā€œnot locked tasks?ā€

2

@Edmondo_Porcu can you use the notLocked query parameter on the Get List for External Tasks API endpoint?

notLocked

Only include external tasks that are currently not locked
(i.e. they have no lock or it has expired).
Value may only be true, as false matches any external task.

Might be a bit of a two step processā€¦

3

Perfect, I think that was what I needed. I just couldnā€™t find the api

4

Hi,

Im puzzled - I thought the intent of fetch and lock was to enable processing nodes to ā€˜reserveā€™ and thus take ownership of unprocessed external tasks. Then when the task is complete, mark it as complete. In other words, the behaviour is the same is the job executor acquisition and processingā€¦

regards

Rob

5

Surprisingly there is no ā€œnotLockedā€ on the java Api 7.5.3. My feeling @StephenOTT is that we should abandon the idea of interacting in memory with the engine and start interacting only via rest :slight_smile:

6

@Webcyberrob your point is fair though obviously you need a polling consumer that fetches tasks and completes them. If you do this asynchronously, you might end up fetching the same task twice (i.e. therefore increasing its version number in the database) and you will get an exception when you try to complete it.

7

Hi @Edmondo_Porcu,

Just to clarify the behavior: ExternalTaskService#fetchAndLock should never return the same task twice unless the lock timeout has expired previously. The idea of this method is to fetch and lock a task in one transaction. Of course locking the task may fail with OptimisticLockingException if there are multiple such requests in parallel. In that case, the exception is caught within the engine and less tasks are returned than requested.

Cheers,
Thorben

8

Thanks @thorben, how does this relate with the notLocked attribute described above?

9

notLocked is a parameter of the query API that can be used to find any external task in any state, for example for monitoring purposes: https://docs.camunda.org/manual/7.5/reference/rest/external-task/get-query/

#fetchAndLock does not offer this parameter, since it addresses non-locked tasks by definition: https://docs.camunda.org/manual/7.5/reference/rest/external-task/fetch/ For the use case of locking and executing tasks, you should use the fetchAndLock methods, not the query API.

1 Like
10