Fetch And Lock API - usage, use priority and limits

[thorben]

@Ivgi

Is there a limit to how many different external tasks (topics) i can ask for in the FetchAndLock API request?

I believe this filter translates to an IN clause in the SQL query. E.g. Oracle can only handle up to 1000 values in one such clause.

@Ivgi

Is there some sort of mechanism to prevent an external task starvation?

There is no such mechanism. We considered implementing something like that for job execution at one time, but I don’t believe there is an efficient way to do so when the queue is built on top of a relational database. So make sure you have enough processing resources :). It may be possible to build e.g. a priority elevation mechanism outside of the engine by using the ExternalService#setPriority API.

@Ivgi

I was wondering what might be a better approach for using the FetchAndLock API?

Is it having 1 c# worker per external task ( 1:1 ) or maybe 1 worker for all of the tasks ( 1 : All ), or maybe some sort of a hybrid, for example maybe seperate the external tasks logically into groups and have 1 worker per each such group polling the active tasks?

I haven’t got experience with this, sorry. Probably depends on your use case and which component (Camunda engine, task poller, task executor) limits the performance. Note that having more than one poller can also be useful in terms of failover.

Cheers,
Thorben