JobExecutor MaxPoolSize and Datasource MaxPoolSize (avoiding deadlocks)

Hello, I’m using the Camunda Springboot Starter.

I’ve noticed that the default max-pool-size of the Camunda JobExecutor is 10.

Question 1: That means that the max concurrently running process instances should be 10, correct?

Question 2: If so, let’s say all process instances are synchronous, which mean that they open a DB transaction at the start of each process, and commit/rollback the transaction at the end of the flow or when an error occurs. Then at-most, max 10 database transactions should open?

My requirement: My requirement, is to hold some internal progress of the process in custom tables. The only way to achieve this in Springboot, is by using Propagation.REQUIRES_NEW in order to open a new nested transaction which is isolated and commits by itself when done.
This requirement, might lead to some weird deadlock situations (the Datasource transaction max pool size is also 10). For example, if all process instances start concurrently, and all 10 transactions get allocated, then there won’t be left transactions from the datasource pool for the Propagation.REQUIRES_NEW to log the internal progress, ultimately leading to deadlock.

Solution: My solution to that, is that the Datasource MaxPoolSize should be atleast double of the Camunda JobExecutor MaxPoolSize (Datasource MaxPoolSize=20+). This way I didn’t face any deadlocks, since each process instance is capable of opening at most 2 transactions for itself.

Question 3: Even though I’ve read that REQUIRES_NEW isn’t the best practice, is this the correct approach to achieve my requirement for internal logging or are there safer ways? Am I on the right track on this issue or increasing the Datasource MaxPoolSize might lead to the same problem but on a different scale?

Question 4: Is there a way I can decrease the JobExecutor max pool size? I tried through application.yml but it still stayed on the default value (MaxPoolSize = 10).

2 Likes

Good questions @nmanitaras , I also have the same doubts on Job Executor and faced in concurrent request. I was running the performance test for 40 users for 10 minutes and then it failed. So anyone can help on below points-

  1. What are the best practice to configure the thread pool of job executor ?
  2. Will there be any impact on other core feature of camunda if increase the MaxPoolSize ?

2 Likes