How to decipher this error log section?

Camunda 7 Topics Discussion & Questions
1

Hello,

we have a process that calls a BPMN subprocess. When processing hundreds of incoming messages (each message starts a new process), it seems that in the beginning of the subprocess or during the start of the subprocess Camunda engine fails, returns to an earlier completed service task and runs again into the same problem. This happens several hundred times in the main process, delays between one minute and twenty minutes.

the log fragment is below:

13-Jun-2016 20:30:15.021 INFO [pool-1-thread-11] org.camunda.commons.logging.BaseLogger.logInfo ENGINE-14008 Adding new exclusive job to job executor context. Job Id=‘946baf61-3162-11e6-b996-1418775051df’
13-Jun-2016 20:30:17.053 INFO [pool-1-thread-11] org.camunda.commons.logging.BaseLogger.logInfo ENGINE-14008 Adding new exclusive job to job executor context. Job Id=‘95b48348-3162-11e6-b996-1418775051df’
13-Jun-2016 20:30:17.169 INFO [pool-1-thread-11] org.camunda.commons.logging.BaseLogger.logInfo ENGINE-14008 Adding new exclusive job to job executor context. Job Id=‘95cf3749-3162-11e6-b996-1418775051df’
13-Jun-2016 20:30:17.192 INFO [pool-1-thread-11] org.camunda.commons.logging.BaseLogger.logInfo ENGINE-14008 Adding new exclusive job to job executor context. Job Id=‘95d74d9d-3162-11e6-b996-1418775051df’
13-Jun-2016 20:32:59.255 WARNING [pool-1-thread-10] org.camunda.commons.logging.BaseLogger.logWarn ENGINE-14006 Exception while executing job 1886bbca-314b-11e6-b996-1418775051df: org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of ‘DELETE MessageEntity[1886bbca-314b-11e6-b996-1418775051df]’ failed. Entity was updated by another transaction concurrently.
13-Jun-2016 20:34:33.681 WARNING [pool-1-thread-3] org.camunda.commons.logging.BaseLogger.logWarn ENGINE-14006 Exception while executing job 1300f34e-3151-11e6-b996-1418775051df: org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of ‘DELETE MessageEntity[1300f34e-3151-11e6-b996-1418775051df]’ failed. Entity was updated by another transaction concurrently.

Jindra

2

Hi Jindra,

it seems that you have a problem with concurrent job execution.

Can you please share your process and answer the following questions:

Do you have a single job executor?
Do you work with an execution concurrently (e.g. complete task, incoming event)?
What are the activities in the process where the job execution fails?

Best regards,
Philipp

3

Hello Philipp, sorry I was not checking the answers. We realized there was a problem with database overload. The problem was in business system database (not camdb) and it was causing timeouts in the respective service tasks. The task was implemented as Java delegate and was calling the database via JDBC, so the database query execution directly affected job executor in Camunda.

We solved the database problem (unrelated to Camunda at all) and the problem described above just disappeared.

However, is there a recommended pattern how to deal with this kind of integration tasks?

Jindra

4

By the way, long database queries blocked the whole business process because typically we had several dozen loan applications coming every minute, but the troublesome database query could take as long as 15 minutes :wink: Obviously, when all 4 jobs were waiting, the remaining several hundred processes could not continue.

5

Hi Jindra,

if you have long running service tasks then I would recommend the external task pattern. It decouples the work from the process engine.

Best regards,
Philipp

1 Like
6

Thank you for a tip. We have other services to connect to where this pattern could be useful.

However, in this case, it was just a wrong database tuning :slight_smile:

Jindra

7

@Philipp_Ossler, hi. I have the same bug. What do you mean as single job executor? Must i set configs maxJobsPerAcquisition or corePoolSize/maxPoolSize to 1?
Appreciate any help, i try to solve this issue for a month…

Here is my example: OptimisticLockingException at a parallel gateway

Some details:

  • the error occurs at the first time after long idle of services (one of them - mail-send connector), that i call.
  • service calling and parallel gateway are in subtask
  • With this error job has 3 retries with 5 min. period - even if i set async retry to R1/PT1M. May be this is any default retry config, @Philipp_Ossler?

I’ve found only one 5 min default period - lockTimeInMillis - maybe this retries execution i see?

8

Hi,

I also have a question wrt “Entity was updated by another transaction concurrently”.

Does the locking happen from jobs in the same job executor thread?
Or there might be case where same entity is updated by jobs that belong to different threads?

Thx