Deadlock issue

Hi,

I have been seeing Deadlock issue while external task is being completed. It doesn’t happen in normal conditions. When there are hundreds of workflows are running, it occurs sometimes.

Tech stack is

  1. SpringBoot - 2.1.7.RELEASE
  2. MySQL - 5.7.12

I have also looked into the following documentation on MySQL deadlock issue https://dev.mysql.com/doc/refman/8.0/en/innodb-deadlocks.html , It’s not really helpful.

Following is the stacktrace:

    org.camunda.bpm.engine.ProcessEngineException: ENGINE-03004 Exception while executing Database Operation 'UPDATE_BULK updateExternalTaskSuspensionStateByParameters org.camunda.bpm.engine.impl.db.ListQueryParameterObject@1f13c66b' with message '
### Error flushing statements.  Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExternalTaskEntity.updateExternalTaskSuspensionStateByParameters (batch index #5) failed. 4 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: Deadlock found when trying to get lock; try restarting transaction
### Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExternalTaskEntity.updateExternalTaskSuspensionStateByParameters (batch index #5) failed. 4 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: Deadlock found when trying to get lock; try restarting transaction'. Flush summary: 
 [
INSERT HistoricJobLogEventEntity[84029ef3-e132-11ea-a736-a21d4a9facf2]
INSERT HistoricJobLogEventEntity[84038955-e132-11ea-a736-a21d4a9facf2]
INSERT HistoricExternalTaskLogEntity[840229c2-e132-11ea-a736-a21d4a9facf2]
INSERT HistoricActivityInstanceEventEntity[EndEvent_ta_1:8402ed14-e132-11ea-a736-a21d4a9facf2]
INSERT HistoricActivityInstanceEventEntity[EndEvent_ta_2:8403b066-e132-11ea-a736-a21d4a9facf2]
DELETE ExternalTaskEntity[730f224a-e131-11ea-a9aa-e6785f66904b]
UPDATE_BULK updateExternalTaskSuspensionStateByParameters org.camunda.bpm.engine.impl.db.ListQueryParameterObject@1f13c66b
UPDATE_BULK updateJobSuspensionStateByParameters org.camunda.bpm.engine.impl.db.ListQueryParameterObject@f8084f5
DELETE TimerEntity[72bfc992-e131-11ea-a9aa-e6785f66904b]
DELETE TimerEntity[730f2248-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc6e-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc6f-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc70-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc71-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc72-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[72b0fc73-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302c60e-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302c610-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302c612-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302ed25-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302ed27-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302ed29-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7302ed2b-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[730ead11-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[7318bf3d-e131-11ea-a9aa-e6785f66904b]
DELETE VariableInstanceEntity[731a45df-e131-11ea-a9aa-e6785f66904b]
DELETE ByteArrayEntity[730ead10-e131-11ea-a9aa-e6785f66904b]
UPDATE_BULK updateTaskSuspensionStateByParameters org.camunda.bpm.engine.impl.db.ListQueryParameterObject@cf82191
DELETE ExecutionEntity[730ed426-e131-11ea-a9aa-e6785f66904b]
DELETE ExecutionEntity[72bf5460-e131-11ea-a9aa-e6785f66904b]
DELETE ExecutionEntity[72b0fc6d-e131-11ea-a9aa-e6785f66904b]
UPDATE_BULK updateExecutionSuspensionStateByParameters org.camunda.bpm.engine.impl.db.ListQueryParameterObject@642d5bcb
UPDATE HistoricActivityInstanceEventEntity[730ed427-e131-11ea-a9aa-e6785f66904b]
UPDATE HistoricActivityInstanceEventEntity[SubProcess_0en3ea5:72bf7b71-e131-11ea-a9aa-e6785f66904b]
UPDATE HistoricProcessInstanceEventEntity[72b0fc6d-e131-11ea-a9aa-e6785f66904b]
UPDATE HistoricProcessInstanceEventEntity[734d63d1-e131-11ea-a9aa-e6785f66904b]
DELETE_BULK deleteAuthorizationsForResourceId {resourceId=72b0fc6d-e131-11ea-a9aa-e6785f66904b, resourceType=8}
]at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationException(EnginePersistenceLogger.java:131)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:355)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:322)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:294)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:263)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:186)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:119)
at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:46)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:44)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
at org.camunda.bpm.engine.impl.ExternalTaskServiceImpl.complete(ExternalTaskServiceImpl.java:56)
at org.camunda.bpm.engine.impl.ExternalTaskServiceImpl.complete(ExternalTaskServiceImpl.java:48)

Please help

Is there any support on this issue? I’m receiving a similar deadlock error.