Disabling jdbcBatchProcessing?

Galen_Hollins · September 18, 2019, 9:29pm

Hi,

If we have jdbcBatchProcessing set to false like this:

 <property name="processEngineName" value="default" />
      <property name="dataSource" ref="bpmDataSource" />
      <property name="transactionManager" ref="transactionManager" />
      <property name="databaseSchemaUpdate" value="true" />
      <property name="jobExecutorActivate" value="true" />
      <property name="history" value="full" />
      <property name="authorizationEnabled" value="true" />
      <property name="jobExecutorDeploymentAware" value="true" />
      <property name="historyCleanupBatchWindowStartTime" value="00:00" />
      <property name="historyCleanupBatchWindowEndTime" value="06:00" />
      <property name="jdbcBatchProcessing" value="false" />

…
would we still expect batch statements like this?

ENGINE-14006 Exception while executing job b2540495-da50-11e9-9655-0242b5bbc4c5: 
org.camunda.bpm.engine.ProcessEngineException: ENGINE-03004 Exception while executing Database Operation 'DELETE MessageEntity[b2540495-da50-11e9-9655-0242b5bbc4c5]' with message '
### Error updating database.  Cause: java.sql.SQLTransactionRollbackException: (conn=29504152) Deadlock found when trying to get lock; try restarting transaction
### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.deleteMessage-Inline
### The error occurred while setting parameters
### SQL: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?
### Cause: java.sql.SQLTransactionRollbackException: (conn=29504152) Deadlock found when trying to get lock; try restarting transaction'. Flush summary: 
 [
  INSERT HistoricProcessInstanceEventEntity[7a622891-da51-11e9-9655-0242b5bbc4c5]
  INSERT HistoricActivityInstanceEventEntity[Task_1irwi18:7a618c4b-da51-11e9-9655-0242b5bbc4c5]
  INSERT ExecutionEntity[7a622891-da51-11e9-9655-0242b5bbc4c5]
  INSERT VariableInstanceEntity[7a622892-da51-11e9-9655-0242b5bbc4c5]
  INSERT VariableInstanceEntity[7a622893-da51-11e9-9655-0242b5bbc4c5]
  INSERT VariableInstanceEntity[7a622894-da51-11e9-9655-0242b5bbc4c5]
  INSERT VariableInstanceEntity[7a622895-da51-11e9-9655-0242b5bbc4c5]
  INSERT MessageEntity[7a624fa7-da51-11e9-9655-0242b5bbc4c5]
  DELETE MessageEntity[b2540495-da50-11e9-9655-0242b5bbc4c5]
  DELETE_BULK deleteByteArrayNoRevisionCheck 7a530d2a-da51-11e9-9655-0242b5bbc4c5
  UPDATE ExecutionEntity[b250cfb0-da50-11e9-9655-0242b5bbc4c5]
]
    at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationException(EnginePersistenceLogger.java:130)
    at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:345)

Or am I misinterpreting what a batch is, and what this flag controls?

Also, we are on Camunda 7.10.0. Does this ticket:
https://app.camunda.com/jira/plugins/servlet/mobile#issue/CAM-10126
Address anything related to the above deadlock?

Thanks,
Galen

tasso94 · September 20, 2019, 6:01am

Hi Galen,

the error log you have shared is not related to jdbc batch processing. However, a deadlock was detected and therefore the transaction was rolled back.

Cheers,
Tassilo

Galen_Hollins · September 20, 2019, 6:02am

Thanks Tassilo,

Any idea if upgrading to 7.12 (and getting CAM-10126) would help with the deadlock?

Thanks,
Galen

tasso94 · September 20, 2019, 6:06am

Hi Galen,

CAM-10126 is unrelated to your deadlock situation.

Have you set the transaction isolation level of your database to READ_COMMITTED?

Cheers,
Tassilo

Galen_Hollins · September 20, 2019, 6:09am

Hi Tassilo,

Yes, we are using MariaDb 10.2,
and

SELECT @@GLOBAL.tx_isolation, @@tx_isolation, @@session.tx_isolation; 

@@GLOBAL.tx_isolation    @@tx_isolation    @@session.tx_isolation
READ-COMMITTED    READ-COMMITTED    READ-COMMITTED

VERSION()
10.2.25-MariaDB

tasso94 · September 20, 2019, 6:13am

Hi Galen,

are all indices according to the SQL create scripts present?

Cheers,
Tassilo

Galen_Hollins · September 20, 2019, 6:15am

Yes, I think so. We haven’t changed them from the camunda default create table scripts.

Here’s an example table:

CREATE TABLE `ACT_RU_JOB` (
  `ID_` varchar(64) COLLATE utf8_bin NOT NULL,
  `REV_` int(11) DEFAULT NULL,
  `TYPE_` varchar(255) COLLATE utf8_bin NOT NULL,
  `LOCK_EXP_TIME_` timestamp(3) NULL DEFAULT NULL,
  `LOCK_OWNER_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `EXCLUSIVE_` tinyint(1) DEFAULT NULL,
  `EXECUTION_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `PROCESS_INSTANCE_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `PROCESS_DEF_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `PROCESS_DEF_KEY_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `RETRIES_` int(11) DEFAULT NULL,
  `EXCEPTION_STACK_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `EXCEPTION_MSG_` varchar(4000) COLLATE utf8_bin DEFAULT NULL,
  `DUEDATE_` timestamp(3) NULL DEFAULT NULL,
  `REPEAT_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `HANDLER_TYPE_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `HANDLER_CFG_` varchar(4000) COLLATE utf8_bin DEFAULT NULL,
  `DEPLOYMENT_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `SUSPENSION_STATE_` int(11) NOT NULL DEFAULT 1,
  `JOB_DEF_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `PRIORITY_` bigint(20) NOT NULL DEFAULT 0,
  `SEQUENCE_COUNTER_` bigint(20) DEFAULT NULL,
  `TENANT_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `CREATE_TIME_` datetime(3) DEFAULT NULL,
  PRIMARY KEY (`ID_`),
  KEY `ACT_IDX_JOB_EXECUTION_ID` (`EXECUTION_ID_`),
  KEY `ACT_IDX_JOB_HANDLER` (`HANDLER_TYPE_`(100),`HANDLER_CFG_`(155)),
  KEY `ACT_IDX_JOB_PROCINST` (`PROCESS_INSTANCE_ID_`),
  KEY `ACT_IDX_JOB_TENANT_ID` (`TENANT_ID_`),
  KEY `ACT_IDX_JOB_JOB_DEF_ID` (`JOB_DEF_ID_`),
  KEY `ACT_FK_JOB_EXCEPTION` (`EXCEPTION_STACK_ID_`),
  KEY `ACT_IDX_JOB_HANDLER_TYPE` (`HANDLER_TYPE_`),
  CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

tasso94 · September 20, 2019, 6:20am

Hi Galen,

to find out if it helps to update to a newer minor version, I would recommend you to search through our issue tracker to spot deadlock issues related to the deletion of jobs for 7.10.0 that have been fixed.

Cheers,
Tassilo

Galen_Hollins · September 20, 2019, 4:11pm

Hi Tassilo,

Thanks. I’ve found a few that may be related. For example:

I’m not sure whether they may help, but I guess it’s worth a try to upgrade

Thanks,
Galen

system · January 30, 2024, 11:56am