We uppdated our Jboss wildfly 14 shared Camunda engine from 7.11 to 7.14.
We have multiple .war applications connecting to the shared engine.
Now we get exceptions on startup
ERROR [org.camunda.bpm.engine.jobexecutor] (pool-24-thread-1) ENGINE-14019 Exception during job acquisition An exception occurred in the persistence layer. Please check the server logs for a detailed message and the entire exception stack trace.: org.camunda.bpm.engine.ProcessEngineException: An exception occurred in the persistence layer. Please check the server logs for a detailed message and the entire exception stack trace.
at org.camunda.bpm.engine.impl.util.ExceptionUtil.wrapPersistenceException(ExceptionUtil.java:263)
at org.camunda.bpm.engine.impl.util.ExceptionUtil.doWithExceptionWrapper(ExceptionUtil.java:257)
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeSelectList(DbSqlSession.java:107)
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.selectList(DbSqlSession.java:99)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.selectListWithRawParameter(DbEntityManager.java:182)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.selectList(DbEntityManager.java:174)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.selectList(DbEntityManager.java:170)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.selectList(DbEntityManager.java:159)
at org.camunda.bpm.engine.impl.persistence.entity.JobManager.findNextJobsToExecute(JobManager.java:222)
at org.camunda.bpm.engine.impl.cmd.AcquireJobsCmd.execute(AcquireJobsCmd.java:66)
at org.camunda.bpm.engine.impl.cmd.AcquireJobsCmd.execute(AcquireJobsCmd.java:44)
at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110)
at org.camunda.bpm.engine.impl.interceptor.JtaTransactionInterceptor.execute(JtaTransactionInterceptor.java:61)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
at org.camunda.bpm.engine.impl.interceptor.CommandCounterInterceptor.execute(CommandCounterInterceptor.java:35)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
at org.camunda.bpm.engine.impl.jobexecutor.SequentialJobAcquisitionRunnable.acquireJobs(SequentialJobAcquisitionRunnable.java:158)
at org.camunda.bpm.engine.impl.jobexecutor.SequentialJobAcquisitionRunnable.run(SequentialJobAcquisitionRunnable.java:76)
at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:29)
at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:789)
at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:44)
at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:809)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)Caused by: org.apache.ibatis.exceptions.PersistenceException:Error querying database. Cause: java.sql.SQLException: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@5fc5ddc0[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 connection handles=0 lastReturned=1615537228526 lastValidated=1615537193845 lastCheckedOut=1615537228784 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@1bd4d76d mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@cf0451c[pool=camunda-processengine-ds] xaResource=XAResourceWrapperImpl@5ba47836[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 pad=false overrideRmValue=false productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Advanced Analyticsand Real Application Testing options jndiName=java:jboss/datasources/ProcessEngine] txSync=null] The error may exist in org/camunda/bpm/engine/impl/mapping/entity/Job.xml ### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.selectNextJobsToExecute ### The error occurred while executing a query ### Cause: java.sql.SQLException: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@5fc5ddc0[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 connection handles=0 lastReturned=1615537228526 lastValidated=1615537193845 lastCheckedOut=1615537228784 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@1bd4d76d mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@cf0451c[pool=camunda-processengine-ds] xaResource=XAResourceWrapperImpl@5ba47836[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 pad=false overrideRmValue=false productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics and Real Application Testing options jndiName=java:jboss/datasources/ProcessEngine] txSync=null]
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:149)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140)
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.lambda$executeSelectList$1(DbSqlSession.java:107)
at org.camunda.bpm.engine.impl.util.ExceptionUtil.doWithExceptionWrapper(ExceptionUtil.java:254)
... 23 more Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@5fc5ddc0[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 connection handles=0 lastReturned=1615537228526 lastValidated=1615537193845 lastCheckedOut=1615537228784 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@1bd4d76d mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@cf0451c[pool=camunda-processengine-ds] xaResource=XAResourceWrapperImpl@5ba47836[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 pad=false overrideRmValue=false productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics and Real Application Testing options jndiName=java:jboss/datasources/ProcessEngine] txSync=null]
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)
at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
at org.apache.ibatis.transaction.managed.ManagedTransaction.openConnection(ManagedTransaction.java:89)
at org.apache.ibatis.transaction.managed.ManagedTransaction.getConnection(ManagedTransaction.java:60)
at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:336)
at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:86)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:83)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147)
... 26 more Caused by: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@5fc5ddc0[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 connection handles=0 lastReturned=1615537228526 lastValidated=1615537193845 lastCheckedOut=1615537228784 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@1bd4d76d mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@cf0451c[pool=camunda-processengine-ds] xaResource=XAResourceWrapperImpl@5ba47836[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@5bf3ef60 pad=false overrideRmValue=false productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics and Real Application Testing options jndiName=java:jboss/datasources/ProcessEngine] txSync=null]
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:1055)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:792)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
... 37 more Caused by: javax.resource.ResourceException: IJ000461: Could not enlist in transaction on entering meta-aware object
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:571)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:977)
... 39 more Caused by: javax.transaction.SystemException: Error enlisting resource in transaction=Local transaction (delegate=TransactionImple < ac, BasicAction: 0:ffffac14acb6:-664a556:604b240f:1e status: ActionStatus.ABORT_ONLY >, owner=Local transaction context for provider JBoss JTA transaction provider)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.checkEnlisted(TxConnectionListener.java:957)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:394)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:564)
... 40 more
Caused by: java.lang.Throwable: Failed to enlist
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.enlist(TxConnectionListener.java:1001)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:379)
… 41 more
We have tried to do the update on several test servers and all get the same exceptions.