OptimisticLockingException when using Task Listeners that start Event Subprocess with External Task

elelbaba · March 16, 2023, 10:03am

What i want to achieve:
On completion of a user task, a service task (external task) is invoked.

My solution looks like this:

User Task 2 has the following task listener:

<camunda:taskListener event="complete">
          <camunda:script scriptFormat="javascript">task.getProcessEngineServices().getRuntimeService().correlateMessage("update", businessKey, new Map([["data","someData"]]));</camunda:script>
</camunda:taskListener>

So the event subprocess is started on completion of User Task 2.

The Problem:
To test, i use the REST API to get and complete user task 2 over and over:

### Get Task 2
GET localhost:8080/engine-rest/task?processInstanceId={{instanceId}}

### Complete Task2
POST localhost:8080/engine-rest/task/{{taskId}}/complete
Content-Type: application/json
{}

At some point, seemingly at random, the completion fails due to this Error:

2023-03-16 10:21:18.376  WARN 12512 --- [nio-8080-exec-1] org.camunda.bpm.engine.rest.exception    : ENGINE-REST-HTTP500 org.camunda.bpm.engine.rest.exception.RestException: Cannot complete task e846ee28-c3db-11ed-9e38-2e0da7043a58: ENGINE-03005 Execution of 'UPDATE ExecutionEntity[e5e04e8b-c3db-11ed-9e38-2e0da7043a58]' failed. Entity was updated by another transaction concurrently.
	at org.camunda.bpm.engine.rest.sub.task.impl.TaskResourceImpl.complete(TaskResourceImpl.java:134)
	at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.camunda.bpm.engine.rest.filter.CacheControlFilter.doFilter(CacheControlFilter.java:45)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.camunda.bpm.engine.rest.filter.EmptyBodyFilter.doFilter(EmptyBodyFilter.java:99)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of 'UPDATE ExecutionEntity[e5e04e8b-c3db-11ed-9e38-2e0da7043a58]' failed. Entity was updated by another transaction concurrently.
	at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.concurrentUpdateDbEntityException(EnginePersistenceLogger.java:141)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.handleConcurrentModification(DbEntityManager.java:413)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:356)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:323)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:295)
	at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:272)
	at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:188)
	at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:119)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.lambda$execute$0(SpringTransactionInterceptor.java:71)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:71)
	at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
	at org.camunda.bpm.engine.impl.interceptor.CommandCounterInterceptor.execute(CommandCounterInterceptor.java:35)
	at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
	at org.camunda.bpm.engine.impl.interceptor.ExceptionCodeInterceptor.execute(ExceptionCodeInterceptor.java:55)
	at org.camunda.bpm.engine.impl.TaskServiceImpl.complete(TaskServiceImpl.java:179)
	at org.camunda.bpm.engine.rest.sub.task.impl.TaskResourceImpl.complete(TaskResourceImpl.java:117)
	... 64 more

StephanHaarmann · March 16, 2023, 10:22am

Hi @elelbaba and welcome to the forum,

The fundamental problem is that you try to change the execution of a process instance from within an event listener that runs in this process instance. I would not recommend this. If the “Some external task” is part of your process, why don’t you add it after the user task?`

This exception is expected: the transaction started by completing the user task won’t complete until the execution listener is complete. At the same time, you’re starting a new transaction via the message. These two transactions are now conflicting.
You can reduce the risk for this error by setting “Asynchronous Before” and “Exclusive” on the start event of the event subprocess.

golup567 · March 16, 2023, 10:25am

This exception comes when you try update same entity(Variable,Task,Execution) by two thread. but your one thread willl finish the task and other one gets this exception.

elelbaba · March 16, 2023, 10:35am

Hi @StephanHaarmann,

thanks for the quick response. The example is simplified quite a bit, in the end i want to invoke an external task for each task event (or at least created, deleted, completed). And possibly add some logic in the event subprocess to use different servicetasks.

Can you explain, why sending the message has to start a new transaction?

StephanHaarmann · March 16, 2023, 10:46am

You’re probably right. It can still be executed in the same thread. Have you tried it without setting variables? Does it occur for all events? There is some conflicting update, i.e., user task and external service task changing the same variable. If this is the case, splitting it into two transactions may solve the issue.
However, I do not recommend this style, but to recommend an alternative, I would need to know more about what you’re trying to achieve.

elelbaba · March 16, 2023, 11:57am

I tried it without setting any variables → same result

For now i have tested create and complete and it occurs in both cases.

Do you have any tips for me on debugging this further to find out what exactly is causing the error? I played with log level DEBUG and i can tell, that the ExecutionEntity that is causing the exception is the top level execution/ the process instance. But i am not sure what is really happening.

I tried something else to challenge my assumptions:
I added a script task before the service task that just waists 5 seconds in a while loop.
I also let the external task implementation just wait for 100 seconds before completing.
What i found is, that the user task is completed after (approximately) 5 seconds implying that the user task complete does wait for the script task but not for the service task to complete. Is this intended?

GotnOGuts · March 16, 2023, 3:47pm

For isolation and testing purposes, could you try moving your event subprocess into its own process, and then use a 'MessageStart" to this other process?
This might help pull the events that are happening in the server apart, allowing you to see exactly what’s happening.

elelbaba · March 16, 2023, 4:11pm

if i understand you correctly, that is what i did to fix the problem temporarily. I got rid of the event subprocess completely and replaced it with another process that has a message start event. This eliminated the Problem (but it generates a ton of process instances).

GotnOGuts · March 16, 2023, 6:16pm

If you merge the events (and DB updates) of those processes together, do you see both the original process and the “External Task” process both being updated at pretty much the same time?

While it introduces a lot of process instances, it is a clearer representation of the business flow.

system · January 31, 2024, 10:43am