Hi Community.
We are facing an error that is raised every time we run multiple workers that consume from the same topic (DummyTopic).
ERROR org.camunda.bpm.client - TASK/CLIENT-03001 Exception while fetch and lock task.
Below is the test that we run:
The wrokflow:
The workflow is a simple workflow of 5 tasks (exetrnal tasks) that have the same topic.
The Extnal task worker:
The worker is a simple java application that uses the “camunda-external-task-client” library to subscribe and consume requests from the Camunda engine (We followed the example mentioned on the documentation camunda-external-task-client-java).
The worker handler is actually not doing any treatment. it only terminates the task:
externalTaskService.complete(externalTask);
The worker configuration is the following:
- Lock Duration = 10 Sec
- Asynchronous response (long polling) = 10 Sec
- Max Task = 1
- disable Backoff Strategy = true
- Topic =“DummyTopic”
This test is actually part of a performance test series that we are running to prove system performance and scalability.
The Worker can consume messages from any task as they all have the same topic name.
We start a big number of process instances (exp: 2000) then we start spinning workers to see how it impact performance.
We noticed that as long as one worker is running no errors are raised. But from the moment we start running more instances from the worker the bellow error start appearing on the worker log:
16:52:43.882 [TopicSubscriptionManager] ERROR org.camunda.bpm.client - TASK/CLIENT-03001 Exception while fetch and lock task.
org.camunda.bpm.client.impl.EngineClientException: TASK/CLIENT-02001 Request 'POST http://localhost:8080/engine-rest/external-task/fetchAndLock HTTP/1.1' returned error: status code '500' - message: status code: 500, reason phrase: {"type":"NullValueException","message":"Cannot find execution with id 76615ff4-6555-11ea-abe0-02425b490cf5 for external task 76615ff6-6555-11ea-abe0-02425b490cf5: execution is null"}
at org.camunda.bpm.client.impl.EngineClientLogger.exceptionWhileReceivingResponse(EngineClientLogger.java:30)
....
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.client.HttpResponseException: status code: 500, reason phrase: {"type":"NullValueException","message":"Cannot find execution with id 76615ff4-6555-11ea-abe0-02425b490cf5 for external task 76615ff6-6555-11ea-abe0-02425b490cf5: execution is null"}
Camunda engine log is showing this error:
15-Mar-2020 16:52:43.869 SEVERE [http-nio-8080-exec-1] org.camunda.commons.logging.BaseLogger.logError ENGINE-16004 Exception while closing command context: Cannot find execution with id 76615ff4-6555-11ea-abe0-02425b490cf5 for external task 76615ff6-6555-11ea-abe0-02425b490cf5: execution is null
org.camunda.bpm.engine.exception.NullValueException: Cannot find execution with id 76615ff4-6555-11ea-abe0-02425b490cf5 for external task 76615ff6-6555-11ea-abe0-02425b490cf5: execution is null
at sun.reflect.GeneratedConstructorAccessor86.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
...
We don’t understand why this error is raised. By right the external task client should only address tasks that are non-locked and not under processing by other external clients.
We noticed that at the end all tasks are consumed without any duplication. Still the error is occurring very frequently on logs and we suspect that on every failed fetch and lock because this error, the engine is querying the DB but client does not get any task. which mean that these DB queries are only loading the database and this may impact performance.
Can any one help explaining this error?