Flow waiting to move to next activity after a particular decision gate

Camunda 7 Topics Discussion & Questions
1

Decision gateway is highlighted in the picture. Flow waits anytime between 3 to 6 seconds or more after this particular decision gateway (highlighted in yellow) to go to next activity. Because of this, message receive task (circled in red) which should accept new incoming messages is not getting activated again to absorb new incoming messages to resume process flow on new messages and thus causing failures. We have seen same behavior on different environments for same decision gate.

image
image1750×535 79.8 KB

Thanks for your help.

2

Hi @manojkumar,

can you please share your complete BPMN process?

3

ChatbotFlowManager.bpmn (68.5 KB)

Here it is

4

Digging little more found out that time gap after decision gateway happens only if next activity is Asynchronous. Job executor is taking 3 to 4 secs for other Asynchronous activities as well to be picked up. So time gap issue is not with decision gateway but is with Job executor picking up jobs that are waiting.

There is no much load on the environment with two pods and one tester testing it. Have this properties now

camunda.bpm.job-execution.core-pool-size=10
camunda.bpm.job-execution.maxPoolSize=20

We are on 7.8.1 camunda.

Anything missing here.

Thanks and appreciate your help and feedback.

5

Can anyone please help with steps to check if job executor performing without issues and to optimize job executor performance? Thanks :slight_smile:

6

Hi @manojkumar,

usually, the job executor is triggered when a new asynchronous job is created. Can you please check if the job executor is busy. Turn on the logger of the job executor to get more insides.

However, if it is important that the message is received by the receive task then you should change your model so that the message can always be received. For example, using a parallel flow or an event sub-process.

Besides that, you should think about model your process again. It looks like a maze and it is really hard to understand :dizzy_face: Link events should be used carefully.

Best regards,
Philipp

1 Like
7

Thanks Philipp for your inputs. We are planning to change the design.

8

We enabled job executor logs and noticed that it is taking 4 seconds or more for any async task to be picked up once the flow execution stops before it.

image
image1124×110 8.33 KB

As you see in the screen shot, there is 4 secs gap for “Send bot response” activity to be picked up. I am also attaching logs which show that this activity is added to job executor after 4 seconds of completion of exclusiveGateway before it. We added these job-execution properties and were under the impression that job executor will almost immediately pick up new jobs. What should be be done to reduce the gap and also how can we make sure job-execution properties are really getting picked up by camunda engine?

camunda.bpm.job-execution.core-pool-size=10
camunda.bpm.job-execution.max-pool-size=20
camunda.bpm.job-execution.wait-time-in-millis=2000
camunda.bpm.max-wait=5000

Appreciate your help help :slight_smile:

{“logType”:“DEBUG”,“logLevel”:“DEBUG”,“logTimestamp”:“2018-08-21T00:30:20.751Z”,“logger”:“org.camunda.bpm.engine.jobexecutor”,“label”:“ENGINE-14012 Job acquisition thread woke up”,“runtime”:{“hostName”:“hltv0682”,“ip”:“10.233.72.63”,“instance”:“rcmacchatbot-1-0-q30a-86bc9ddfdd-m99f8”,“clusterName”:“NONPROD-HYWRCA02-CSI-CST-01”,“namespace”:“com-att-csbobpm-test”,“image”:“dockercentral.it.att.com:5100/com.att.csbobpm.custnetcare.coch.cuco/rcmacchatbot:1.0.53",“platformIdentifier”:“AJSC7_CAMUNDA”,“environment”:“TEST”},“application”:{“deploymentUnitName”:"",“motsApplicationAcronym”:"CS BOBPM”}}
{“logType”:“DEBUG”,“logLevel”:“INFO”,“logTimestamp”:“2018-08-21T00:30:20.755Z”,“logger”:“org.camunda.bpm.engine.jobexecutor”,“label”:“ENGINE-14008 Adding new exclusive job to job executor context. Job Id=‘603deeb9-a4d9-11e8-8686-0a580ae9483f’”,“runtime”:{“hostName”:“hltv0682”,“ip”:“10.233.72.63”,“instance”:“rcmacchatbot-1-0-q30a-86bc9ddfdd-m99f8”,“clusterName”:“NONPROD-HYWRCA02-CSI-CST-01”,“namespace”:“com-att-csbobpm-test”,“image”:“dockercentral.it.att.com:5100/com.att.csbobpm.custnetcare.coch.cuco/rcmacchatbot:1.0.53",“platformIdentifier”:“AJSC7_CAMUNDA”,“environment”:“TEST”},“application”:{“deploymentUnitName”:"",“motsApplicationAcronym”:"CS BOBPM”}}
{“logType”:“DEBUG”,“logLevel”:“DEBUG”,“logTimestamp”:“2018-08-21T00:30:20.808Z”,“logger”:“org.camunda.bpm.engine.jobexecutor”,“label”:“ENGINE-14022 Acquired 0 jobs for process engine ‘default’: []”,“runtime”:{“hostName”:“hltv0682”,“ip”:“10.233.72.63”,“instance”:“rcmacchatbot-1-0-q30a-86bc9ddfdd-m99f8”,“clusterName”:“NONPROD-HYWRCA02-CSI-CST-01”,“namespace”:“com-att-csbobpm-test”,“image”:“dockercentral.it.att.com:5100/com.att.csbobpm.custnetcare.coch.cuco/rcmacchatbot:1.0.53",“platformIdentifier”:“AJSC7_CAMUNDA”,“environment”:“TEST”},“application”:{“deploymentUnitName”:"",“motsApplicationAcronym”:"CS BOBPM”}}
{“logType”:“DEBUG”,“logLevel”:“DEBUG”,“logTimestamp”:“2018-08-21T00:30:20.808Z”,“logger”:“org.camunda.bpm.engine.jobexecutor”,“label”:“ENGINE-14011 Job acquisition thread sleeping for 9943 millis”,“runtime”:{“hostName”:“hltv0682”,“ip”:“10.233.72.63”,“instance”:“rcmacchatbot-1-0-q30a-86bc9ddfdd-m99f8”,“clusterName”:“NONPROD-HYWRCA02-CSI-CST-01”,“namespace”:“com-att-csbobpm-test”,“image”:“dockercentral.it.att.com:5100/com.att.csbobpm.custnetcare.coch.cuco/rcmacchatbot:1.0.53",“platformIdentifier”:“AJSC7_CAMUNDA”,“environment”:“TEST”},“application”:{“deploymentUnitName”:"",“motsApplicationAcronym”:"CS BOBPM”}}

9

This

should be camunda.bpm.job-execution.max-wait=5000. If the property is set then the wait time should be not more than 5 seconds.

Can you add more logs?
I don’t see the log statement when the job is executed.

10

Corrected the parameter. Job executor is now performing well and under 5 seconds in most cases :slight_smile: