Camunda got stuck in timer

Three of the requet got stuck in the timer. After that, none of the incoming request pass this stage. I have pasted the entries from act_ru_task table. can you give some pointers to debug this issue?

59385ae0-b229-11ea-8c0c-de292a120127 2696 ever-living NaT true 3 2020-08-17T13:22:35.721000 0 history-cleanup {“countEmptyRuns”:8,“immediatelyDue”:false,“minuteFrom”:0,“minuteTo”:59} 1 0 1 2020-06-19T12:35:24.849000
6ee65a32-e08a-11ea-82f4-3ed6360a5612 1 timer NaT true 6ee63320-e08a-11ea-82f4-3ed6360a5612 60f0492e-e08a-11ea-82f4-3ed6360a5612 ocrActivityId:1:631fbb0d-e084-11ea-873c-3ed6360a5612 ocrActivityId 3 2020-08-17T13:06:18.892000 0 timer-intermediate-transition IntermediateThrowEvent_0eg39yp 6314495b-e084-11ea-873c-3ed6360a5612 1 631fbb0e-e084-11ea-873c-3ed6360a5612 0 1 nexus 2020-08-17T13:06:15.892000
dc02535f-e08a-11ea-82f4-3ed6360a5612 1 timer NaT true dc022c4d-e08a-11ea-82f4-3ed6360a5612 cfe23adb-e08a-11ea-82f4-3ed6360a5612 ocrActivityId:1:631fbb0d-e084-11ea-873c-3ed6360a5612 ocrActivityId 3 2020-08-17T13:09:21.947000 0 timer-intermediate-transition IntermediateThrowEvent_0eg39yp 6314495b-e084-11ea-873c-3ed6360a5612 1 631fbb0e-e084-11ea-873c-3ed6360a5612 0 1 nexus 2020-08-17T13:09:18.947000
283235fd-e086-11ea-873c-3ed6360a5612 130 timer 2020-08-17T12:41:30.944000 cc69ec6b-9a05-4eaf-a450-f3d37879d695 true 28320eeb-e086-11ea-873c-3ed6360a5612 115899ce-e086-11ea-873c-3ed6360a5612 livenessActivityId:1:6301f9d9-e084-11ea-873c-3ed6360a5612 livenessActivityId 3 2bc1c56c-e086-11ea-873c-3ed6360a5612 ENGINE-03005 Execution of ‘DELETE VariableInstanceEntity[1ffb8cab-e086-11ea-873c-3ed6360a5612]’ failed. Entity was updated by another transaction concurrently. 2020-08-17T12:35:42.284000 0 timer-intermediate-transition IntermediateCatchEvent_05ddnnp 63002517-e084-11ea-873c-3ed6360a5612 1 6301f9da-e084-11ea-873c-3ed6360a5612 0 65 nexus 2020-08-17T12:35:39.284000
d2c64555-e086-11ea-82f4-3ed6360a5612 1 timer NaT true d2c47093-e086-11ea-82f4-3ed6360a5612 c8d12321-e086-11ea-82f4-3ed6360a5612 ocrActivityId:1:631fbb0d-e084-11ea-873c-3ed6360a5612 ocrActivityId 3 2020-08-17T12:40:28.467000 0 timer-intermediate-transition IntermediateThrowEvent_0eg39yp 6314495b-e084-11ea-873c-3ed6360a5612 1 631fbb0e-e084-11ea-873c-3ed6360a5612 0 1 nexus 2020-08-17T12:40:25.467000

Found an interesting article, its says to increase jobExecutor threadpool size.

So trying with the jobexecutor configuration mentioned in here.

By default

maxJobsPerAcquisition = 5
maxPoolSize = 10

which is very low. Wondering why camunda choose to put a such a low value for jobExection.

Another factor, especially when only running single test processes, is the maxWait property which is 60s per default.

Short timers might look like stuck because it can take up to a minute until they progress, so it might also help to lower this value to ~5s. Of course you should keep in mind that this might increase the idle load, which might be bad on a prod system.

1 Like