Service jobs are executed more than once without exceptions thrown

Hi everyone,

I’m facing a weird issue. We’ve seen some multiple executions of a set of service tasks that did not cause any exceptions. I cannot reproduce it and I also cannot grasp the cause of this behaviour.

Stacktrace

2016-09-09 12:55:44,776 DEBUG [be.powerdale.bpm.pop.milestone.NotificationApproved] (pool-7-thread-86) NotificationApproved milestone reached.
2016-09-09 12:55:44,784 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-86) Creating new building in EV platform
2016-09-09 12:55:44,785 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-86) Saving building for company 223
2016-09-09 12:55:44,789 DEBUG [be.powerdale.bpm.integration.evplatform.core.BaseClient] (pool-7-thread-86) {"name":"Home f l","longitude":null,"latitude":null,"type":"HOME","companyId":null,"address":{"city":"c","street":"s","zipcode":"2","streetNumber":"1","country":"BELGIUM"}}
2016-09-09 12:55:44,800 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-86) Creating new building ...
2016-09-09 12:55:44,816 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-86) New Building Id 352

2016-09-09 12:55:44,875 DEBUG [be.powerdale.bpm.pop.milestone.NotificationApproved] (pool-7-thread-87) NotificationApproved milestone reached.
2016-09-09 12:55:44,886 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-87) Creating new building in EV platform
2016-09-09 12:55:44,888 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-87) Saving building for company 223
2016-09-09 12:55:44,889 DEBUG [be.powerdale.bpm.integration.evplatform.core.BaseClient] (pool-7-thread-87) {"name":"Home f l","longitude":null,"latitude":null,"type":"HOME","companyId":null,"address":{"city":"c","street":"s","zipcode":"2","streetNumber":"1","country":"BELGIUM"}}
2016-09-09 12:55:44,901 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-87) Creating new building ...
2016-09-09 12:55:44,914 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-87) New Building Id 353

2016-09-09 12:55:44,955 DEBUG [be.powerdale.bpm.pop.milestone.NotificationApproved] (pool-7-thread-86) NotificationApproved milestone reached.
2016-09-09 12:55:44,969 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-86) Creating new building in EV platform
2016-09-09 12:55:44,970 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-86) Saving building for company 223
2016-09-09 12:55:44,970 DEBUG [be.powerdale.bpm.integration.evplatform.core.BaseClient] (pool-7-thread-86) {"name":"Home f l","longitude":null,"latitude":null,"type":"HOME","companyId":null,"address":{"city":"c","street":"s","zipcode":"2","streetNumber":"1","country":"BELGIUM"}}
2016-09-09 12:55:44,980 DEBUG [be.powerdale.bpm.integration.evplatform.core.EVClient] (pool-7-thread-86) Creating new building ...
2016-09-09 12:55:44,992 INFO  [be.powerdale.bpm.pop.delegate.PersistBuilding] (pool-7-thread-86) New Building Id 354

Model

The service tasks themselves are not marked as asynchronous.

In short; I’ve defined the milestone as asynchronous before as the previous task is a User Task and the submit of the form was taking a long time. I did not give it a RTC as I do not want to retry the service tasks causing them to roll back. I checked the logging far any exceptions, but I could not find any so I do not know what causes the multiple executions.

Any insights would be greatly appreciated!

Hi,

how long does all service tasks take . Maybe it is a problem of job lock.
The default job Lock in 5 minutes. If your service tasks take more time, the lock is released and the job executor will aquire and execute your job again.
Maybe that is the problem?

best regards,

Markus

Hi @nvanbelle,

could you attach your process definition?

Cheers,
Askar

Hello @aakhmerov, here it is;

Kind regards,
Nico

@nvanbelle,

are you always seeing 3 executions only?

No in most cases just one execution. I am not able to reproduce it but it does happen occasionally :thinking:

@nvanbelle, are you sure that your job is not failing? it looks like the transaction is getting rolled back and job is getting executed again. could you increase log level and see if there are any exceptions happening?

1 Like

Hmmmm so yes, I found out that an exception was thrown in the persistUser step which caused the whole lot to execute again and again. (For some reason the error message was not logged by the correct jboss log handler but that is another issue).

So if I want to end the transaction after milestone notification approved and before save new user -so only save company and save building share a transaction-, I just tick the Asynchronous after for milestone + Asynchronous before on save new user and everything will be hunky dory then I assume?

@nvanbelle, yes, sounds good to me.
More info on transactions is available here https://docs.camunda.org/manual/7.5/user-guide/process-engine/transactions-in-processes/

Cheers,
Askar

Thank you very much for your efforts.