I need to create a workflow with several parallel processes that should run asynchronously at a certain point, but the workflow should wait for the results before proceeding.
If the process runs without setting async=before or async=after, the workflow runs sequentially. However, when is set async=before for Service Tasks 1, 2A, 2B, and 3, the process immediately goes to the end without waiting for any results.
In general i try to receive the next process:
first get the results from Service Tasks 2A and 2B running in parallel and asynchronously.
Then, get the result from Service Task 3.
During this time, Service Task 1 should also run asynchronously.
Only after receiving all results, the workflow should proceed to “Analyse results of previous steps.”
How should I correctly set up async to solve this problem?
As far as i understood, asynch is referring to save points to which the trainsaction gets rolled back in case it fails. Either a new transaction can be started at the beginning or the end of a task (asynch before and after).
Additionally no parallel gateways are needed when there is only one incoming flow. The process will wait at a parallel gateway until there is one token at every incoming flow.
Also the last exclusive gateway is not necessary. An exclusive gateway sets a condition for every outgoing flow. But if there is only one flow than this one will always get triggered.
I tried to adjust ur process model accordingly and the following structure should describe it. Correct me if it does not apply to your planned process flow.
In this approach the process will wait in Gateway 1 until 2 tokens arrive. One from the flow of Service task 2A + Analyse process 2A.1 + Analyse process 2A.2 and from the other flow of Service Task 2B + Analyse process 2B.
Here the analyse 2A.1 and the analyse 2A.2 should be executed in parallel. Service Task2A finished and continues the process flow in both outgoing flows.
After their completion the process then continues to service task 3
Meanwhile service task 1 was completed.
For both of them is waited in Gateway 2.
The parallel behaviour works depending on your implementation.
If each service task has its own topic and its own worker that subscribes to them the behaviour should be that Service Task 1,2A,2B get created in the enging once the process gets started. Then all of them get process in parallel by their workers. Which one finishes first depends on how long each one of them individually takes and when the workers poll them.
Maybe u can provide more on the topic structure and the structure of your workers to make the problem more clear.
I hope some this helped and made it easier to follow for others.
I updated workflow and result is same. Without using async that gateway 1 wait until Service task 2A + Service Task 2B (with analyses) will be completed, as gateway 2 wait service task 1 and service task 3.
But the problem is still here: regarding listeners I see that at least service task 2A, 2B work step by step, first one of the task started and when it comes to gateway1 start another one. Same with Service task 1 and service task 3.
This process work too long and I want to reduce a time due to real parallel execution (at the same time)
If use async on task 1, 2A, 2B → as earlier the process immediately goes to the end without waiting for any results from any tasks. I also mentioned that after Analyse process 2A.1 receive exception ENGINE-14006 Exception while executing job: OptimisticLockingException. To see the full stacktrace set logging level to DEBUG.
and task 2B execute 2 times and because of that task 3 and the following process also 2 times
Service Task 1 has his own Java class implementation process.
2A, 2B, 3 use the same Java class implementation process.
To be more precise Tasks 2A and 2B are subprocess.
I recreated your example.
To be clear your problem is that the parallel execution is not working properly. I managed to achieve that with your architecture without setting any asynch attributes.
Below you can see a screenshot of the camunda cockpit of one running process instance.
As you can see service tasks1 was processed completly (therefore one token is at gateway2)
Also service-task 2 is processed and service task 3 is currently beeing processed.
I replaced the DMN tasks with user tasks to not have to implement DMN logic and to show that these states are reached.
Your architecture is correct and works for me.
Im using the external task client implementation.
And i have one external task handler for every service task (topic).
The engine creates topics to which external workers can connect to. This is the recommended approach ( Invoking services from a Camunda 7 process | Camunda 8 Docs )
Maybe your issues occur because you use java delegates or another implementation approach.
Using external task solves the issue cause the engine just creates topics for every service task. Those topics can be subscribed to in parallel.
If you only use one worker then it can only be handled sequentially since one worker can only subscribe to one topic at a time.
I mainly use the external task implementation approach. I recommend to use it since its best practice if theres no reason against it.
Maybe more information about your implementation method could make the problem more clear to me. Is it possible for your use case to use the external task implementation method?
