HowTo Implement async req/response with external System


i’m struggeling with a very simple scenario and could use some advice.
I’d like to learn what the rock-solid textbook solution for this case would be.

My Camunda Workflow is supposed to call an external service (using a REST call) and that external service is responding synchronously using the REST interface of the Camunda engine (or another one as indirection if that will become necessary)

How is that supposed to be modeled so that it works under all circumstances (especially timings)?

At first glance it looks very simple

  • a service task implemented by a Java delegate calls the external service
  • following this service task the next step in the process is an intermediate message catching event which is supposed to receive the external services response

During testing we ran into the case that the external service responds quickly enough so that the response-message arrives even before the message sending-routine returns. In that case the response message could not be correlated.

I’ve tried out a number of variations

  • responding by setting variables instead of sending a message as response (processing in a conditional intermediate event)
  • sending the message from a sub-process that has a boundary event (reacting to either a message or a variable change depending on the communication variant)
  • sending the request from the main process, catching the respnse in a non-interrupting message-event sub-process that then communicates the received message via variable-setting to the main process
  • and with a massively bad conscience, I’ve tinkered with introducing delays - which I do not really consider a valid solution

After all these attempts the problem still persists. I keep running into situations where either

  • a message subscription does not yet exists
  • I do not even find the Execution which is supposed to to process a variable-change
  • I provoke an OptimisticLockingException - e.g. by manupulating the main processes variables while it has not yet returned from a task

I’ve done all my experiments in unit tests having the Camunda engine embedded in Spring Boot. Also I’ve payed attention to isolating the players Camunda-Engine vs external service into their own transaction in order to emulate the real-deal. I didn’t implement the forth and back communication with the external service as Rest calls but went with Spring beans instead.

Suggestions? How does that obvious and ultra-simple solution look I couldn’t come up with? :wink:

Cheers, Micha