External task and Waiting for callback

Hello Everyone,

I am running camunda independently and we have a separate service camunda_external_task_listener, created using the library provided by Camunda GitHub - camunda/camunda-external-task-client-js: Implement your BPMN Service Task in NodeJS.

Flow

Approach1

  1. camunda_external_task_listener listens to task details on SendTaskInformation topic.
  2. camunda_external_task_listener passes task details to RabbitMQ for other services (let’s say ServiceB).
  3. camunda_external_task_listener mark complete “SendTaskInformation” task after passing task details successfully to RMQ…
  4. ServiceB sends “TaskDone” Msg to camunda and the flow is completed.

Problem in approach1
As we have already sent message on RMQ for serviceB step4 can occur even before Step3 is completed. i.e. ServiceB sends “TaskDone” msg even before step3 is completed and And hence camunda will throw an error “No process is waiting for TaskDone Msg”.

Approach2

  1. camunda_external_task_listener listens to task details on SendTaskInformation topic.
  2. camunda_external_task_listener mark complete “SendTaskInformation” task.
  3. camunda_external_task_listener passes task details to RabbitMQ for other services (let’s say ServiceB).
  4. ServiceB sends TaskDone Msg to camunda and the flow is completed.

Problem in approach2
To resolve Approach1 issue if we first mark the task complete and then send task information to ServiceB at step3, what if at step3 due to any reason like connection failures is not successfully completed, we won’t be able to mark the task state as it’s already marked completed at Step2.

Please suggest what should be done so that ServiceB sends “taskdone” msg only after external task is completed? I am beginner in camunda community, I think I am doing some silly mistake in designing.

Welcome @AnuragJain!

But this is not the right solution as what if Step3 is not successfully executed due to any reason like connection failures?

If I’m understanding your scenario here, I would expect camunda_external_task_listener to handle any of these errors. You would have the option to complete the external task in error or with incident. That way, you wouldn’t continue through to your end event until you have successfully posted to MQ and successfully completed the external task. At which point, I don’t think you need the message event.

@jgigliotti first of all Thank you so much for replying, I truly appreciate :slight_smile: it.

I have edited my question for more clarity on what I am asking.

The solution you are suggesting only possible in approach1 - that is mark task in error if not able to publish successfully to RMQ. But if I use approach1, I face different problem i.e. ServiceB sends “TaskDone” msg early even before we mark “SendTaskInformation” task completed.

In approach2 - approach1 problem gets solved but this creates different problem that is as I have already marked the task completed and now I am not able to publish RMQ msg then as task is already marked complete it’s not possible to revert the same.

Seeking your input and help :slight_smile:

As a consultant, if I were sitting down to solve this problem, I’d have a bunch of questions about your problem space, e.g. why do you need to have the message indicating completion from ServiceB (and probably follow up questions like, what happens when the service doesn’t complete and that message never arrives?).

Having said that, in an attempt to answer the question as presented, a possible solution may look something like this:

Screen Shot 2020-05-12 at 10.49.28 AM

What I think you’re needing to do is ensure you have successfully completed the external task, but also that you have received the completion message from ServiceB, not necessarily that the events happen in any particular order. But again, I’d still pose the question, how are you handling the scenario where the external task does what it’s supposed to do, but there’s an error in ServiceB and you never receive the message?

Hope that

@jgigliotti

ServiceB is a microservice, it can perform a few types of tasks. Task information comes via Camunda as variables can be changed as per the business logics. Just giving one TaskType example

TaskType1 - send tweet msg <msg_variable> and on <count_varaible> retweet send task status.

  • Task status == success when successfully tweeted and retweet count matches.
  • Task status == failure when there is some network failure
  • No task status after 15 minute = terminate the task.

Your suggestion will solve the problem occurred in Approach1 - but do you think it’s the right approach as TaskStatus is only available when we successfully send TaskInformation on RMQ?

It seems like a plausible approach, but it’s hard to say for certain without knowing more about the various failure modes once the external task attempts to call out to ServiceB. It seems that there may be some overlap where not receiving the TaskStatus message could, or should, be interpreted the same as TaskStatus == failure. Ultimately, you’re going to be the best judge of those various cases; it seems like you’re on the right track.