I am new to Camunda and in the final stage of implementing this into our software to try it out in production.
I ran into the following issue:
We have some external tasks that we complete through the nodejs client. That works alright. Now this task is completed and moved from Send To AMQP to Waiting for AMQP response. Then when our microservices are finished with the task and published a message back to AMQP that we are listening on with a nodejs worker we want to progress the token to the next Task, but how do we reference it?
We came to a solution that fetches the execution by a variable: /execution/variables=uuid_eq_someuuid&taskDefinitionKey=waitingForResponse
then we got the matching task with it: /task/?executionId=theId&taskDefinitionKey=waitingForResponse
and finally we could complete the task: /task/id/complete
Only problem with this solution is that to get the execution in this way takes 4 seconds, and sometimes more then 15 seconds.
We are to process millions of jobs per day and this amount of time is unacceptable. Is there a faster way to complete these tasks?
PS> I know the AMQP stuff in the middle is not needed for camunda, because the external tasks could also be doing the task right away, but we already have a lot of microservices running, and this solution would prevent us from having to implement Camunda in 4 different languages in god knows how many microservices. And prevent us from being stuck to Camunda. We are still trying it out…
Why is it taking so long to get the task? Are you sure this is for “get” of the task? or this is for “complete” of the task?
Jaeger told me (it is the GET of the execution, not the task), see images:
I also experimented with getting the task directly without the execution first, but with my multi-instance subprocesses that didn’t work as we would get back the task of all instances and I only want to complete one.
It was not happening while testing, because we had only a few tasks in the DB probably. Now I ran 33000 tasks (which is peanuts compared to what we are about to run in production) and it gives these response times, so maybe we can put some indexes on some tables somewhere, just don’t know which fields and tables… ???
Hi @mac_doggie,
when you provide the task Id into the payload of the AMQP message and use it for completion, is the fastest way (from the process instance runtime view) in the moment.
To see how to improve the query time in the database, you should ask your database admin. Perhaps they can recreate the indexes with runstats or a similar command to improve the access time. They can also run an explain on the statement and easily improve the access time in the database, if the first step didn’t help.
I havn’t done it myself but watched the experts doing this, and it was nice to see the time melting from seconds down to milliseconds.
Hope this helps, Ingo
experimented with getting the task directly without the execution first, but with my multi-instance sub-processes that didn’t work as we would get back the task of all instances and I only want to complete the task for one of the iterations.
When I complete my external Task after I have send a message to AMQP, my external Task is finished, and so I do not have a taskID. I do have another variable that I put in the AMQP message that I can use to retrieve the taskID of the newly created follow-up task. That is why I query it by variables. I tried getting the task based on that variable, but in a multi-instance task that didn’t work as expected.
e.g. I have a listing that contains three imageURL, so I have a multi-instance sub-process handling the imageURL. I want to process the images. First step in multi-instance sub-process is external Task. I Fetch & lock, send to AMQP and finish the task. token goes to the next Task in line, which is externally processing through AMQP. AMQ message is read from completion queue and I want to complete the externally processing through AMQP task. I can not just fetch & lock, as this is a specific task and not just the first one waiting, so I do not create an external Task out of this, but a user Task. I need to somehow select the right task so I fetch it based on the uuid variable that I passed to the AMQP message. For some reason when I fetch the task I get back all three instances… and so I can not complete the task for the image I just externally processed as I don’t know which of the three it was. So I selected it by fetching the execution by variable. Now I only got back one execution, so I could select the task that belonged to that one, and complete that task. Only when my database grew that query started to take longer and longer.
I have the feeling that I did over complicate this construction. Why would I need to do three http requests to complete my task, this feels a bit off, so I can spend time trying to make the query faster with indices, but perhaps someone can just tell me, hey you should just do it like this and then you only need one or two http requests… Like maybe I can even complete the task directly when I only have the uuid variable that I stored inside, without fetching the task and or execution first, or maybe I can get the taskID of the new Task directly in the externalTask when I complete the externalTask. Then I can pass that one to my AMQP in stead of selecting it later by my own uuid variable…
Hi
all you need to call is
/external-task/56ef2abf-b152-11e9-b638-eea37b94dd89/complete
with in body
workerId and variables
both external task id and worker Id are available when you listen to task
example curl
curl -X POST \
http://localhost:8080/engine-rest/external-task/56ef2abf-b152-11e9-b638-eea37b94dd89/complete \
-H 'Accept: */*' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Authorization: Basic ZGVtbzpkZW1v' \
-H 'Cache-Control: no-cache' \
-H 'Connection: keep-alive' \
-H 'Content-Length: 270' \
-H 'Content-Type: application/json' \
-H 'Cookie: connect.sid=s%3AJ0K_W5NkChXQfUZmyxeD5o3_f1PqnjSo.9R5gFw0N7cJYosk43Y5ebpYm70hroiZBInjQD8Xhkag; _csrf=NIzG7PLARiE4KoMYu7Xy0NWB; XSRF-TOKEN=11HzMXKd-zhFfxpK4xC0Wl0hnTuu50s8gDAs' \
-H 'Host: localhost:8080' \
-H 'Postman-Token: d4ddef85-6405-4ead-a7ad-39d8e5055eef,b7617a88-abc8-4679-9a46- b71c87b349c0' \
-H 'User-Agent: PostmanRuntime/7.15.2' \
-H 'cache-control: no-cache' \
-d '{
"workerId": "some-random-id",
"variables": {
"token": {
"value": "58910760308816927748255576327306446755611085667338273547756458017978959384094262830129346432171677850002434075517434526858497827",
"type": "String"
}
}
}'
