We use many “service tasks” in our BPMN model and set a status for each service task. We write this status in a global variable, which we call processStatus.
To determine the current status, we use the “search” API call from Operate.
The ProcessInstanceKey and the variable name are specified here.
However, we found that the search result is not returned in real time. In some situations, the query for our status variable takes two or more seconds.
We use the Zeebe client in Java and access the Camunda Cloud via SaaS.
Does anyone know a way to determine the content of a variable from a process in real time in Camunda 8 or is this not possible in real time?
Hi @SMueller001, welcome to the forums! I do not believe this is possible in true real time, due to the distributed way Zeebe works. The data is exported to Elasticsearch, which is then queried by the other services like Operate. This architecture documentation may help also.
What is your use case for needing to immediate real-time variable values? How are you using the processStatus in your application? Depending on why you are tracking the status that way, and what your ultimate goal is, there may be other ways the achieve the same result.
Hello @nathan.loding ,
Thank you for your answer and help.
I have added a small example in the attachment to make things clearer.
For each service task, we write a status value to a variable called “processStatus”. For example, READING_VALUE_COLLECTION, READY_FOR_BILLING or PRELIMINARY_BILLING and so on. The variables are always set within the service task for the output in the BPMN model.
We have a Zeebe Java client which offers various Rest-Endpoints, for example to read the current status from the active instance. This status endpoint is then retrieved by another external service, for example, which then retrieves the status every 500 milliseconds in order to be able to react to it depending on the status.
As already mentioned, we need a way to retrieve such a status almost in real time. If this is basically not possible with the Camunda Cloud, we would have to manage it ourselves in a database or are there faster ways to read a variable from the Camunda process.
Hi @SMueller001 - I’m not sure I understand the case for it needing to be real time. Why is it important for your application to monitor each process to the millisecond level? What benefit/need is there for updates multiple times per second?
In many ways, the benefit of using a process orchestration engine like Camunda is that your application no longer needs to worry about the process itself. Having your application poll the running processes multiple times per second feels like it’s going against the intent of using the engine. Your application shouldn’t need to query the running process to figure out what to do next (“be able to react to it”); but rather the process should be able to manage the state and know what happens next.
In our process, we use some “MESSAGE INTERMEDIATE CATCH EVENT”, which are triggered externally by a third-party application. In order to find out whether our process is already at this message event, this external application constantly queries the status every 500ms seconds. This is not directly in real time, but almost, as this status is queried approx. 1 to twice per second.
As already mentioned, we set a global variable with the name “processStatus” and write a new status to this variable after each service task. To determine the current value of this variable, we use the “search” API call from Operate.
@SMueller001 - it sounds like the process isn’t waiting for an external application to complete a task, but rather the external application is waiting for the process to reach a certain task. If the process was waiting, the real-time queries don’t matter, and that’s the typical design.
Does the message contain data that is needed for the next step(s) in the process? I think a better approach would be to have a service task or Connector make a call out to the external application to fetch whatever is needed. That way you aren’t hammering any APIs to check for data, and are instead reacting to the process itself.
You said it’s a third party system; do you have any control over the system? There’s obviously a lot of context not shared here, but I don’t think this is a very sound solution. I’ve shared your question with some of our BPMN experts to get their take too, though; I don’t pretend to be a BPMN expert (yet! )
From what I recall, in the C7 world, you had to make sure that the engine was ready to receive the message before sending the message, or you would get an error.
In C8, you send the message, along with how long the message is valid for. If the engine is already waiting for the message, it will use it immediately. If the engine is not yet ready for the message, the message gets queued for as long as the lifespan of the “valid”, if the engine gets to the point where it’s ready for the message, it will take it from the queue.
The only piece of this puzzle that’s missing then is: What does the external system do if the engine isnt ready within the timeframe?
But… if you set the “message validity” (TTL) long enough, then the engine will become ready.
So… Short answer is… you don’t need to poll the engine to see if it’s ready for the message anymore.
)