I use Camunda as a gateway between frontend and backend. A user starts process by clicking a button in the frontend then camunda makes request to the backend to get some data for form and returns form back to the frontend. But in my case there’s a hundreds of users in a website so every time when users interact with frontend creating process is heavy operation for server. Maybe I have some gaps in my knowledge about Camunda. Please correct me if I’m in wrong way
@MagzhanUnited - can you share your process model? Is that the entirety of your process, or after the form is completed by the user are there more tasks in the process?
If that is the entirety of the process, Camunda probably isn’t the best tool for that (unless you’re planning to expand the process). Camunda is optimized for process orchestration across services, which are often far more complex than just 2-3 steps. If you are only fetching some data then returning a form, it is probably far more performant and efficient to have your backend handle that directly.
Hello, i am facing the same issue. i have to wait about 6 to 10 seconds after starting a process
Hello, i am facing the same issue
I believe there remains some misunderstanding here. Camunda is a process orchestration platform, powered by a distributed, asynchronous workflow engine (Zeebe). It is not a routing engine.
Camunda is designed and optimized for complex and high-volume process automations across multiple internal and external services. To use @MagzhanUnited’s process as an example, that process is far more efficient as a switch statement in your own application. (This is an area where our documentation could definitely use some improvement; I’ve shared these questions with the team, and we’re going to come up with a plan to improve it - if you have further feedback on this aspect of the documentation, please message me, I’d love to hear it!)
I think this blog post by one of our co-founders, Bernd Rücker, helps explain where process orchestration is beneficial and where it is not the right solution:
Camunda probably isn’t the right fit if you want to click a button and display a form or two to your user. However, imagine a loan application at a bank: once the application form is submitted, a larger series of tasks and decisions is initiated. You must validate the application, validate the applicant’s identity, pull their credit report, perform a risk assessment, and, depending on the loan type, many other steps, before the funds are disbursed to the account. Automating the loan application process end to end is a great use of Camunda and process orchestration.
Camunda enables creating and managing these processes, and what the individual tasks in each process do, with a no/low-code approach, a pro-code approach, or a mix. However, Camunda is not a drop-in no-code solution for your backend.
Now let’s talk about that “delay” and some solutions …
Depending on the size of your cluster in SaaS (or the resources you’ve allocated to your Self-Managed deployment), the speed at which a process instance starts is somewhat variable. If the first task in your process is a form, a delay of a few seconds from sending the request to start the process to the time that form is available in Tasklist or via the API is somewhat expected. Camunda is capable of running millions of process instances at a single time, so there are optimizations that can be done to help minimize that time delay. But remember that Zeebe is an asychronous and event-driven engine, not a synchronous and transactional one.
If a user interaction requiring a form is the start of your process, you can skip the process instantiation step entirely and start a process directly with a form.
If you have a more complex process that requires forms or human interaction in the middle (and you aren’t using Tasklist), I would consider using the “push” method I described in my previous reply: instead of your application polling the process to watch for these events, have the process tell your application when those events happen.
A similar “push” method can be used for events beyond user tasks; for instance, if your application needs to know when a process has ended, you can include a final service task right before the end event that notifies your application. (Generally speaking, it is a best practice to explicitly model as much of this as possible.)
Phew, that was a lot of writing!
Thanks for reading through this long reply! I felt it needed a much more detailed explanation, as we have received this same question from several users this week. Hopefully this helped clarify things; if not, please let me know what other questions you have. As mentioned earlier, this is something we need to improve in our documentation and I’ll be working with the team to make those improvements. If you have any specific feedback or thoughts on the documentation, please send me a message, I would love to hear it
5 Likes
Still facing with that issue. I create process using Zeebe client in nodejs then get active tasks directly from elasticsearch:
async activatedTasks(processInstanceKey: string) {
const url = "http://localhost:9200/tasklist-task-8.4.0_/_search";
// Define the query
const query = {
query: {
bool: {
must: [{ match: { processInstanceId: processInstanceKey } }],
},
},
};
// Send the POST request using Axios
const response = await axios.post(url, query, {
headers: {
"Content-Type": "application/json",
},
});
return response.data;
}
If I do request I get response with empty hits so I need to delay code for about 10 seconds or sometimes 5-6 seconds.
I can’t start process with a form as you mentioned since my form requires data from rest api. So I tried to export form directly from default exporter which is ElasticSearch
@MagzhanUnited - as explained before, that is expected, because the data needs to be processed by Zeebe, sent to the exporter, written to Elasticsearch and indexed. Camunda is an asynchronous and event-driven platform. The process you presented is not an ideal use case for Camunda and process orchestration; if you are simply trying to create a form wizard that displays different forms based on different data conditions, there tools that are much better suited to that. I also presented other possible solutions that work with the event-driven nature of Camunda.
Over the next several releases, there will be some features that will help with what you’re trying to achieve (specifically in the 8.6 release in October). However, you will still likely have issues trying to use the asynchronous engine synchronously for your front-end.
I wanted to do the same thing as here: https://youtu.be/HhxghzKMFBg?si=fMm2NvU1T_NQ0LN8
Also my aim isn’t just displaying form on certain conditions. User tasks have to wait for its assigning and execution. I’m sure that Camunda is what I need.
@MagzhanUnited - that demonstration is 5 years old, and is using an older version of Camunda 7. C7 was a synchronous engine that could be embedded within your application. C8 is a ground-up rewrite that uses a faster, more scalable engine that is now asynchronous and event-driven. That solution does not work the same way with C8, and some C7 patterns need to be re-designed for C8.
As mentioned several times, you have a few options:
- Use an event-driven architecture style, and have the model push changes to your application. This bypasses the exporter, which is the primary cause for the “delay.”
- Use a custom exporter that does the same thing: pushes events to your application.
- The 8.6 (October '24) and later releases will include not only improvements to the overall speed at which the exports occur, but also some additional features that will help with getting a more real-time view of the process. However, because C8 will still be asynchronous and event-driven, it will still not be the same as what you’re expecting.
Hi there,
Firstly, I want to express my gratitude for your time. Secondly, I’m having difficulty understanding two main aspects:
- How can I personalize my own exporter?
- How do exporters contribute to minimizing delays?
Hi @Rania_Chakroun !
On a high level, exporters are explained here: (Exporters | Camunda 8 Docs).
To get started, it’s best to look at the projects from the community: Repositories · GitHub
The exporter config allow you to define the batch size and time yourself. You can, for example, define an export time of 1 second, which means events are exported every second.
Note that the performance of Zeebe can decrease when export processing is too compute-intense. That’s why the default is 5 seconds as we give guarantees on processing speed before on output speed. But here we are speaking about cutting-edge processing like financial transaction use cases where it goes into the millions per second.
Best,
Christian
2 Likes
is C7 still sync?
1 Like
@MagzhanUnited - C7 is still a synchronous engine, yes; however, it is an end-of-life product, with development ending next year with only enterprise accounts receiving security updates after.
I was a bit vague with what new features are coming in 8.6+ - I’m going to talk with our product team and see what can be shared in advance. I don’t want to accidentally make any false promises.
Ok, everyone, thanks for your replies. Now I understand the nature of Camunda 8.
We still would be happy if there’s a solution to the problem, and yesterday I was informed that Zeebee 8.5 introduces Zeebee User Tasks. With Zeebe User Tasks we can directly query the workflow engine for the status of the task. However, searching for tasks is still done with the Tasklist API, so it doesn’t solve our problem.
Hi @kristoffer.jalen - I saw your email chain also, but thought a more public reply here would be beneficial. You are correct, searching for tasks is still enabled by Tasklist, which requires the data to be exported before the task will appear in the search results. As of right now, that architecture isn’t changing in the 8.6 release. However, you might notice a spoiler in the documentation (here and here):
Task listeners are scheduled to be released in 8.6. (This is one of the “some changes are coming in 8.6 but I’m not sure what I can publicly comment on yet” features I was alluding to earlier! They will support Zeebe User Tasks, but I admit I am not sure the full scope of the feature and what other task types may or may not be supported.) Task listeners will help a lot with what you’re trying to achieve, and do it in an event-driven manner, rather than needing your application to query the process.
In addition to what Nathan has written, there will also be real-time GET endpoints in 8.6, allowing to retrieve a task after creation by it’s ID.
I solved this problem with job worker that directly inserts data into elasticsearch without Zeebe
2 Likes
@nathan.loding, in There's a delay after a process instance is started - #9 by nathan.loding, you say:
I would recommend not using a User Task, and instead use a Service Task with either a job worker or Connector to push the task to your application, rather than your application trying to pull it.
I don’t understand how to model, nor how to implement, this. You mentioned using a Service Task. I can’t see how that can work since I need a form. A form must be associated to a process definition via a User Task. I tried using a Service Task and added a “customer header” with the formKey as value, but that obviously doesn’t work, because the form cannot be found unless it’s associated with a User Task.
If I would use a user task implemented as a job worker, and then my worker is actually able to extract the processDefinitionKey, the taskId, and the formKey. These three values are those I need to get the form, and to complete the task. The worker can send these values to the browser using WebSockets. Is that a good idea?
Can I use a Zeebe user task? But then how should my model push the data to the application? There is no worker for a Zeebe user task. Should I use Zeebe user tasks in combination with a Service Task that sends an event to my application via some event streaming platform, like Kafka or Azure Event Hub? Is that what you were suggesting here:
A similar “push” method can be used for events beyond user tasks; for instance, if your application needs to know when a process has ended, you can include a final service task right before the end event that notifies your application.
@kristoffer.jalen - good questions! Of course the answer is “it depends,” but let me try to answer a few of your questions:
- You are correct, when using a service task you can’t easily pass a form via ID. Because the forms are just JSON templates, and you are using a custom task list UI (instead of using Tasklist), you do not necessarily have to use user tasks. My suggestion was meant as an alternative approach; it may not be strictly semantic BPMN but may work for your use case and requirements.
- You can implement a job worker that listens for user tasks (if you search the forum, there are several people who have done this), but the downside is that your job worker is triggered for every user task. This means you need to implement logic in a single worker to handle all the possible user tasks in your process. It’s not my favorite approach but it is workable.
- Zeebe user tasks do not work with job workers. But a Zeebe user task has its lifecycle managed by Zeebe, rather than managed by Tasklist, which has some advantages; however, the feature you are really looking for is planned for the 8.6 release in October, which is the real-time GET endpoints mentioned by Christian.
- Regarding the “push method” I mentioned, you don’t necessarily need Kafka or Event Hub. It was meant to show that you don’t need to query the process all the time, and instead you can use other tasks (such a REST Connector, or Kafka Connector, or just a service task with a job worker) to send data to your application. This bypasses the exporter and any delays in data availability outside Zeebe and sends the information directly to your application as soon as the process gets to that task.
Hope that helps, but please let me know what other questions you have! And hopefully we’ll have a chance to connect at CamundaCon!
1 Like