With Camunda 7, we have a Clustered BPM Engine model built on MSSQL.
And, slowly we are moving the process models to the new Camunda BPM. We imagine that some of these process models should run more smoothly than others. For example, if we think of a banking BPM structure, transfer processes (transferring money from one bank to another bank) execute much faster (should be completed in the same day in most cases) than creating a new customer (customer onboarding) process.
In addition to this, we are also planning to create some batch jobs with Camunda BPM within the same system. For instance, starting a new process for each of the 10,000 documents placed in a certain directory. Or you can think of another situation like this, which creates an excessive load on the system. It can happen several times in the same day, also in completely spontaneous time frames.
We want to make sure that “the more important processes”, such as transfer processes, are not affected by system overloads like batch processes, or by general system failures due to a problem with the customer onboarding process.
Is there a option with Camunda that we see in some other systems where the resources on the servers can be shared between processes? For example, if the Camunda BPM instance server has 8 CPU cores, can it be said that 6 of these CPU cores should always run for “the more important processes”?
A dedicated Camunda BPM instance for “the more important processes” such as transfer processes? This is not preferred, we want to handle every processes to be within the same pool.
What kind of Camunda BPM architecture would you design in such a situation?
I think a priority at the process level would be more useful than a priority at the task level. I should be able to say that the Transfer process has a very high priority but the Customer Onboarding process is with normal priority. I think “Priorities at the Process Level” in the Camunda documentation is exactly what we need.
In order to give a process level priority, we need to use this part in Camunda Modeler, right?
When we give a process level priority like this, all wait states used in this process will be processed with higher priority (first this and than others) than wait state job executions in other processes, right?
Another question about giving Job Execution Priority to the process level on Camunda Modeler, there is no such thing as a predefined table of values, as I can see.
I think this part is completely left to the user. So, Camunda does not say something like Job Execution Priority on the Modeler has a minimum priority of 0 and a maximum priority of 10, and you should prioritize your processes accordingly. I guess, I have to create these min/max values like 0 and 10 myself.
Why do you think process level priority won’t work? I guess, you have some experience with it.
Considering that there is a Load Balancer DNS in front of the Camunda REST API and this address is accessed remotely by the FrontEnd application, what kind of design would you suggest if we proceed with multiple Camunda BPM instances?
Before I (or anybody else) am able to answer this question you should exactly describe your setup, i.e. whether you use an embedded engine or a central one or a remote one – and why. Answering such a question is actually not easy and depends on many things. It is usually a subject of a consulting project involving the analysis of your environment, requirements, the structure and goals of the organization etc. Don’t expect a good answer here on the forum. An answer here will surely be a fallacy because noone knows all these facts.
I can summarize our BPM topology with Camunda like this:
We have a FrontEnd application in top of Camunda BPM.
FrontEnd application accesses Camunda REST API with LB1 Load Balancer DNS address (Start process, Complete user task etc.)
We have a second LB2 Load Balancer DNS address, this part is only used to connect to Camunda web applications (Cockpit, Tasklist, Admin). The FrontEnd app does not know this address, it’s used only for operatioal stuff.
I can say that we use “Clustered Camunda BPM” as architecture. LB1 has 6 Camunda BPM instances, LB2 has 2 Camunda BPM instances. All Camunda BPM instances connect to same database runs on MSSQL.
Actually with “Why do you think process level priority won’t work?” I wanted to ask if there was a known issue with setting process level priority in general, because you said it’s not a good idea for the case.
I don’t really expect “to the point” solution to my question in the Forum, but if that is the case, I would like to know what other ways might be. So, if we consider a Camunda architecture like the one above, would there be parts of your answer to my question that you would like to update?
This might work or might not work. We don’t know the load and the numbers. You still have just one DB. It might become a bottleneck. If you are sure that you won’t run into problems here then why would you need job priorities? In my experience, the DB is the limiting factor.
Yes, the load and numbers can be quite high. The part that creates the threshold can be the database. I’m aware of such situations. We are already talking about what needs to be done so that the processes that we call “more important” (such as the transfer processes) are not affected by other process executions on any “high load” day. What I expect from you here is a “general design suggestion”, not an exact solution based on metrics.
