Background on application Setup:
Application A currently runs its own Camunda engine and has multiple replicas in production.
We are planning to introduce Application B, which is exactly same as A in nature but differs in the way it stores and processes metadata (stored as Camunda variables) to execute workflows. Application B shares the same Camunda database tables and process definitions as Application A and is intended to eventually replace Application A.
Problem Statement:
Both applications have the capability to advance any workflow process, including those initiated by the other application, to the next step.This functionality is managed by us, meaning that if a process is started by Application A, the UI can call Application B to continue that process.
However, an issue arises when timer events occur, as Camunda’s Job acquisition process takes control and both applications start processing timer events from each other without our intervention. Therefore, it becomes necessary to manage timer events of Application A and B separately, while still allowing process tasks to be executed on the application of our choice.
Requirement:
1. Only the application where events like timer originated should process it.
2. Workflow process tasks should be processed by the application chosen through the UI. This will facilitate the smooth deprecation of the older Application A. We have already achieved this using a common Camunda database.
Questions:
1. How can we meet Requirement #1 while preserving Requirement #2 ? ( Multi tenancy seems to meet requirement #1 but not requirement #2)
2. Is it possible to make timer events reach the application that initiated the process ? ( Deployment aware feature ensures timer events are restricted to deployments not the process which started it. And in our case deployments are managed by a CICD process which is common to both applications. So enabling deployment aware config with this setup prevent timer events from reaching both the applications. )
3. Can the timer event be broadcasted to both application threads, considering that only one application would successfully process the event by our design?
As much as I often caution against deployment aware configurations, you are describing a heterogeneous cluster. Heterogeneous cluster requires deployment aware.
Deployment aware servers need to have the processes that they can handle registered to them.
You can have ProcessA_Version1 available only on Nodes 1 and 2, while having ProcessA_Version2 only available on Nodes 3 and 4, while having all of Nodes 1, 2, 3, and 4 connected to the same database. The timers for ProcessA_Version1 will only execute on node 1 or 2. The timers for ProcessA_Version2 will only execute on node 3 or 4.
Spend a LOT of time reading up on how to implement Deployment-Aware, as it IS tricky.
Thanks for the response.
Deployment aware won’t work for me for 2 reasons.
-
We have a common CICD process with its own process engine to deploy process definitions.
And application A & B has their own process Engine. Now, when I make both applications deployment aware. Both won’t be aware of the process deployed by the CICD as that belongs to another process engine.
-
Even if I get rid of the above CICD and make individual apps to deploy their process definition. With deployment aware, requirement #1 will be met but not requirement #2 .
When the Application Engine for Application A starts up, it registers the deployments (which are made by your CICD) to say “I can complete these processes”
When your CICD deploys a new version of Application A, it has to tell all the Application A servers to register that they can run Application A. This is why I was saying Deployment Aware is tricky and a lot more work. However, it is the ONLY way to run a single-DB heterogeneous cluster.
I’m not 100% sure what you mean by:
Tasklist is cluster aware, so if you start “Application A - Process 1” it will be run only on Application A’s servers. If you start “Application B - Process 1” it will be run only on Application B’s servers.
User Tasks can then be handled “on a single pane of glass” as the user tasks for both Application A and Application B are in one tasklist.
it registers the deployments (which are made by your CICD)
Can both application A & B register for the same deployments ? - This is to meet Requirement #2.
Also, which Camunda API the application is supposed to use for above registration ?
Workflow process tasks should be processed by the application chosen through the UI
By above statement I mean : Since deployment & process definitions are common for both application A & B,
process started by application A can be continued by application B.
Say, UI was talking to application A for a process A. Now, due to upgradation we will make UI to use application B to continue process A. This is something which are able to achieve with existing setup of non-deployment awareness.
I see requirement #1 and #2 can be achieved mutually exclusive but need inputs on how achieve both together.
Yes, HOWEVER, it means that Engine A1, A2, B1, and B2 can handle any process instance for any of those deployments.
If the deployment is the same, then there is no difference between the processes, artifacts, etc, and then there is no reason that Engine B1 or B2 cannot handle a process instance for Application A.
The problem keeps coming back to your statements that Processes for Application B are updated and cannot be handled by Application A’s engines, but that the processes for application A and Application B are the same.
Either the processes are the same or they’re not. They cannot be both the same and not the same at one time.
2 Likes
Here is a bit more detail on the way applications are working right now.
Both applications to be capable to handle the common deployments but only when it directed to do so (by the UI). Reason is the both applications have their own way of storing a Camunda variable which helps the app to drive the process flow.
Say, Application A’s camunda variable is of type com.applicationA.contextA while application B has the type com.applicationA.contextB. Now when UI directs process started by application A to B, we have an intermediary logic to transform contextA to contextB and update that variable before control goes to B’s camunda engine (and we would also ensure no further requests comes for that particular process in future back to application A - so that we gradually move all in progress traffic to new application B and deprecate A ).
So above conversion works fine for user/service tasks. But for timer, control goes to camunda and it hands over the contextA to application B. As we don’t have a transformation logic here, it would fail for timers.
Which means that the processes for Application A and Application B are not the same, and you NEED to run in a Heterogeneous setup to do this.
Yeah thats right. But in a heterogeneous setup how can I meet the below requirement.
- Only the application where events like timer originated should process it.
- But workflow process / service tasks can be processed by any application chosen through the UI.
Deployment aware will not allow me to achieve the second requirement.
Any other suggestions/alternatives to try ?
You still have not explained what Item 2 really means.
Maybe you can submit screenshots to make it clearer.
In a Heterogeneous cluster, Deployment Aware is a requirement.
In the Camunda world, an Application consists of a specific set of process definitions, and a specific process definition doesn’t belong to more than one application. When you select “Application B” you automatically get “Application B”'s version of the processes, which can only be executed on Application B. If you select Application A, it can only be executed on Application A.
In the Camunda world, an Application consists of a specific set of process definitions, and a specific process definition doesn’t belong to more than one application. When you select “Application B” you automatically get “Application B”'s version of the processes, which can only be executed on Application B. If you select Application A, it can only be executed on Application A.
In our context, Application A & B don’t have their own version of processes as it is common to them and belongs to CICD. This makes Application A & B to process any version as per custom logic.
And this is the requirement #2.
My two requirements can be put down as having a heterogeneous cluster with
- application awareness for timer events
- no application awareness for service/user tasks
Note: For #2, we already have custom implementation to achieve that. So no extra configuration needed.
I have mentioned it just to preserve that requirement while trying to solve (1) because introducing deployment awareness to support (1) would hamper (2).
Any solution to achieve (1) without deployment awareness is the need.
Can application register for an instance of process, so that all timer events will reach only that application ?
“Application” is a group of specified process definitions.
“Application” runs within a specified process engine.
So, no an application cannot register for an instance of process, partially because an application doesn’t register for processes, and B because the engine will handle all instances of a particular version of a process definition.
A Timer is a special type of task. You are asking for a process engine to be both instance aware and not instance aware at the same time. Instead of using a custom implementation to make the engine unaware of process versions, let the engine handle it itself. Then you can just run strictly in Deployment aware mode.