We have multiple embedded process engines that all use the same database. (multiple application-instances using the same database)
For example, we have one process engine in the development environment and a second process engine that is started locally.
Our Camunda models consist to a large extent of blocks that are processed by the JobExecutor (Asynchronous Before, …).
How can we configure / implement that each ProcessEngine / JobExecutor only processes its own (self-generated) jobs? We would like to process the jobs of the respective ProcessEngines isolated.
Unfortunately, this is not directly possible. Why are you using one database for multiple engines?
I recommend using at least separate database schemata. As longs as all engines access the same schema, they may execute the jobs created by the other engines.
We are using multiple application instances (each running an embedded Camunda Engine). We are using the same DB-schema because of workload distribution and application-instance outage scenarios.
However when we are developing on new or modifications of existing worfklows each developer connects to the same database-schema as our application instance on dev-environment therefore potentially interfering with the dev-environment.
we were thinking if maybe the multitenancy feature can help us here or if we can modify / override the job acquisition mechanism. Do you think this would be a feasible approach?
Did I understand correctly that your recommendation in this case would be that each developer has their own DB schema on which they can run the local Camunda engine.
I was writing a long response (see below) and then remembered something helpful: In your configuration, you can set the job worker to be deployment aware. The property camunda.bpm.job-execution needs to be set to true. An engine does only pick up jobs for the deployments registered to the engine. This information is not persisted. Therefore, I recommend to (re-)deploy all processes for one engine at startup. You can read more about the Deployment-Aware Job Executor in the following thread:
Often, dev environments operate only on an in-memory database, such as h2. This makes the dev setup lightweight and easy to set up. Yet, I completely understand that it is desirable to have a dev environment that resembles the production setup. As you’ve observed already, each engine can access the jobs of all other engines when they operate on the same database.
Some logical separation is necessary to isolate them. This can be done via separate schemata or separate databases. I’d recommend having one database (or database schema) per application. If multiple developers work on the same application, changes should be tested locally (if necessary, with an in-memory setup). Only if tests pass, the changes should be deployed to the dev-environment that is used by multiple developers (for the next stage in testing).
Now, to your suggestion: using multi-tenancy with a custom job acquisition mechanism. I think it is theoretically possible. Each job includes its tenant. So, all required information are available, but I think I’d be tedious.