I am making a test project with Camunda. I created my own small workflow and it works fine. Now I have some questions about how the database should be distributed. For this project I use the Spring Boot template where Camunda is in embedded.
Down here I will describe some examples and questions I have based on the knowledge I have about applications, databases and Camunda.
In my mind when starting with Camunda I imagined that Camunda would have its separate database from the domain/business data. The reason is because I thought it would be “cleaner”. I tried to implement this i.e. to have Camunda run on the default H2 database (for development) and my domain data in a PostgreSQL database. I did not manage to implement this with two datasources in Spring Boot.
After researching for a long time I read about that Camunda uses transactions and that these won’t work/have to create them yourself when you have separate databases. This made me think that I would need to have my domain data/tables in the same database as what Camunda stores its data in.
If this is true it made me think what an existing business would do. They would probably not want to have Camunda’s data in their specific domain made database but have it separated. Their data may also be in a NoSQL database which Camunda does not support.
Also if it’s true that transactions don’t work then how does it work when Camunda is not embedded? I would have a Spring Boot application (or any type of application) that talks with REST to Camunda. Camunda will run on its own server and have its own database (i.e. no transactions possible).
So to summarise my questions:
Should Camunda Engine data/tables live in the same database as my domain data? If not what are the advantages and disadvantages of doing so compared to running Camunda in its own database.
If I implement Camunda in an existing project is it always better to use Camunda as standalone service?
If it’s better to have separate databases then how do I implement this in an embedded Spring Boot application? Because there are now two datasources in the Spring Boot app so it may be better to use the standalone architecture?
@eggy in my environment we really dont like to mix camunda metadata with our domain data.
We have domain microservces and camunda talks to them. In camunda database we try to store only process variables important to drive the flow of the process, many of them are IDs to get data from another services.
We had some projects that connected to mysql for camunda and mongodb for domain data too, using mongoTemplate in JavaDelegates, and it worked very well, but maybe you think it wasnt the best architecture as you will be mixing responsibilities hehe.
Think about the workload you will have… maybe you will have much more reads of your business data, old process data, etc. You can have a facade that doesnt use the same camunda service hardware resources to get this data.
Maybe you would like to have something like a spring webflux service searching data directly from mongodb when its serving GET requests, and make it calls other service with camunda sql connection to movement the process for other http verbs. This way you’ll free your facade resources to serve more read requests.
Anyway, i think its always best to have your domain data designed separated from your process metadata.
Hey, thanks for your answer. So if I understand correctly you advise to always separate the Camunda database and your application database.
Do you then also advise when starting a new project to use Spring Boot with Camunda embedded? Because to realise this you would need two datasources in Spring Boot. Something I didn’t manage to do yet but it also sounds not that nice to me. Note: I am a beginner in backend development in Java. Is it better to have Camunda as standalone and talk with it through the REST api?
Hmm, i have used camunda as standlone with wildfly, deploying several different process to the same instance, and since camunda 7.7 I started to use it as embedded in a java springboot microservices and never came back haha.
Maybe this was because our infrastructure is heavy focused on kubernetes, then my embbeded camunda benefit from the horizontal pod scaling we already have. Other great benefit is that it fits more in the microservices architecture as you will have separated camunda instances for each microservice, instead of deploying many business process of different contexts in the same camunda engine and database. In production, with heavy load youll want your job executor resources focused…the more process you have in the same camunda, bigger will be your job queue…
Then you can call camunda REST endpoints of this microservices or as i like to do, creating your own domain REST Api normally using @RestController and call camunda methods from your java, using camunda spring beans like “runtimeService”, “repositoryService” and "taskService. They provide methods like startProcessInstance, correlate message, query for process list, complete user tasks, etc.
Here in my company, we have different springboot camunda microservices for payment process, for user onboarding process, for account creation process, loan services, etc. Almost all of them store business data in another database like mongodb.
If you use mysql/postgresql for camunda and mongodb for you application data, just try to put the dependencies for those two databases in your project. You’ll see it just “works”, because mongodb uses another spring properties, different from the ones used by sql connection string, hehe. If you are using a sql database for business data, then its a little more complicated as you will have to configure a custom connection for this second database, but i had projects working this way too.
If you have any trouble doing it, reply here or send me a message and ill try to help you with some examples.
Thanks for the answer. I was now experimenting with having Camunda standalone. I will soon try to embed it with Spring Boot that then has two SQL connections again.
So I understand you upload 1 specific process per Camunda instance. In the docs they call this homogeneous setup right? (The Job Executor | docs.camunda.org)
Do you maybe have any preference between using Delegate Expression / External (using a topic)?
Its not only one process definition for each camunda… i like to separate it the same way we separate our domain REST services, with several methods of the same context/domain…the same with several process definitions of the same context, hehe.
Ive used a lot of external tasks in RPA projects, but usually i tend to use Delegate Expression and implement the java delegates in the same camunda springboot.
But if you want to be free from java and implement your logic in any other language, you do better with the external task, that are perfect for this, but you will have many more projects, as you will have your camunda engine and your workers as separated projects, and the workers will do polling at your camunda rest api to get and complete tasks.
If you are starting and have java in your environment, i recomend you go with the java delegate expression as it will be much easier to build and debug as you take your first steps with camunda
Thanks for the answers, I just now managed to implement two SQL datasources. I will use your advise and experiment some more. Also with standalone Camunda processes and external tasks.
