I’m looking to configure 2 separate data sources - One for Process engine and other one for the business entities in the Spring boot Camunda starter project. I’ve tried few options based on little documentation in Camuda docs and Springboot docs and nothing seems to be working out for me. If anyone could provide an example of the configurations I need to create to achieve this, it would be of great help.
Thank you,
Prosenjit Das
It’s not really a Camunda issue - I think looking up a tutorial on multiple datasources in spring boot would server you better.
What exactly is your use case, what are you hoping to achieve by separating the databases? It will add a whole lot more complexity to your system … so imho you should have a very good reason.
Think about this:
Imagine you have a process with a service task “update customer data”. In a regular camunda setup, completing the service task (move to next element, update history, …) and the changes on you “customer” business entity are written in the same transaction, so if either fails, you have a clean retry. If you use separate datasources, you might as well store to a remote system … you won’t have that guarantee. So every now and then your users will file issues like “data not up to date although I finished the task”, “user task still visible although I see that the latest data was entered” … and you will start a fight against distributed transactions, deal with composition, 2PC, … for what?
I’d like to think dynamic and static business data. My static data (“what is the customers name now”) is stored in custom tables and business entities, my dynamic data ("I have a process running to change that customers name, I am just waiting for some docs) is stored in camundas tables. But they both belong to my domain and I keep them in the same DB.
Thanks Niall. I understand.
I see your point and really makes sense. Thanks for that.
I’m very new to Camunda and I just wanted to explore this based on my experience in other BPM platforms where the standard is to have a primary data source to keep platform specific tables and manage your application specific data separate. They could be in the same database instance but separate schema and corresponding data sources. There are several advantages in keeping your business entities/custom tables separate from platform specific tables from a maintainability perspective. This gives you the power to independently mange your app specific tables without even worrying about impacting the critical platform specific tables any way. And in some cases, If I’m embedding Camunda process engine inside my existing application, I might not want to have Camunda tables in my application specific schema/DB and create a separate data source for it. Alternatively,I could go the API way of integrating with Camunda being hosted in a separate container but that might not be something you would always want to do. It really depends. I might not event want to see the Camunda tables while managing my custom tables. I know there are other ways to mange this but keeping them separate is one of the good ones. I’m assuming one is not going to/supposed to touch the Camunds tables as part of application maintenance activity and they should be kept and managed separately.
Again, It’s been just a week since I started exploring Camunda and want to learn what is the best/standard practice of doing certain things.
Thanks,
Prosenjit
Hi Prosenjit,
You can achieve this using “Primary” annotation. Here’s the documentation: https://docs.camunda.org/manual/7.12/user-guide/spring-boot-integration/configuration/#defaultdatasourceconfiguration
Regards
Thanks for the response. I had actually figured it out and got it working. Thanks for your reply anyway.
Hi @jangalinski, your article was very helpful for me. But I have one more question: I understood that I should store the ID of my business entity in Camunda’s business key, and have the tables for the business entities in the same database with Camunda. Now, after a while there comes the moment in which Camunda’s history clean up wants to delete the historical process instances, variables, and so on. Can this history clean up be extended to delete the business entities as well? Or what else do you recommend?
Kind regards,
Christian
Hi Christian. I am not aware of any automation you could use to delete non-camunda entities when the process TTL expires … but I would also doubt that this would be what you want to do … consider an order process … the order entity is a business object that you will probably keep forever … while the historic order process instance might not be of interest anymore after a few weeks … so what would be a use case where you want to delete business data once process data is not needed anymore?
Jan
Hi Jan, the use case is very simple: the business data are offers.
There is going to be an awful lot of offers. To keep them in the database, is really just a waste of disk space.