Hey guys,
I am looking for some best practices on how to build a Self-Managed Camunda Platform 7 in AWS Cloud. Do you have any suggestions what AWS managed resources to use?
I found in documentation suggesting the following setup that should cover about 80% of use cases “a three-broker cluster with two Elasticsearch instances, Operate, two Zeebe Gateways and Tasklist”. So In that case the AWS arcihtecture would be like EKS cluster that has 9 cointainers, Application load balancer on top of that, some database (which one is better? Thinking Postgres sql but open to suggestions ).
I would appreciate any suggestions or links here. Thanks.
Zeebe indicates Camunda 8, not Camunda 7.
I’m not sure I would recommend Camunda 7 for a AWS environment, since Camunda 8 is designed to be cloud native (but as yet is not dynamically scalable)
So, I guess I would ask… Why specifically C7 if you’re going into AWS?
Hello my friends!
@GotnOGuts please let me better understand your vision regarding this… why don’t you recommend Camunda 7 for an AWS environment? In my knowledge, I see that Camunda 7 works very well in the AWS environment, including my main and largest projects using it today.
It is a fact that Camunda 8 was created for a cloud environment, but there are advantages and disadvantages on each side. It would be great to hear your different vision and opinion!
=========================================================================
@Amit_Kumta I’ll give you some ideas based on my projects… but this may vary according to your needs:
- I use ECS, but you can use EKS without any problems if you prefer;
- EC2 instances that you will choose according to your needs, but if you don’t want to manage the infrastructure and leave it to AWS, you can use Fargate (I’ve never used it, but they say it’s good…), but… everything has a cost hehehe!
- Database I use Postgres… you can use Postgres/RDS;
- Elasticsearch;
- S3 for storing documents;
- Cloudwatch for troubleshooting, to monitor logs and performance.
Hope this helps!
William Robert Alves
2 Likes
Thanks William.
Assuming you had a multi AZ deployment, did you use 2 or 3 AZ? Does Camunda 7 work with 3 AZ’s?
Did you use single region or multi region? Does Camunda 7 work well with multi region deployments?
How should Postgres be designed , multi-region multi-master or single master with read replicas across regions? What will the performance be for failover for both options. What will the performance be for writes?
We use 2 AZ, but I believe it works well with 3 AZ.
We use a single region and I believe this is the most common we see in Camunda deployments. However, I don’t see any problems with deploying it in multi-regions, besides the complexity.
As for choosing the database design, you can choose what best suits your usage needs… if you choose a multi-region/multi-master, you gain in resilience, but it increases complexity and can be affected by latency between regions…
What is your priority? Which operation tends to occur more, reading or writing?
I would use a single master with read replicas between regions, as in my use case, read is more important.
But as I said, this varies from case to case.
William Robert Alves
Thanks William. This helps.
We are heavy writes and reads both. We are using Camunda to process about 200k transactions a day. Maybe more +/- 100K.
I wanted to know more about why you decided to go with post gres db for your use case?
Also curious to know how large your Camunda 7 on AWS implementation is, how many transactions are you processing.
Also with single region 2 az deployment, are you concerned about resiliency or planning to move to 3 AZ’s or multi region for any use case?
Thanks
Hello my friend!
We chose to use Postgres because of its reliability and support for “ACID” transactions, and it meets our needs well.
As for the size of one of our applications, I can consider saying that it is medium-sized, but unfortunately I am not authorized to disclose the number of daily instances being processed for confidentiality/security reasons. I apologize and I hope you understand me!
Regarding the issue of resilience, we are aware of availability and have not ruled out the possibility of expanding to 3 AZ or multi-region… we are constantly monitoring the health of our application, but for our current use case we still do not see a great need of change.
I hope my answers help you find a direction for your use case.
Regards.
William Robert Alves
now that AWS offers ssd gp3 starting from 3000 iops, you should be fine running camunda even on small/medium sizes of postgresql RDS instances. My main problem was always with iops limit before.
Also i choose postgresql as it is well tested on camunda and doesnt require any addiotional configuration (as you would need on mysql transaction isolation level). Also i always run it as single master multi az.
About cpu and memory, you should do some test and adjust your job acquisition configs as you prefer and see if youll need more cpu. I only saw a high cpu usage when we had many (200k+) timer jobs with due date for the same time, making our job acquisition query scan a lot of rows, even with the best indexes. But it was a bad design on our side and after a redesign it all went well.
About your general setup theres not something very complex to think about. I would run it with springboot starter on ECS or EKS. You can scale up and down the number of instances as you need, but know that every new instance add to the resource usage of your database by the job acquisition pollings and task completes.
If youre starting now, i would keep an eye on your json variable sizes, as it can lend to a much higher database usage and higher memory and cpu consumption from your camunda instances to serialize/deserialize and hold them all. Current, i prefer to hold only IDs and things like that, and ask some other database like mongodb for the real data on my delegates when nedded.
We also disabled camunda history as it uses a lot of resources too and we didnt need it as we wasnt using optimize.
also be aware that, in your first post you said something about zeebe, operate, etc. These are for camunda 8 only, which is a very different product (talking about infrastructure).
Camunda 8 doesnt have a sql database.
2 Likes