Deploying Camunda 7 in an autoscale k8s cluster

Camunda 7 Topics Discussion & Questions
1

Hi,

Has anyone deployed Camunda 7 in an autoscale k8s cluster? Our use case must support bursts of workflow creation/execution, and I would hate to scale resources to meet the worst case scenario. If you did, any gotchas or recommendations you can share?

Thanks!

2

Hi @AlAnt

Without knowing anything about your setup, it might be worth looking into using external tasks if possible. That way the work performed by the process is given to external worker applications that are executed outside the process engine and these can be scaled according to needs. This way you might not have to scale the Camunda process engine. You could have multiple external task worker applications running in your K8S cluster. Each can be configured to scale according to its needs. Fx the most intensive tasks in your process can be configured to support heavy load (and scaling) in the K8S cluster, while others may not have the same requirements for scaling.

Hope that helps.

BR
Michael

2 Likes
3

Hi @mimaom,

Thanks for your reply. I guess my next question is - if we embed Camunda in our own process and deploy that in an autoscaling group, are there any unforeseen consequences to doing that? For example, a workflow may be started by one instance, pause waiting for input, but then have the response delivered to another instance. Not sure what would happen under those circumstances.

4

Hi @AlAnt

The clustering feature of Camunda 7 is based on database state. This means that you can start multiple identically (same DB) configured process engines and you will then have a cluster.

One thing to be aware of is the job executor and the feature of deployment awareness. Have a look at this forum entry: Deployment-Aware Job Executor

BR
Michael

5

Hi @mimaom

Good to know about the deployment-aware job executor. It sounds like we need to be extra careful when tuning our backing database to make sure spikes in the number of workers do not overwhelm it. There is no point in scaling up the number of workers if the database cannot take the load.

Thanks for your help!

6

I fail to see how the “deployment aware” setting is related to autoscale. “Deployment aware” should be set correctly even if you have a fixed size cluster.

7