I have been working on automating a small bicycle factory model. Recently I tried to implement an external job worker using NodeJS (not Java/Spring Boot) with my Camunda 8 SaaS which is working perfectly fine and I am looking to implement more job workers.
Now the question is, what could be the best possible way of implementing multiple job workers? Meaning, does Camunda provide any engine where multiple job workers can be implemented and when the engine is started, all the workers start automatically inside the business model? Right now I have to start the worker manually so, if I have multiple workers then I would need to start all the workers manually every time I run the project. One possible way that I thought is to containerize all the workers inside Docker but that also doesn’t seem to me the best possible approach because then I would probably be having lots of containers and I do not have the best resources for that.
PS: I only intend to implement workers in NodeJS since I am not much familiar with Java/Spring Boot. As reference, here I am attaching my model:
The workers do not run inside the business model. Camunda 8 is a distributed system, so the business model runs inside the Zeebe engine, and workers run decoupled.
You can create a single Node.js application with all the workers that you need (a monolithic worker application), or as individual workers - application per worker.
It sounds like scaling different parts of the system at different rates is not an issue for you, so just create a single application with multiple workers.
You can do that by constructing a single ZBClient, and then creating different workers from it, with the handlers in different files/modules.
Then you just start that one application to run the workers.
Here is the hierarchy of how to deploy / start that worker application, from easiest/fastest to most investment in infrastructure:
Start it on your local machine - eg: open a terminal in VSCode and run the application there.
Run it on a remote machine - eg: start a VM in the cloud, ssh in, run a screen session for process persistence when you log out, and then run the application there.
Dockerise your application, log into a cloud VM and start a detached Docker container manually.
Go all out and deploy the Docker container via Kubernetes.
I usually stop at step 3 for my personal projects.
Thank you sooo much for such a detailed answer and now I have a better understanding of what possibly could be the best approach when implementing multiple job workers which is, by using a single NodeJS application with all the workers.
