JGeek: Multitenancy in Zeebe Engine
I am searching for documentation that details on how multitenacy is implemented in Zeebe engine but unable to do so. The key details I am trying to understand is whether zeebe is multi-tenant ? i.e.
- how to define tenants when deploying a zeebe cluster?
- how does the process execution happens across brokers?
- Can we configure brokers to execute processes only for specific tenants?
I also came across a <https://forum.camunda.io/t/is-camunda-platform-8-based-on-zeebe/36941/11|forum post> that stated camunda 8 is not multi-tenant yet which confuses me. More details on this would be highly appreciated.
Thanks.
lzgabel: Hi <@U03F69Y0V8A>
For more details, follow this issue: https://github.com/camunda/zeebe/issues/3439
zell: https://forum.camunda.io/t/multi-tenant-for-camunda-8/37885/5?u=zelldon
JGeek: Thanks @zell for the answers on the thread. Frankly I would like to understand the data flow before I ask more questions on multi-tenancy.
I will ask them here instead of distracting on the forum thread.
In our microservice when I define a method that is marked with @ZeebeWorker annotation - where is that method executed ? Inside our microservice jvm or inside zeebe broker jvm? If its inside the microservice jvm then what code of the process instance execution does the zeebe broker execute?
Josh Wulf: The Zeebe broker executes the BPMN orchestration. The implementation of any of the service tasks (all tasks are service tasks in Zeebe) is done in workers.
In Camunda 8 SaaS you have connectors like the HTTP worker and also the DMN worker running in the cluster. But these are hosted workers.
All implementations run in “your own JVM”, not the broker’s.
JGeek: > The Zeebe broker executes the BPMN orchestration.
I am new to BPMN so I might be wrong but so far my understanding is that BPMN execution mainly involves execution of user and service tasks. The service tasks would have the most important application logic coded in it. If all of this executes in the microservice jvm, I assume even a 2 broker node and 1 gateway node zeebe cluster could suffice.
What could be the reasons of scaling the zeebe broker nodes horizontally? Ability to handle concurrent workflow process submissions?
Pardon me for my knowledge on zeebe workflow engine. I am understanding it as we discuss.
Thanks
Josh Wulf: It’s for massive parallel execution. They do not dynamically scale (yet), but it means that you can build clusters that handle significantly larger loads than other architectures.
JGeek: > It’s for massive parallel execution.
Parallel execution of workflow process instances right? Not of tasks within the workflow process instance
Josh Wulf: If you have parallel multi-instance service tasks, they benefit also.
JGeek: @Josh Wulf - I am confused. You mentioned earlier that service tasks are not executed on zeebe brokers so I didn’t follow your last comment. If there is a data flow diagram that explains what is executed on what it would be great to get a clear picture about zeebe’s architecture. We are evaluating multiple workflow engines and are leaning towards camunda 8 hence its important for us to understand how workflows get executed. Thanks for hearing out.
zell: > What could be the reasons of scaling the zeebe broker nodes horizontally? Ability to handle concurrent workflow process submissions?
Just to add something here. I think one major part is to make Zeebe fault-tolerant. Because if you have just one node your complete process execution is stuck, but if you have multiple nodes participating on one partition (configured via replication factor) you can continue with your process execution even if a node fails, which is likely to happen sooner or later.
JGeek: > https://camunda.com/blog/2022/04/why-performance-of-parallel-tasks-in-camunda-platform-8-is-so-good/
Thanks for sharing the link. I understand that apart from concurrent workflow executions, scaling is also needed for multi-instance service task.
@zell and @Josh Wulf - One thing that I don’t understand clearly is given that the service task execution is done by the microservice, won’t zeebe broker just needs to spawn a thread for each (instance) service task since the core business logic is done by the worker? Are we saying that there are workloads that end up requiring the zeebe broker to spawn few thousands of threads and that’s when we add more brokers?
zell: I guess you should check out the camunda summit recordings https://page.camunda.com/recordings-camunda-community-summit-2022 and
watch "Building a Workflow Engine Based on Stream Processing" by @saig0 which explains pretty good how things are executed and how it works.
But just to answer this here also directly. There are no extra threads spawned. Process execution is based on stream processing. There exist one stream processor per partition which acts on Commands, which have been written to the log. Process execution is moved forward via writing new commands to the logstream.
Please take a look at the documentation https://docs.camunda.io/docs/components/zeebe/technical-concepts/internal-processing/ the ZEP might be also an interesting resource https://github.com/zeebe-io/enhancements/blob/master/ZEP004-wf-stream-processing.md#summary
Josh Wulf: @JGeek - for the future, please ask different questions in new threads. We have an archivist bot that can ship threads off to the Forum. With this one there are two different questions in the same thread.
Josh Wulf: Let me see if I can come up with a title that makes this discoverable… 
Note: This post was generated by Slack Archivist from a conversation in the Camunda Platform 8 Slack, a source of valuable discussions on Camunda 8 (get an invite). Someone in the Slack thought this was worth sharing!
If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.