Zeebe-Kafka Integration design understanding

Getting Started
#1

Hello,

I was trying to understand the use case for Zeebe to run different workflows for different clients with each clients having multiple events. The source of the events is in kafka. Have shown a high level design which I want to achieve. Could some one help me in understanding if this is best fit use case for Zeebe at the moment and how this can be achieved.

Zeebe-Kafka-Integration
Zeebe-Kafka-Integration1818Ă—1328 127 KB

1 Like
#2

@yogi38in this looks quite good from a high level perspective. My only question would be if you actually need use the same Zeebe cluster from all your customers? You just need to have in mind that if you customers have different requirements or you need strong isolation between, you can separate their clusters.

I wonder where are you planning to run these architecture? Are you running in K8s? in a cloud provider?

Cheers

#3

Hello Sal,

Thanks for quick response. We might have dynamic workflows for new customers and might use same Zeebe cluster. Haven’t thought about this but good point. Considering dynamic nature we would not have separate cluster for different customers. We want to associate a new or same workflow with new customer and may be thinking about templating workflows. We have also thought about Camunda but somehow that does not fit into our event driven architecture I feel.

We are planning to run this on aws cloud.

Thanks,
Yogesh J

#4

@yogi38in that sounds good… if you are looking into EKS check out our Helm Charts: http://helm.zeebe.io
Having Zeebe and Kafka together makes a lot of sense for these kind of architectures.

I am guessing that you knew about this already: https://zeebe.io/blog/2019/08/official-kafka-connector-for-zeebe/

HTH

1 Like
#5

Thanks. Yes I have gone through the Kafka connect Zeebe. Wanted to know the feasibility of above design in multi client - multi workflow scenario. Do you think current zeebe release has the capability to achieve the described design so that I can go ahead with small poc for same.

thx,
Yogesh J

#6

Thanks. Yes I have gone through the Kafka connect Zeebe. Wanted to know the feasibility of above design in multi client - multi workflow scenario. Do you think current zeebe release has the capability to achieve the described design so that I can go ahead with small poc for same. If you have any example resembling similar design might help too.

thx,
Yogesh J

#7

@yogi38in as I said before…from the diagram I cannot see anything wrong. I am not sure also what do you mean by multi client - multi workflow scenario. but I would recommend you to do a PoC and let us know if you find any problems. I cannot see any issue with your design.

Cheers

1 Like
#8

Thanks

1 Like
#9

@salaboy I think @yogi38in is referring to multi-tenancy workflow deployment similar to camunda.

1 Like
#10

perfect!!

#11

@yogi38in and @aravindhrs there are not such a thing in Zeebe, as you can provision a cluster per customer, the isolation is provided at that level. Also a big difference from Camunda BPM is that there is no Database here… it is just an event stream.

#12

Thanks @salaboy. As we just have event streams I assume even a single Zeebe cluster can still provide some degree of isolations for multiple workflows as we don’t plan to provision a cluster per customer looking at dynamic nature of customers.

#13

@yogi38in sounds good to me… but keep in mind that multiple clusters shouldn’t be a problem either.

#14

@salaboy: did you mean “should not be” or I am mis understanding.

1 Like
#15

@yogi38in yes… typo… sorry

#16

hi @salaboy, may be this question is not for this forum but do you think the above architecture can still be achieved using Camunda instead of Zeebe.

#17

@yogi38in it really depend on the load that you are planning to put to the system. Zeebe is designed to work better than Camunda under high load and one of the main purposes of it, is to enable you to scale better. But in general terms you can achieve the same functionality with both. If you are starting a project where you are planning a cloud native architecture (which seems you do) I would try out Zeebe, knowing that if you find it not fit for purpose you can always revert to Camunda BPM.

1 Like
#18

Thanks @salaboy. One of the point which we do have to consider is the stable production version of Zeebe. When do you think Zeebe would be production ready.

#19

@yogi38in sorry for the late reply… this post might help with your question: https://zeebe.io/blog/2019/07/announcing-zeebe-0-20-production-ready/

#20

great thanks. If handy could you also share a post where we can know which functionalities are free and which requires licenses. Also may be how the support is offered currently for getting zeebe to work and analyse performance for production.