Hey @walt-liuzw,
thanks for the detailed report and reaching us out.
There are different things to increase performance (throughput).
As first could you please show your zeebe cfg?
Because performance depends on different parameters
- How many partitions do you have?
- How many threads/cpu assigned to the broker etc.
- What kind of machine do you use?
- Are you running in a cloud env or local?
I will try to shortly explain why you don’t see any improvement with more workers. If you have for example one partition ( which I assume) then this partition has a maximum throughput it can achieve. This is based on different properties. I mentioned some above.
If you add workers you just increase the consumers, but it might not be more to consume.
I will try to explain that with an analogy.
You can think of the broker as a post office. The partition is one guy which is working there. He has to validate the address, do the stamps etc. He has a maximum throughput. You then have a postman which takes the letters from the office and bring it to the end users. If you add more postman’s you will not increase the letters you can send, since you’re limited by the guy working in the office.
This means you can either put more workers in the office (increase the partitions) or you fire the old guy and hire a new guy which has more throughput (increase the resources, use better machine etc.). You can also do both.
Will having a cluster affect the throughput instead of having one broker ? Of course. If we use the same analogy again, then we have for example three guys working in the office. But only one does the work, the other just have to sign it that this is fine what the one is doing. The throughput will degrade, but if one of the guy is sick the others can take over, which is the benefit you have with a cluster. This again assumes one partition.
Hope this helps to understand it better.
Greets
Chris