How does zeebe ensure jobs are ordered and executed exactly once

How does zeebe ensure jobs are ordered and executed exactly once

Hey @robin

not sure what you mean by ordered but there is no such guarantee. If you have multiple instances with the same job type it is not guaranteed which is activated first by workers.

There is also nothing like exactly once it is more a at least once. It can always happen that if a worker takes really long time (longer then the job timeout) another worker picks the job up and starts to work on it. If the first worker completes the job, and afterwards the second worker the second complete will fail.

Does this help?

Greets
Chris

How is that done?
What are the implementation details for competing consumers? Is it through the lock? But locks cause performance degradation

Hey @robin

good question :slight_smile:

There is no lock involved, it is by design.

A job is tied to a process instance, which is tied to a partition. Each partition has an append only log.

If two commands come in they end on a structure called dispatcher, which is a ring buffer (or many-to-one queue). This allows to bring them in a sequential order. They are processed on by another, which means the first complete is accepted (what ever the first complete is). The second complete will be rejected.

This is of course simplified.

This also means that it can happen that the second worker might be faster then the first and completes before the first worker.

Hope that helps.

Greets
Chris

Thanks, is there any more detailed documentation for me to learn about backpressure, ring buffers?

Hey @robin

regarding backpressure you can read a bit here

Regarding ring buffers I think this part is not documented since it is quite deep inside Zeebe. But you can have a look about ring buffers or many to one queues in general example by aeron Concurrent Collections - Aeron Cookbook

Greets
Chris

Thank you very much