How to control back-pressure from Java Client?

Camunda 8 Topics Discussion & Questions
1

Hey guys.

I know that we implement back-pressure in Zeebe as it is also mentioned in the docs. Now I searched for some mechanics in the current Java Client but did not find any. Is it

Thanks a lot and cheerio
Bernd

2

Hi Bernd,

Let’s consider this scenario:

  • Zeebe 0.8.0
  • The user code makes requests at a rate that is higher than what the client library can send the requests at

Then this is influenced by the zeebe.client.maxRequests configuration parameter you linked. Up to that number, the user code can submit asynchronous requests (via #executeAsync). If all these requests are in-flight and the user makes another request, then the #executeAsync blocks for up to 15 seconds (or more specifically the time that is configured via this parameter https://github.com/zeebe-io/zeebe/blob/master/client-java/src/main/java/io/zeebe/client/ClientProperties.java#L36) until one of the other response futures has been resolved and accessed by user code.

I wrote Zeebe 0.8.0 above, because we are quite certain we will change this in the future such that #executeAsync never blocks.

Does that make sense or are you thinking of another scenario?

Cheers,
Thorben

3

Thanks! I actually also thought about the Jobs/WorkItems/Tasks streamed into the client, this rate definitely also needs to be controlled, so that the broker does not send more jobs than the client could handle respectively distributes them correctly among multiple clients. How does this work?

4

This is controlled by parameters. Looking at Zeebe 0.8.0, this can be done as follows for task subscriptions: https://github.com/zeebe-io/zeebe/blob/0.8.0/client-java/src/main/java/io/zeebe/client/task/TaskSubscriptionBuilder.java#L59. For topic subscriptions, this can be done as a client configuration setting: https://github.com/zeebe-io/zeebe/blob/0.8.0/client-java/src/main/java/io/zeebe/client/ClientProperties.java#L64. If you set these parameters to 10,000, the broker makes sure that the client never receives more than 10,000 tasks/events at a time that it has not yet handled. So if a subscriber is rather slow, the broker will slow down with publishing events as well.

In future versions, we want to make the capacity parameter for topic subscriptions configurable on the builder.

edit: internally the client acknowledges handled events/tasks to the broker, and this is how the broker knows when to push more of them.

5

Thanks a lot - that answers my question and I was indeed to stupid to find this method (and I looked at the susbcrription builder). Sorry! Probably “back-pressure” should be a keyword mentioned in the Javadocs?

6

No worries :slight_smile:

That’s a good idea. In addition, the javadoc should explain for a setting what it is useful for instead of just listing the immediate effect. I raised an issue: Client: Improve settings documentation · Issue #798 · camunda/zeebe · GitHub

1 Like