Parallel execution of Camunda 8 Outbound connector jobs

Camunda 8 Topics
1

Hi,

I have a question regarding connectors.
I noticed that at least when running it locally a connector task from a process instance can only be executed when the last one using this connector has finished its job. So e.g. when I implement a wait of 10 seconds and start five new process instances then the first one finished after around 10 seconds, the second after around 20, the third after 30 and so on.
Is this just a limitation because of the local development environment or can I configure it so that there are more underlying workers available?

Thanks in advance for hints
Mario

2

Hi @micudaj ,

Please take a look at this: The REST Connector sends only one request at a time for the entire process - #4 by mark.farkas

Based on this post adding the properties below should solve your issue:

zeebe.client.worker.max-jobs-active=32
zeebe.client.worker.threads=10
3

Hi @mark.farkas ,

Thanks for the reply. Unfortunately, I tried this already and it didn’t fix the issue.
Meanwhile I found a Github issue about it where it says that it seems to be a Zeebe client issue: Increasing the client worker threads does not have any impact on parallel job executions · Issue #14662 · camunda/zeebe · GitHub

4

Hi @mark.farkas,

is there an update on this? I’m experiencing the same issue.

Thanks!
Andre

5

Hi @astro

Do you mind sharing, which version of Camunda/Connectors are you running? Also, what is your setup (Helm Charts, Docker, etc)?

Thank you!

6

Hi @igpetrov,

I’m running C8 using docker-compose from this repository’s main branch. In terms of the connector’s SDK, I stick with version 8.3.0.

The same setup applies for @micudaj as well.

Does that help?

Thanks :slight_smile:
Andre

7

Hey @astro, did you try using the latest 8.3.7 version?

8

For me it is neither working with 8.3.7 nor with 8.4.4. same behaviour

9

@micudaj @astro

We are releasing a fix for this behavior in the upcoming 8.3.8 and 8.4.5 releases beginning of March.

10

You can potentially test it today if you like by manually pulling the latest spring-zeebe release candidates into your projects in case you are using a local runtime.

11

hi @sbuettner. I just tried it with version 8.4.5 and still have the same issue. Is the fix included in 8.4.5?

12

@micudaj You should now be able to specify the number of worker threads as described in: GitHub - camunda-community-hub/spring-zeebe: Easily use the Zeebe Java Client in your Spring or Spring Boot projects via an environment variable for example: ZEEBE_CLIENT_WORKER_THREADS=10

13

@sbuettner I have set these values, but still the same behaviour.

zeebe.client.worker.max-jobs-active=32
zeebe.client.worker.threads=10

14

Are you using the runtime docker image? Or do you embed the runtime into your own application?

15

I am running it as in the connector template repository.

I set version.connectors to 8.4.5.

16

Hi @micudaj, I’ve looked into this issue and found a solution. The fix for this bug will be available in connectors version 8.5.0-alpha3, 8.4.6, and 8.3.9. In essence, the issue was with com.google.api.client.http.HttpTransport —the default values for the total number of connections and the maximum per-route connections were both set to 2 . You can track the progress of this fix through the ticket at Inconsistent Parallel Task Execution with REST Outbound Connectors · Issue #2182 · camunda/connectors · GitHub

1 Like
17

Hi @Oleksiivanov, thank you very much for your effort! I subscribed to the issue and will check it once it is available!

18

Hi @Oleksiivanov ,

I just came back to this topic as I was updating the SDK version of our connector and I rechecked the issue.
It is still not working for us and I am not sure if the issue is matching out problem. In our case the connector is not picking up additional tasks for the service task. It waits to pick up pending once until after the first has been finished, but then trigger multiple at once if waiting.

Any ideas?

I added this picture of expected and actual behaviour with max-jobs set to 2 in hope its makes it clearer.

image
image2330×1256 182 KB

19

@sbuettner, could you please look into this, as I’m no longer part of the connectors team

1 Like
20

Hi @micudaj, thanks for your report. Can you tell us which version of Camunda and Connectors you executed your tests against?