Camunda Number of requests limit

Camunda 7 Topics
1

Hi,

I have a Camunda WF, and I am initiating that WF from a Java project , and I want to create a batch request(multiple requests at same time), but right now what is happening is that if I select , suppose 100 requests, what will happen is that it will take up only 20 requests at a time and proceed as the requests gets processed , which will take a lot of time to complete, and that too not considering if multiple users are creating a batch request, is there any work around for this, any tuning/manipulation that can be done in the standalone.xml file or in any other part?, so that the requests get executed either parallelly or take less time to complete through any other work around.

I am using Camunda 7 community edition, camunda versoin is 7.14 and wildfly is 20.0.1 and the “/start” URL is being used to trigger the WF. No transaction boundaries are set in the process as well.

Thanks in advance!!

2

Did you configure the job executor pool?
Have a look at The Camunda JBoss/WildFly Subsystem | docs.camunda.org
You can define how big the pool is for the job executor threads.
That could be the limiting factor.
But as you said you don’t have any transaction boundaries, so the limitation could also come from Wildfly that might be configured to not process more than 20 requests at a time.

3

Hi @rohwerj ,

How would I go about checking if the limitation is coming from wildfly?, should I check in the standalone.xml file and if so, which property do I need to change and is there any limit till which I can define the number of requests that can be processed at a time?

Thanks

4

The documentation for the undertow subsystem (which is handling the web requests) is here
https://docs.wildfly.org/20/Admin_Guide.html#Undertow
Also executor thread pools can be defined via ee subsystem.
https://docs.wildfly.org/20/Admin_Guide.html#EE
Haven’t used wildfly in a while, so I have no further configuration examples.

5

Oh ok, will go through these , thanks for the help!!

6

Hi,

In the provided examples , the queue length is set to “1000000”, what kind of system configuration would that require?

Thanks

7

As far as I understand it, that is only the size of the queue where incoming requests are stored until they get executed.
The parameters

        core-threads="5"
        max-threads="25"

control how many threads are available to execute these requests simultaneously.

8

Okay, thanks for clarifying!

9

Hi ,

I have tried the above changes and modified the values at two places as shown below:

            <core-threads>5</core-threads>

            <max-threads>25</max-threads>

            <queue-length>50</queue-length>

            <job-acquisitions>

                <job-acquisition name="default">

                    <properties>

                        <property name="lockTimeInMillis">

                            300000

                        </property>

                        <property name="waitTimeInMillis">

                            5000

                        </property>

                        <property name="maxJobsPerAcquisition">

                            10

                        </property>

                    </properties>

                </job-acquisition>

            </job-acquisitions>

        </job-executor>

                <managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" core-threads="5" max-threads="25" keepalive-time="5000" queue-length="50" reject-policy="RETRY_ABORT"/>

            </managed-executor-services>

But even now the request processing in Camunda is limited to 20 requests at a time, so if I create 500 requests it will take a lot of time.

Is there any thing else , any alternate way that can be used to tackle this issue, the result should be that no matter the number of requests , they should be executed parallelly and in as less time as possible.

Thanks!!

10

To try to help isolate this, could you put an “Async After” on your start event, upload the new version of your process, and then repeat the tests?
That will help determine if the cause is in Wildfly or in Camunda, since you could be running into Wildfly thread limits rather than Camunda thread limits.

11

I meant this as an example. Did you try increasing these values? For example to 50 and 100?

This will definitely also help to investigate further, where the restriction is coming from.

12

Hi @GotnOGuts ,

Ok sure, let me try with that, and what should I be looking for after making this change , any particular value or something in the logs?

Thanks!

13

No, I haven’t tried increasing the values that much , tried with lower values though, which had the same result.

14

Okay. Let’s take a look at all involved components.
The camunda job executor has a thread pool, but it only gets active for jobs in the ACT_RU_JOB table. Those come from async continuations or other wait states in your process (e.g. user tasks, timers, messages, signals).
As you said you don’t have that, so the whole process instance will be executed in the callling thread. This will be the http thread, as you said you’re calling via REST. In Wildfly this is handled by undertow, which is the http server and we tried configuring that threadpool.
As the whole process is executed in one transaction, it could also be the database pool, as a connection is needed and kept open for each process instance.
It could also be some component in front of your server (proxy or load balancer) that is limiting requests.
And last it could be your client (e.g. some http client libraray), that restricts requests to not overload the system.
Try to look at all of the components and to isolate which is causing the limitation.

15

Oh ok, thanks for the detailed breakup, will look into these points and check where the issue is.