Camunda service task rest call third-party service that can handle only 10 requests per minute

Camunda 7 Topics Discussion & Questions
, ,
1

Problem Statement:

We are encountering an issue in our system where we need to send over 200 requests per minute using a service task implemented in Java. However, the third-party service we are interfacing with can only handle 10 requests per minute.

To manage this, we attempted to configure the Camunda job executor to process 200 jobs in sequence, with each job executing after a 1-minute wait. Here is the configuration we used:

camunda.bpm.job-execution:
  enabled: true
  deployment-aware: false
  core-pool-size: 1
  keep-alive-seconds: 60
  lock-time-in-millis: 60000
  max-jobs-per-acquisition: 1
  max-pool-size: 1
  queue-capacity: 3
  wait-time-in-millis: 60000
  max-wait: 60000
  backoff-time-in-millis: 60000
  max-backoff: 60000
  backoff-decrease-threshold: 100
  wait-increase-factor: 2

Despite this configuration, we are still not achieving 100% process success. The processes are not being handled as expected, and we are seeking assistance on how to properly configure the job executor properties or any alternative solutions that can help us achieve this goal.

Can anyone provide guidance or suggestions on how to resolve this issue using configuration properties?

Thank you in advance for your help!

2

Hello my friend!
Welcome to the community!

I honestly don’t know if the best way to handle this situation would really be in Camunda… I don’t see it as a good practice to change the performance of your application or your job executors because of a call to third-party services…

I’ll give you some ideas of what can be done and the implementation will be up to your team’s expertise:

  • You can create an external task that sends the request to a queue like RabbitMQ, Kafka, SQS or another… and perform the control in the queue itself.

  • You can create a kind of gateway/middleware that receives requests from Camunda and sends them within the stipulated time. You can even use ready-made libraries for this purpose… there is one from Google (Guava rate limiter) that can help you with this. Below is the link to the Java doc for this lib:

https://guava.dev/releases/19.0/api/docs/index.html?com/google/common/util/concurrent/RateLimiter.html

I hope this helps.

William Robert Alves

3 Likes
3

Hi
Is it really a limit on number of requests per minute your need, or could you rather just limit the number of concurrent http requests being made ?

You could configure the http connection pool to limit the number of concurrent requests.

Regards
Alf