Job retried too soon

Camunda 7 Topics Discussion & Questions
1

I have the following setup:

  • 2 instances of a Camunda (7.13) Spring Boot application containerized using docker.
  • Both instances point to the same database
  • 3rd party rest API, accessed via proxy API

The issue:

  1. One of the instances runs a process that during a service task, sends a REST call to the proxy API, which forwards it to the 3rd party API.
  2. 3rd party API takes a long time to process the request and after exactly one minute, it gives a Gateway Timeout response to the proxy, which converts it to a 202 response (which is expected by the Camunda Spring Boot app in order to continue)
  3. Before the Gateway Timeout happens, there is another request made by one of the Camunda Spring Boot app instances, that makes the 3rd party give an error that the resource already exists, which makes the process enter error state.

This has happened multiple times. Here’s an example timeline of the exact times:
[13:10:07] : Request 1 sent
[13:10:58] : Request 2 sent
[13:11:07] : Response for Request 1
[13:11:58] : Response for Request 2 containing error

I believe there might be an issue related to the fact that there are two instances, but I’m not sure how to solve it. The lock-time-in-millis property is left as default, so the job shouldn’t be picked up again until after 5 minutes if I am not mistaken.
Here is my Camunda config:

camunda:
  bpm:
    history-level: none
    authorization:
      enabled: true
    admin-user:
      id: id
      password: pass
      email: email@email.com
      first-name: Admin
      last-name: Admin
    default-number-of-retries: 1
    filter:
      create: All
2