DEADLINE_EXCEEDED - Failed to activate jobs for worker

Camunda 8 Topics
1

I am using Camunda 8.50 and spring boot 3.2.5. While starting the spring boot application, sometimes we are getting below exception

WARN 3240 — [lt-executor-153] io.camunda.zeebe.client.job.poller : Failed to activate jobs for worker relatedPersonsWorker#extractRelatedPersons and job type extractRelatedPersons

io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 19.999779200s. Name resolution delay 0.000000000 seconds. [closed=[], open=[[buffered_nanos=20000835700, waiting_for_connection]]]
** at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.62.2.jar:1.62.2]**
** at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481) ~[grpc-stub-1.62.2.jar:1.62.2]**
** at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574) ~[grpc-core-1.62.2.jar:1.62.2]**
** at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72) ~[grpc-core-1.62.2.jar:1.62.2]**
** at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742) ~[grpc-core-1.62.2.jar:1.62.2]**
** at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723) ~[grpc-core-1.62.2.jar:1.62.2]**
** at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.62.2.jar:1.62.2]**
** at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ~[grpc-core-1.62.2.jar:1.62.2]**
** at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[na:na]**
** at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[na:na]**
** at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]**

Same is getting resolved automatically after sometime.

Does anybody facing the same issue ? what’s the permanent solution for this issue ?

2

@Murugesh_P,

DEADLINE_EXCEEDED: deadline exceeded after 19.999...s

This is from the gRPC ActivateJobs call:

  • The client (Spring Zeebe client) waited ~20 seconds for a response
  • No jobs were returned in that time
  • The call was canceled by the client (not the server)

Why It Happens

  1. No jobs were available for the job type extractRelatedPersons
  2. The Gateway held the request open (long polling)
  3. When the timeout (requestTimeout) was hit, the client gave up

This is not a fatal error — just a warning that there were no jobs available during that poll cycle.

Root Cause?

This is expected behavior if:

  • Jobs of this type are rare or not yet created
  • The worker is coming up before the engine is ready
  • The Zeebe Gateway is under pressure and slow to respond
  • You’re using many workers starting up at once

How to Handle It

1. Silence the Warning (Optional)

If this is expected in your system, you can silence the warning log or lower the log level for this class:

logging:
  level:
    io.camunda.zeebe.client.job.poller: ERROR

2. Increase requestTimeout

The default is 20 seconds. You could increase it to 30s or even more if jobs take time to become available.

In Spring Boot config: (find the appropriate attribute for the version)

zeebe:
  client:
    job:
      poll:
        timeout: 30s

3. Add Retry Backoff (handled automatically by client)

The Camunda Java client automatically retries job polling with backoff. Just make sure you don’t suppress these exceptions manually if you’re managing backoff logic.

4. Pre-warm Your System

If all workers start before workflows are deployed or started:

  • You’ll see a burst of these errors until jobs exist.
  • You can delay worker start or use backoff on failure.

Should You Be Anxious?

No. This warning is common and expected — especially in:

  • Systems with infrequent job creation
  • Cold-start scenarios
  • Many workers per job type

But it can be a performance issue if:

  • You’re seeing thousands of these per second
  • You’re not using long polling properly
  • Your Gateway CPU is spiking
3

Hi aravindhrs

Thanks for your guidance. In my case, the the external service tasks is getting struck whenever this warning is occurring. Sometimes, those service tasks are taking more than 10 minutes to complete.

4

@Murugesh_P Have you upgrade the Camunda version?

5

Hi @aravindhrs,

Issue is resolved after I increase the request-timeout

zeebe:
client:
request-timeout: 300s

Thanks for your guidance

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.