Camudna 8.8 - Operate jobs aren't processed by Connectors worker

Hi,

I have the problem that my jobs in Operate don’t get a worker from Connectors assigned. see these metadata infos from a node:

{
	"calledProcessDefinitionName": null,
	"calledDecisionDefinitionName": null,
	"jobRetries": 3,
	"jobDeadline": null,
	"jobKey": "2251799813692806",
	"jobType": "io.camunda:http-json:1",
	"jobWorker": "",
	"jobCustomHeaders": {
		"elementTemplateId": "io.camunda.connectors.HttpJson.v2",
		"elementTemplateVersion": "12",
		"retryBackoff": "PT0S",
		"resultVariable": "response"
	},

so nothing happens with the process and is stuck at this node.

I configured authentication via Entra ID with the orchestration service with the official documentation, see my config:

connectors:
  enabled: true
  contextPath: "/connectors"

  configuration: |-
    camunda:
      client:
        mode: self-managed
        grpc-address: "http://${namespace_name}-${stage}-zeebe-gateway:26500"
        rest-address: "http://${namespace_name}-${stage}-zeebe-gateway:8080"
        auth:
          method: oidc
          client-id: ${microsoft_client_id_connectors}
          client-secret: "${connector_secret}"
          token-url: "https://login.microsoftonline.com/${microsoft_tenant_id}/oauth2/v2.0/token"
          audience: ${microsoft_client_id_orchestration}
          scope: "${microsoft_client_id_orchestration}/.default"

  security:
    authentication:
      method: "oidc"
      oidc:
        clientId: ${microsoft_client_id_connectors}
        audience: ${microsoft_client_id_orchestration}
        secret:
          existingSecret: "${namespace_name}-${stage}"
          existingSecretKey: "connectors-secret"
        tokenScope: "${microsoft_client_id_orchestration}/.default"  

I checked the connection with a port redirect and with my Connectors client credentials and it is working but nothing is returned for jobs:

curl -i -sS   -H "Authorization: Bearer $TOKEN"   -H "Content-Type: application/json"   "http://127.0.0.1:8080/v2/jobs/activation"   -d '{
    "type":"io.camunda:http-json:1",
    "worker":"diag",
    "timeout":60000,
    "maxJobsToActivate":1
  }'
HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: default-src 'self'; base-uri 'self'; script-src 'self' https: *.chargebee.com *.mixpanel.com ajax.cloudflare.com static.cloudflareinsights.com; script-src-elem 'self' cdn.jsdelivr.net ; connect-src 'self' https: *.mixpanel.com cloudflareinsights.com *.appcues.net wss://api.appcues.net cdn.jsdelivr.net; style-src 'self' https: 'unsafe-inline' cdn.jsdelivr.net *.googleapis.com *.chargebee.com; img-src data: 'self'; form-action 'self'; frame-ancestors 'self'; frame-src 'self' https: *.chargebee.com blob: ; object-src 'self' blob:; font-src 'self' data: fonts.camunda.io cdn.jsdelivr.net; worker-src 'self' blob:; child-src; script-src-attr 'none'
Referrer-Policy: strict-origin-when-cross-origin
Permissions-Policy: accelerometer=(), ambient-light-sensor=(), attribution-reporting=(), autoplay=(), bluetooth=(), browsing-topics=(), camera=(), compute-pressure=(), cross-origin-isolated=(), deferred-fetch=(), deferred-fetch-minimal=(), display-capture=(), encrypted-media=(), fullscreen=(), gamepad=(), geolocation=(), gyroscope=(), hid=(), identity-credentials-get=(), idle-detection=(), language-detector=(), local-fonts=(), magnetometer=(), microphone=(), midi=(), otp-credentials=(), payment=(), picture-in-picture=(), publickey-credentials-create=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), speaker-selection=(), storage-access=(), summarizer=(), translator=(), usb=(), web-share=(), window-management=(), xr-spatial-tracking=()
Cross-Origin-Opener-Policy: same-origin-allow-popups
Cross-Origin-Embedder-Policy: unsafe-none
Cross-Origin-Resource-Policy: same-site
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 29 Jan 2026 13:33:24 GMT

{"jobs":[]}

I set the necessary permissions for process_definition in identity for the Connectors client id too.

Do I miss something in my config or is something wrong?

Kind Regards,

Julian

Hi @Cris_Ron,

This is a known issue with OIDC authentication in Camunda 8.8. The problem you’re experiencing - where jobs show an empty jobWorker field and aren’t being activated by Connectors - is typically caused by authentication token type mismatch.

Root Cause

When using OIDC with Microsoft Entra ID, job workers (including Connectors) must authenticate with client credentials tokens, not user access tokens. If you use a user-flow token (e.g., from a browser login), workers will connect but no jobs will be activated, and you typically won’t see clear errors in the logs.

Solution

Looking at your configuration, it appears mostly correct, but let me highlight the key points to verify:

1. Verify Your Connectors Client Configuration

Your Connectors configuration looks good, but ensure it’s using client credentials:

connectors:
  enabled: true
  contextPath: "/connectors"

  configuration: |-
    camunda:
      client:
        mode: self-managed
        grpc-address: "http://${namespace_name}-${stage}-zeebe-gateway:26500"
        rest-address: "http://${namespace_name}-${stage}-zeebe-gateway:8080"
        auth:
          method: oidc
          client-id: ${microsoft_client_id_connectors}
          client-secret: "${connector_secret}"
          token-url: "https://login.microsoftonline.com/${microsoft_tenant_id}/oauth2/v2.0/token"
          audience: ${microsoft_client_id_orchestration}
          scope: "${microsoft_client_id_orchestration}/.default"

2. Check Your Orchestration Cluster Configuration

Ensure your Orchestration Cluster is properly configured to accept tokens from your Connectors client. You need something like:

orchestration:
  security:
    authentication:
      oidc:
        clientId: "${microsoft_client_id_orchestration}"
        audience: "${microsoft_client_id_orchestration}"
        # ... other OIDC settings
    initialization:
      defaultRoles:
        connectors:
          clients:
            - "${microsoft_client_id_connectors}"  # This is crucial!

3. Key Things to Verify

  1. Audience matching: The audience in your Connectors config must match what the Orchestration Cluster expects
  2. Client authorization: Your Connectors client ID must be listed under orchestration.security.initialization.defaultRoles.connectors.clients
  3. Scope format: For Entra ID, use <ORCHESTRATION_CLIENT_ID>/.default

4. Troubleshooting Steps

  1. Check Connectors pod logs:

    kubectl logs deployment/camunda-platform-connectors -n <namespace>
    

    Look for authentication errors, 401/403 responses, or token-related issues.

  2. Verify token claims: Decode the token your Connectors client receives and ensure:

    • aud claim matches your orchestration audience
    • Token is issued via client credentials flow (not user flow)
  3. Test with a simple job worker: Try creating a simple external job worker with the same OIDC configuration to see if it can activate jobs.

References

Could you share:

  1. Your complete Orchestration Cluster OIDC configuration (from Helm values)
  2. Any error messages from the Connectors pod logs
  3. Confirmation that your Connectors client ID is listed in the defaultRoles.connectors.clients section

This will help pinpoint the exact configuration issue.

Best regards,
Zee

Here is my orchestration config:

orchestration:
  enabled: true
  contextPath: "/"
  fullURL: "https://${namespace_name}.${service_uri}${stage_uri}${domain}/"

  camunda:
    persistent:
      sessions:
        enabled: true
    rest:
      query:
        enabled: true
 
  ingress:
    grpc:
      enabled: true
      className: ${ingress_controller}
      annotations:
        cert-manager.io/cluster-issuer: ${clusterissuer_name}
        traefik.ingress.kubernetes.io/router.entrypoints: "websecure"
        traefik.ingress.kubernetes.io/router.tls: "true"
        # traefik.ingress.kubernetes.io/router.middlewares: "${namespace_name}-${stage}-strip-connectors@kubernetescrd"
        acme.cert-manager.io/http01-edit-in-place: "false"
        acme.cert-manager.io/http01-ingress-class-name: ${ingress_controller}
        cert-manager.io/common-name: "grpc.${namespace_name}.${service_uri}${stage_uri}${domain}"
        cert-manager.io/duration: "720h"
        cert-manager.io/renew-before: "24h"
        cert-manager.io/subject-organizations: ${subject_organizations}
        cert-manager.io/subject-organizationalunits: ${subject_organizationalunits}
        cert-manager.io/subject-localities: ${subject_localities}
        cert-manager.io/subject-provinces: ${subject_provinces}
        cert-manager.io/subject-countries: ${subject_countries}
        cert-manager.io/private-key-size: "4096"
      host: "grpc.${namespace_name}.${service_uri}${stage_uri}${domain}"
      tls:
        enabled: true
        secretName: "${namespace_name}-${stage}-grpc-tls"

  migration:
    data:
      enabled: false

  index:
    prefix: "${namespace_name}-${stage}"

  profiles:
    operate: true
    tasklist: true

  clusterSize: "3"
  partitionCount: "3"
  replicationFactor: "3"

  security:
    authentication:
      method: "oidc"
      oidc:
        secret:
          existingSecret: "${namespace_name}-${stage}"
          existingSecretKey: "orchestration-secret"

  configuration: |-
    camunda:
      data:
        secondary-storage:
          elasticsearch:
            url: "${elastic_service_protocol}://${elastic_service_url}:${elastic_service_port}"
            username: "${namespace_name}-${stage}"
            password: "${elastic_service_password}"
            security:
              enabled: true
              self-signed: true         
              verify-hostname: false   
            index-prefix: "${namespace_name}-${stage}"
      security:
        authentication:
          method: "oidc"
          oidc:
            client-id: ${microsoft_client_id_orchestration}
            issuer-uri: "https://login.microsoftonline.com/${microsoft_tenant_id}/v2.0"
            client-secret: "${orchestration_secret}"
            audiences: [ "${microsoft_client_id_orchestration}", "${microsoft_client_id_webmodeler}" ]
            redirect-uri: "https://${namespace_name}.${service_uri}${stage_uri}${domain}/sso-callback"  # ochestration contextpath
            client-id-claim: azp # client_id alternativ
            username-claim: preferred_username
            groups-claim: groups
            scope: [ "${microsoft_client_id_orchestration}/.default", "openid", "profile", "offline_access" ]
            prefer-username-claim: true
        initialization:
          defaultRoles:
            admin:
              users:
                - ${initial_claim_user}
              groups:
                - ${initial_claim_group}
            connectors:
              clients:
                - "${microsoft_client_id_orchestration}"
                - "${microsoft_client_id_connectors}"

    zeebe:
      broker:
        gateway:
          enable: true
        exporters:
          elasticsearch:
            classname: io.camunda.zeebe.exporter.ElasticsearchExporter
            args:
              url: "https://${elastic_service_url}:${elastic_service_port}"
              index:
                prefix: "${namespace_name}-${stage}-zeebe"
                createTemplate: true
              authentication:
                username: "${namespace_name}-${stage}"
                password: "${elastic_service_password}"

  env:
    - name: LOGGING_LEVEL_ROOT
      value: debug # info
    # - name: MANAGEMENT_SERVER_BASEPATH
    #   value: "/orchestration"
    # - name: SERVER_SERVLET_CONTEXTPATH
    #   value: "/orchestration"
    - name: CAMUNDA_REST_ENABLED
      value: "true"
    - name: JAVA_TOOL_OPTIONS
      value: >-
        -Djavax.net.ssl.trustStore=/opt/certs/externaldb.jks
        -Djavax.net.ssl.trustStorePassword=${truststore_password}
        -Djavax.net.ssl.trustStoreType=JKS
    - name: CAMUNDA_OPERATE_ELASTICSEARCH_URL
      value: "https://${elastic_service_url}:${elastic_service_port}"
    - name: CAMUNDA_TASKLIST_ELASTICSEARCH_URL
      value: "https://${elastic_service_url}:${elastic_service_port}"

  extraVolumes:
    - name: elastic-ca
      secret:
        secretName: elastic-jks   
        defaultMode: 0444

  extraVolumeMounts:
    - name: elastic-ca
      mountPath: /opt/certs
      readOnly: true

Thanks for sharing your orchestration configuration! I can see a few potential issues that might be causing the problem.

Analysis of Your Configuration

Looking at your orchestration config, I notice:

  1. Audiences configuration: You have both orchestration and webmodeler client IDs in the audiences array:

    audiences: [ "${microsoft_client_id_orchestration}", "${microsoft_client_id_webmodeler}" ]
    
  2. Connectors clients: You have both orchestration and connectors client IDs listed:

    connectors:
      clients:
        - "${microsoft_client_id_orchestration}"
        - "${microsoft_client_id_connectors}"
    

Potential Issues

Issue 1: Audience Mismatch

In your Connectors configuration, you’re using:

audience: ${microsoft_client_id_orchestration}

But your orchestration cluster accepts audiences: ["${microsoft_client_id_orchestration}", "${microsoft_client_id_webmodeler}"]

This should work, but let’s verify the token is being issued correctly.

Issue 2: Client Configuration

Having the orchestration client ID in the connectors.clients list is unusual. Typically, you should only have the dedicated Connectors client ID there.

Troubleshooting Steps

1. Check Connectors Pod Logs

Please run this command and share the output:

kubectl logs deployment/camunda-platform-connectors -n <your-namespace> --tail=100

Look specifically for:

  • Authentication errors
  • Token validation failures
  • Connection errors to Zeebe gateway
  • Any 401/403 HTTP responses

2. Verify Token Claims

When you tested with curl, you used a token. Can you decode that token (using jwt.io or similar) and check:

  • What’s in the aud (audience) claim?
  • What’s in the azp (authorized party) claim?
  • Is it a client credentials token (no sub claim for a user)?

3. Test Token Validation

Try this curl command to test if your token is accepted by the orchestration cluster:

curl -i -H "Authorization: Bearer $TOKEN" \
  "http://127.0.0.1:8080/v2/topology"

4. Simplified Configuration Test

Try temporarily simplifying your orchestration audiences to only include the orchestration client ID:

audiences: [ "${microsoft_client_id_orchestration}" ]

And in the connectors clients, only include the connectors client ID:

connectors:
  clients:
    - "${microsoft_client_id_connectors}"

Expected Behavior

With proper OIDC configuration, you should see:

  1. Connectors pod starts without authentication errors
  2. Connectors successfully connects to Zeebe gateway
  3. Jobs get activated and show a worker name in the jobWorker field

Next Steps

Please share:

  1. The Connectors pod logs
  2. The decoded token claims from your test
  3. Results of the topology API test

This will help identify whether it’s a token issue, audience mismatch, or authorization problem.

References:

Best regards,
Zee

The decoded token looks like following:

curl -sS -X POST -H "Content-Type: application/x-www-form-urlencoded" -d "client_id=$clientid" -d "client_secret=$secret" -d "grant_type=client_credentials" -d "scope=$scope" $url | jq -r '.access_token' | cut -d'.' -f2 | base64 -d

{"aud":"c2f4dec1-7752-4325-980e-704d22b81e01","iss":"https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxx/v2.0","iat":1770030731,"nbf":1770030731,"exp":1770034631,"aio":"ASQA2/8cAAAA+o+nh+4JYlHjKzb7MqTjx82gxZ/kdNJt4sMWNZWdGfA=","azp":"5779d277-610d-4666-9030-7816bf3c638e","azpacr":"1","oid":"b2a1dc2d-174e-4214-876f-dafd915efd26","rh":"1.ARMBWpSlYnJs_0W8P3_shhXWisHe9MJSdyVDmA5wTDK4HgEAAAATAQ.","sub":"b2a1dc2d-274e-4214-876f-dbfd915efd26","tid":"62a5945a-6c72-35ff-bc3f-7fec8615d68a","uti":"YsTpXW3W-UiaLVhCnE1OAA","ver":"2.0","xms_ftd":"_p3cMG6126wvlU5Oa_8CBxy5C5at9xRLVs7kZOM3kHsBZnJhbmNlYy1kc21z"}

There are no errors in the connectors log.
This is the topology output:

curl -i -sS -H "Authorization: Bearer $TOKEN"   "http://127.0.0.1:8080/v2/topology"

HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: default-src 'self'; base-uri 'self'; script-src 'self' https: *.chargebee.com *.mixpanel.com ajax.cloudflare.com static.cloudflareinsights.com; script-src-elem 'self' cdn.jsdelivr.net ; connect-src 'self' https: *.mixpanel.com cloudflareinsights.com *.appcues.net wss://api.appcues.net cdn.jsdelivr.net; style-src 'self' https: 'unsafe-inline' cdn.jsdelivr.net *.googleapis.com *.chargebee.com; img-src data: 'self'; form-action 'self'; frame-ancestors 'self'; frame-src 'self' https: *.chargebee.com blob: ; object-src 'self' blob:; font-src 'self' data: fonts.camunda.io cdn.jsdelivr.net; worker-src 'self' blob:; child-src; script-src-attr 'none'
Referrer-Policy: strict-origin-when-cross-origin
Permissions-Policy: accelerometer=(), ambient-light-sensor=(), attribution-reporting=(), autoplay=(), bluetooth=(), browsing-topics=(), camera=(), compute-pressure=(), cross-origin-isolated=(), deferred-fetch=(), deferred-fetch-minimal=(), display-capture=(), encrypted-media=(), fullscreen=(), gamepad=(), geolocation=(), gyroscope=(), hid=(), identity-credentials-get=(), idle-detection=(), language-detector=(), local-fonts=(), magnetometer=(), microphone=(), midi=(), otp-credentials=(), payment=(), picture-in-picture=(), publickey-credentials-create=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), speaker-selection=(), storage-access=(), summarizer=(), translator=(), usb=(), web-share=(), window-management=(), xr-spatial-tracking=()
Cross-Origin-Opener-Policy: same-origin-allow-popups
Cross-Origin-Embedder-Policy: unsafe-none
Cross-Origin-Resource-Policy: same-site
Content-Type: application/json
Transfer-Encoding: chunked
Date: Mon, 02 Feb 2026 11:22:05 GMT

{"brokers":[{"nodeId":0,"host":"10.10.10.10","port":26501,"partitions":[{"partitionId":1,"role":"leader","health":"healthy"}],"version":"8.8.9"}],"clusterSize":1,"partitionsCount":1,"replicationFactor":1,"gatewayVersion":"8.8.9","lastCompletedChangeId":"-1"}

Great! The token analysis shows everything is working correctly from an authentication perspective:

Token Analysis :white_check_mark:

  • Audience (aud): c2f4dec1-7752-4325-980e-704d22b81e01 (your orchestration client ID)
  • Authorized Party (azp): 5779d277-610d-4666-9030-7816bf3c638e (your connectors client ID)
  • Token Type: Client credentials (proper M2M token)
  • API Access: Topology endpoint works fine (200 response)

This confirms your OIDC configuration is correct and tokens are being validated properly.

The Real Issue

Since authentication is working but jobs aren’t being activated, this suggests the Connectors runtime itself might not be running or connecting properly.

Next Debugging Steps

  1. Check if Connectors is actually running and trying to activate jobs:

    kubectl logs deployment/camunda-platform-connectors -n <namespace> --follow
    

    Look for:

    • Startup messages indicating Connectors is initializing
    • Any job activation attempts
    • Connection messages to Zeebe gateway
    • Any warnings about job types or worker registration
  2. Check Connectors pod status:

    kubectl get pods -l app.kubernetes.io/name=connectors -n <namespace>
    kubectl describe pod <connectors-pod-name> -n <namespace>
    
  3. Verify Connectors is polling for the correct job type:
    The job in your process has type io.camunda:http-json:1. Check if Connectors logs show it’s registering workers for this job type.

  4. Test with a different job type:
    Try creating a simple service task with job type test and see if that gets activated by your curl command:

    curl -i -sS -H "Authorization: Bearer $TOKEN" \
      -H "Content-Type: application/json" \
      "http://127.0.0.1:8080/v2/jobs/activation" \
      -d '{"type":"test","worker":"diag","timeout":60000,"maxJobsToActivate":1}'
    

Potential Issues to Check

1. Connectors Runtime Not Started

If there are no logs from Connectors, the pod might not be starting properly.

2. Job Type Mismatch

Connectors might not be registering for the io.camunda:http-json:1 job type.

3. Network Connectivity

Even though your curl works, Connectors might have different network access to the Zeebe gateway.

4. Configuration Loading

The Connectors runtime might not be loading your OIDC configuration properly.

What to Share Next

Please provide:

  1. Full Connectors pod logs (especially startup sequence)
  2. Pod status and description
  3. Results of the test job type curl command

This will help identify whether it’s a Connectors runtime issue, job type registration problem, or something else entirely.

The good news is your OIDC setup is working perfectly - we just need to figure out why Connectors isn’t picking up the jobs!

Best regards,
Zee

I think the

connectors:
  clients:
    - "${microsoft_client_id_connectors}"

could have been helpful in solving this issue but I’m not sure.

Anyways the problem is gone for the moment.

Issue Resolved! :tada:

Great news that your Connectors jobs are now being activated properly!

Root Cause & Solution

You’re absolutely right that the key fix was likely in the orchestration configuration:

connectors:
  clients:
    - "${microsoft_client_id_connectors}"

What This Fixed

The defaultRoles.connectors.clients configuration grants the Connectors role to specific client IDs, allowing them to:

  • Activate and complete jobs
  • Access process definitions
  • Perform other Connectors-related operations

The Problem

Initially, you had both the orchestration and connectors client IDs listed:

connectors:
  clients:
    - "${microsoft_client_id_orchestration}"
    - "${microsoft_client_id_connectors}"

While this should have worked, having only the dedicated Connectors client ID is the recommended and cleaner approach.

Key Takeaways

  1. OIDC Authentication: Your token configuration was correct from the start
  2. Authorization: The missing piece was proper role assignment for the Connectors client
  3. Best Practice: Use dedicated client IDs for different components (Orchestration vs Connectors)

For Future Reference

When setting up Connectors with OIDC:

  • :white_check_mark: Configure client credentials authentication
  • :white_check_mark: Match audience claims correctly
  • :white_check_mark: Ensure Connectors client ID has the connectors role
  • :white_check_mark: Use dedicated client IDs per component

Thanks for sharing the solution - this will help other users facing similar issues!

References:

Best regards,
Zee

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.