Hi Team,
I am installing Camunda 8.3.4 version on my Non prod Openshift environment using camunda helm charts. However, Operate fails to start . I get an error -
Caused by: java.io.IOException: Failed while requesting access token with status code 401 and message Unauthorized.
4617 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsProvider.fetchCredentials(OAuthCredentialsProvider.java:158) ~[zeebe-client-java-8.3.4.jar!/:8.3.4]
46252024-01-02 11:54:27.029 INFO 7 — [ main] i.c.o.z.PartitionHolder : Partition ids can’t be fetched from Zeebe. Return empty partition ids list.
To fix this I tried setting the env property in zeebee gaeway - ZEEBE_GATEWAY_SECURITY_AUTHENTICATION_MODE as none. But the error still persists. I even tried updating the zeebeGateway template>deployment/yaml with this property.But it didnt worked as well.
Please provide some guidance. I had same issue with the 8.2.11 version and this property fixed the issue.
Hi @ShilpaN.Chalke - do you want to run your stack with Identity for authentication or with no authentication? Can you share your values.yaml file (with secrets redacted)?
Hi @nathan.loding , not sure If I understood your qns correctly
I would prefer to run the entire camunda stack (except connectors, webmodeler and console ) with Identity for authentication (i.e basic authentication and auto generated clientId and secrets by helm) , but if there is an issue… for now for non prod environment I am ok with just basic authentication as well (just with basic demo/demo ) should also work.
So these are the values i have configured for latest helm -
for operate.global
operate:
## @param global.identity.auth.operate.clientId defines the client id, which is used by Operate in athentication flows.
clientId: operate
## @param global.identity.auth.operate.audience defines the audience, which is used by Operate.
audience: operate-api
## @param global.identity.auth.operate.existingSecret can be used to reference an existing secret. If not set, a random secret is generated.
# The existing secret should contain an `operate-secret` field, which will be used as secret for the identity-Operate communication.
existingSecret:
## @param global.identity.auth.operate.redirectUrl defines the redirect URL, which is used by Keycloak to access Operate.
# Should be publicly accessible, the default value works if a port-forward to Operate is created to 8081.
# Can be overwritten if ingress is in use and an external IP is available.
redirectUrl: "http://xyz.com"
for Zeebe-Gateway
zeebe-gateway:
## @param zeebe-gateway.replicas defines how many standalone gateways are deployed
replicas: 2
## @extra zeebe-gateway.image configuration to configure the zeebe-gateway image specifics
image:
## @param zeebe-gateway.image.registry can be used to set container image registry.
registry: ""
## @param zeebe-gateway.image.repository defines which image repository to use
repository: repoName
## @param zeebe-gateway.image.tag can be set to overwrite the global tag, which should be used in that chart
tag: 8.3.4
## @param zeebe-gateway.image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
pullSecrets: []
## @param zeebe-gateway.sidecars can be used to attach extra containers to the zeebe gateway deployment
sidecars: []
## @param zeebe-gateway.podAnnotations can be used to define extra gateway pod annotations
podAnnotations: {}
## @param zeebe-gateway.podLabels can be used to define extra gateway pod labels
podLabels: {}
## @param zeebe-gateway.logLevel defines the log level which is used by the gateway
logLevel: info
## @param zeebe-gateway.log4j2 can be used to overwrite the log4j2 configuration of the gateway
log4j2: ''
## @param zeebe-gateway.javaOpts can be used to set java options for the zeebe gateways
javaOpts: >-
-XX:+ExitOnOutOfMemoryError
## @param zeebe-gateway.env can be used to set extra environment variables in each gateway container
env:
- name: ZEEBE_GATEWAY_SECURITY_AUTHENTICATION_MODE
value: NONE /// it by default set this value to identity
for operate
## @section Operate Parameters
## @extra.operate configuration for the Operate sub chart.
operate:
## @param operate.enabled if true, the Operate deployment and its related resources are deployed via a helm release
enabled: true
## @extra operate.image configuration to configure the Operate image specifics
image:
## @param operate.image.registry can be used to set container image registry.
registry: ""
## @param operate.image.repository defines which image repository to use
repository: repoName
## @param operate.image.tag can be set to overwrite the global tag, which should be used in that chart
tag: 8.3.4
## @param operate.image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
pullSecrets: []
## @param operate.sidecars can be used to attach extra containers to the operate deployment
sidecars: []
## @param operate.initContainers can be used to set up extra init containers for the operate pods, useful for additional exporters
initContainers: []
# contextPath: "/operate"
## @param operate.contextPath can be used to make Operate web application works on a custom sub-path. This is mainly used to run Camunda web applications under a single domain.
contextPath: ""
## @param operate.podAnnotations can be used to define extra Operate pod annotations
podAnnotations: {}
## @param operate.podLabels can be used to define extra Operate pod labels
podLabels: {}
## @extra operate.logging configuration for the Operate logging. This template will be directly included in the Operate configuration yaml file
## @param operate.logging.level.ROOT
## @param operate.logging.level.io.camunda.operate
logging:
level:
ROOT: INFO
io.camunda.operate: DEBUG
## @extra operate.service configuration to configure the Operate service.
service:
## @param operate.service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
type: ClusterIP
## @param operate.service.port defines the port of the service, where the Operate web application will be available
port: 80
## @param operate.service.annotations can be used to define annotations, which will be applied to the Operate service
annotations: {}
## @extra operate.resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
## @param operate.resources.requests.cpu
## @param operate.resources.requests.memory
## @param operate.resources.limits.cpu
## @param operate.resources.limits.memory
resources:
requests:
cpu: 600m
memory: 400Mi
limits:
cpu: 2000m
memory: 2Gi
## @param operate.env can be used to set extra environment variables in each Operate container
env:
- name: CAMUNDA_OPERATE_ENTERPRISE
value: "false"
- name: CAMUNDA_OPERATE_ELASTICSEARCH_USERNAME
value: "username"
- name: CAMUNDA_OPERATE_ELASTICSEARCH_PASSWORD
valueFrom:
secretKeyRef:
name:keyname
key: key
- name: CAMUNDA_OPERATE_ZEEBEELASTICSEARCH_USERNAME
value: "username"
- name: CAMUNDA_OPERATE_ZEEBEELASTICSEARCH_PASSWORD
valueFrom:
secretKeyRef:
name: keyname
key: key
- name: CAMUNDA_OPERATE_IDENTITY_ISSUER_URL
value: "http://keycloak/auth/realms/camunda-platform"
## @extra operate.configMap configuration which will be applied to the mounted config map.
configMap:
## @param operate.configMap.defaultMode can be used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
# See https://github.com/kubernetes/api/blob/84c4768022421f4d420a8d74e6699077021563ce/core/v1/types.go#L1615-L1623
defaultMode: 0744
Hi @ShilpaN.Chalke - is that the complete values.yaml file? It looks edited. Can you share the full file and only edit the values where needed (like the secrets)?
If you want to use Identity, then the authentication mode should be set to identity and not none.
I am using Identity mode of authetication
global.identity.auth.enabled is true
All the pods of camunda are running except for Operate -
It fails with error -
83 DEBUG 7 — [SelectorManager] j.i.h.debug : [HttpClient-4-SelectorManager] [282s 546ms] HttpClientImpl(4) Next deadline is 3000
2024-01-17 10:41:50.213 WARN 7 — [ main] i.c.o.z.PartitionHolder : Error occurred when requesting partition ids from Zeebe client: null
io.camunda.zeebe.client.api.command.ClientStatusException: null
at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.transformExecutionException(ZeebeClientFutureImpl.java:116) ~[zeebe-client-java-8.4.0.jar!/:8.4.0]
at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:54) ~[zeebe-client-java-8.4.0.jar!/:8.4.0]
Caused by: io.grpc.StatusRuntimeException: CANCELLED
at io.grpc.Status.asRuntimeException(Status.java:537) ~[grpc-api-1.60.0.jar!/:1.60.0]
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)
Caused by: java.net.SocketTimeoutException: Connect timed out
at sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:551) ~[?:?]
2024-01-17 10:42:14.256 INFO 7 — [ main] i.c.o.z.PartitionHolder : Partition ids can’t be fetched from Zeebe. Try next round (51).
Yes, please share the complete values.yaml file (with secrets removed). It’s hard to say what might be causing issues without seeing the full configuration.