Camunda 8 on GKE using ingress-nginx default setting

[atultewari]

Hello Camunda,
We are attempting to deploy 8.6.2 to a GKE cluster using nginx as the ingress controller. Jack’s post related to GKE using gke-ingress was helpful. But as a first step, we want to get the default Camunda 8 platform running with nginx first.

We deployed ingress-nginx using command

helm upgrade --install  ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace

We have a values.yaml for Camunda as follows:

global:
    identity:
    auth:
      publicIssuerUrl: "http://dev.camunda.bbbb.io/auth/realms/camunda-platform"
      operate:
        redirectUrl: "http://dev.camunda.bbbb.io/operate"
      tasklist:
        redirectUrl: "http://dev.camunda.bbbb.io/tasklist"
      optimize:
        redirectUrl: "http://dev.camunda.bbbb.io/optimize"
  ingress:
    enabled: true
    className: nginx
    host: "dev.camunda.bbbb.io"

operate:
  contextPath: "/operate"

tasklist:
  contextPath: "/tasklist"

optimize:
  contextPath: "/optimize"

zeebeGateway:
  ingress:
    enabled: true
    className: nginx
    host: "zeebe.dev.camunda.bbbb.io"

We used the following command to deploy:

helm install camunda camunda/camunda-platform -f camunda-values.yaml 

Following deployments successfully deploy and start:

• Zeebe Gateway
• Optimize

We’ve waited for more than 60 minutes and none of the other deployments successfully start. It is quite possible we may be missing additional properties within the values.yaml.

Any insights why such a simple configuration fails to startup?

Thanks!

[cpbpm]

Please do share full values.yaml. Did you allocate required resource for ElasticSearch. Please do share the error details.

[atultewari]

Thanks for your response. My previous post includes the entire values.yaml for Camunda. For elasticsearch, there are no override resources.

Following pods are failing with same error “Readiness probe failed”.

• Connectors
• Tasklist
• operate
• auth
• keycloak

Does keycloak require SSL, as in, it’s publicIssuerUrl needs to be SSL?

Error in keycloak

keycloak 13:20:44.30 INFO ==> ** Starting keycloak **
Appending additional Java properties to JAVA_OPTS
2024-11-12 13:21:02,755 WARN [io.agroal.pool] (agroal-11) Datasource '<default>': FATAL: password authentication failed for user "bn_keycloak"
2024-11-12 13:21:02,758 WARN [org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator] (JPA Startup Thread) HHH000342: Could not obtain connection to query metadata: java.lang.NullPointerException: Cannot throw exception because the return value of "java.util.function.BiFunction.apply(Object, Object)" is null
at org.hibernate.resource.transaction.backend.jta.internal.JtaIsolationDelegate.doTheWork(JtaIsolationDelegate.java:202)
at org.hibernate.resource.transaction.backend.jta.internal.JtaIsolationDelegate.lambda$delegateWork$3(JtaIsolationDelegate.java:91)
at org.hibernate.resource.transaction.backend.jta.internal.JtaIsolationDelegate.doInSuspendedTransaction(JtaIsolationDelegate.java:123)
at org.hibernate.resource.transaction.backend.jta.internal.JtaIsolationDelegate.delegateWork(JtaIsolationDelegate.java:88)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.getJdbcEnvironmentUsingJdbcMetadata(JdbcEnvironmentInitiator.java:276)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:107)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:68)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:130)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:238)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:215)
at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:52)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:136)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:247)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:215)
at org.hibernate.boot.internal.SessionFactoryOptionsBuilder.<init>(SessionFactoryOptionsBuilder.java:276)
at io.quarkus.hibernate.orm.runtime.recording.PrevalidatedQuarkusMetadata.buildSessionFactoryOptionsBuilder(PrevalidatedQuarkusMetadata.java:70)
at io.quarkus.hibernate.orm.runtime.boot.FastBootEntityManagerFactoryBuilder.build(FastBootEntityManagerFactoryBuilder.java:84)
at io.quarkus.hibernate.orm.runtime.FastBootHibernatePersistenceProvider.createEntityManagerFactory(FastBootHibernatePersistenceProvider.java:72)
at jakarta.persistence.Persistence.createEntityManagerFactory(Persistence.java:80)
at jakarta.persistence.Persistence.createEntityManagerFactory(Persistence.java:55)
at io.quarkus.hibernate.orm.runtime.JPAConfig$LazyPersistenceUnit.get(JPAConfig.java:157)
at io.quarkus.hibernate.orm.runtime.JPAConfig$1.run(JPAConfig.java:64)
at java.base/java.lang.Thread.run(Thread.java:1583)

2024-11-12 13:21:03,759 INFO [org.infinispan.CONTAINER] (Thread-5) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2024-11-12 13:21:05,260 WARN [org.jgroups.stack.Configurator] (Thread-5) JGRP000014: ThreadPool.thread_dumps_threshold has been deprecated: ignored
2024-11-12 13:21:05,449 INFO [org.infinispan.CLUSTER] (Thread-5) ISPN000078: Starting JGroups channel `ISPN` with stack `kubernetes`
2024-11-12 13:21:05,458 INFO [org.jgroups.JChannel] (Thread-5) local_addr: 42600b84-bbf3-4d12-82ed-4d03e0892399, name: camunda-keycloak-0-44632
2024-11-12 13:21:05,545 INFO [org.jgroups.protocols.FD_SOCK2] (Thread-5) server listening on *.57800
2024-11-12 13:21:06,183 WARN [io.agroal.pool] (agroal-11) Datasource '<default>': FATAL: password authentication failed for user "bn_keycloak"
2024-11-12 13:21:06,242 INFO [com.arjuna.ats.jbossatx] (main) ARJUNA032014: Stopping transaction recovery manager
2024-11-12 13:21:06,346 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (development) mode
2024-11-12 13:21:06,346 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to obtain JDBC connection
2024-11-12 13:21:06,346 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: FATAL: password authentication failed for user "bn_keycloak"
2024-11-12 13:21:06,346 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) For more details run the same command passing the '--verbose' option. Also you can use '--help' to see the details about the usage of the particular command.

[atultewari]

As it turns out, we forgot to delete the pvc from a previous bad install run. After deleting all PVCs, and restarting with a clean install, we were able to successfully deploy Camunda and all it’s components.

Thank you Chander, for probing those questions. It helped us review our entire cluster thoroughly.

Regards,
Atul