New server - pods configuration (crashLoopBack)

Hello,
I’m trying to setup a server dev for a Camunda POC on Unbuntu and i encounter some issue with operate & tasklist pods which are restarting in loop (status : CrashLoopBackOff).
crashPods

After getting logs of these pods it appears that it’s due to a failed connection with ElasticSearch :

2022-07-29 09:12:56.907  INFO 13 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'methodSecurityMetadataSource' of type [org.springframework.security.access.method.DelegatingMethodSecurityMetadataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2022-07-29 09:12:57.532  INFO 13 --- [           main] o.s.b.w.e.t.TomcatWebServer              : Tomcat initialized with port(s): 8080 (http)
2022-07-29 09:12:57.542  INFO 13 --- [           main] o.a.c.h.Http11NioProtocol                : Initializing ProtocolHandler ["http-nio-8080"]
2022-07-29 09:12:57.543  INFO 13 --- [           main] o.a.c.c.StandardService                  : Starting service [Tomcat]
2022-07-29 09:12:57.543  INFO 13 --- [           main] o.a.c.c.StandardEngine                   : Starting Servlet engine: [Apache Tomcat/9.0.60]
2022-07-29 09:12:57.690  INFO 13 --- [           main] o.a.c.c.C.[.[.[/]                        : Initializing Spring embedded WebApplicationContext
2022-07-29 09:12:57.690  INFO 13 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 5993 ms
2022-07-29 09:12:58.196  WARN 13 --- [           main] o.s.b.a.e.EndpointId                     : Endpoint ID 'usage-metrics' contains invalid characters, please migrate to a valid format.
2022-07-29 09:12:58.393  INFO 13 --- [           main] i.c.o.WebappModuleConfiguration          : Starting module: webapp
2022-07-29 09:12:58.510  INFO 13 --- [           main] i.c.o.z.ZeebeConnector                   : Use plaintext connection to zeebe
2022-07-29 09:12:59.412  WARN 13 --- [           main] i.c.o.e.ElasticsearchConnector           : Username and/or password for are empty. Basic authentication for elasticsearch is not used.
2022-07-29 09:13:00.513  WARN 13 --- [           main] i.c.o.u.RetryOperation                   : java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
2022-07-29 09:13:03.518  WARN 13 --- [           main] i.c.o.u.RetryOperation                   : java.util.concurrent.ExecutionException: 
... // after 50 retry...
2022-07-29 09:14:57.626  INFO 13 --- [           main] i.c.o.u.RetryOperation                   : Connect to Elasticsearch cluster [elasticsearch] at http://elasticsearch-master:9200 - Waiting 3 SECONDS. 40/50
2022-07-29 09:15:27.654  WARN 13 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'io.camunda.operate.schema.migration.Migrator': Unsatisfied dependency expressed through field 'retryElasticsearchClient'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'io.camunda.operate.es.RetryElasticsearchClient': Unsatisfied dependency expressed through field 'esClient'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'esClient' defined in class path resource [io/camunda/operate/es/ElasticsearchConnector.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.RestHighLevelClient]: Factory method 'esClient' threw exception; nested exception is io.camunda.operate.exceptions.OperateRuntimeException: Couldn't connect to Elasticsearch. Abort.
2022-07-29 09:15:27.673  INFO 13 --- [           main] o.a.c.c.StandardService                  : Stopping service [Tomcat]

How can i setup this connection ? I’m fairly new to Kind/Helm world, and the application.yml i found were in a docker/volumes… deployed.

Might it be in the ./kube in root directory ?

Many thanks for any kind of help !

Hi @GLE

Is your issue the same as this one: Setting up Camunda Platform 8 Self-managed solution using docker?

Josh

Hello,

Not exactly, i used helm + kind to install the platform.
I first did :
sudo helm install camundapoc camunda/camunda-platform

Leadings to the previous issue, but i just tried with :
sudo helm install camundapoc camunda/camunda-platform -f var/camunda-platform-core-kind-values.yaml
(taken from : Camunda Helm charts | Camunda Platform 8)

I have less pods (no optimize, identity, keycloak), but they seems stable.

Question :
Can this server can be reachable with others computers (developers building service) without the identity & keycloak pods ? In documentation it says it’s for ‘local dev’ so i’m not sure. The goal is to deploy a remote server to play with as developer.