Camunda optimize elastic search issue when using docker compose

Camunda 8 Topics Discussion & Questions
1

Hi, I am using below mentioned docker-compose.yaml to start camunda optimize. Elastic search instance is created and verified localhost:9200 is accessible. But, optimize container is unable to access elastic search host. I tried CLI in optimize container but no luck. Not sure why it is unable to connect to ES. But, while running optimize-demo.sh, I am able to start optimize. Issue is only while using docker-compose.yaml.

docker-compose.yaml:
version: ‘3’
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.0
container_name: elasticsearch
hostname: elasticsearch
environment:
- cluster.routing.allocation.disk.threshold_enabled=true
- cluster.routing.allocation.disk.watermark.low=1024mb
- cluster.routing.allocation.disk.watermark.high=512mb
- cluster.routing.allocation.disk.watermark.flood_stage=256mb
- cluster.name=elasticsearch
- bootstrap.memory_lock=true
- discovery.type=single-node
- action.auto_create_index=false
- transport.host=0.0.0.0
- http.host=0.0.0.0
- “ES_JAVA_OPTS=-Xms512m -Xmx512m”
- client.transport.sniff=false
ports:
- “9200:9200”
mem_limit: 1g
networks:
- optimize
healthcheck:
test: [“CMD-SHELL”, “curl --silent --fail localhost:9200/_cluster/health || exit 1”]
interval: 10s
timeout: 10s
retries: 3
optimize:
image: registry.camunda.cloud/optimize-ee/optimize:3.7.2
container_name: optimize
hostname: optimize
environment:
- JAVA_OPTS=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
- OPTIMIZE_CAMUNDABPM_REST_URL=http://localhost:8080/engine-rest
- OPTIMIZE_ELASTICSEARCH_HOST=elasticsearch
volumes:
- ./environment-config.yaml:/optimize/config/environment-config.yaml
- ./custom-config.yaml:/optimize/config/custom-config.yaml
- ./OptimizeLicense.txt:/optimize/config/OptimizeLicense.txt
- ./engine-rest-plugin-jar-with-dependencies.jar:/optimize/plugin/engine-rest-plugin-jar-with-dependencies.jar
restart: always
ports:
- “8090:8090”
mem_limit: 2g
networks:
- optimize
depends_on:
- elasticsearch
networks:
optimize: {}

Error:

01:13:23.175 [main] ERROR o.c.o.s.e.OptimizeElasticsearchClientFactory - Failed getting number of cluster nodes.

java.net.ConnectException: Connection refused

at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:865)

at org.elasticsearch.client.RestClient.performRequest(RestClient.java:275)

``

at org.elasticsearch.client.RestClient.performRequest(RestClient.java:262)

at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1628)

at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1598)

at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1568)

at org.elasticsearch.client.ClusterClient.health(ClusterClient.java:130)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientFactory.getNumberOfClusterNodes(OptimizeElasticsearchClientFactory.java:81)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientFactory.waitForElasticsearch(OptimizeElasticsearchClientFactory.java:59)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientFactory.create(OptimizeElasticsearchClientFactory.java:41)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientConfiguration.createOptimizeElasticsearchClient(OptimizeElasticsearchClientConfiguration.java:35)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientConfiguration.optimizeElasticsearchClient(OptimizeElasticsearchClientConfiguration.java:30)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientConfiguration$EnhancerBySpringCGLIB$10d39711.CGLIB$optimizeElasticsearchClient$0(<generated>)

at org.camunda.optimize.service.es.OptimizeElasticsearchClientConfiguration$EnhancerBySpringCGLIB$10d39711$FastClassBySpringCGLIB$d04e4231.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)

at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)

``

at org.camunda.optimize.service.es.OptimizeElasticsearchClientConfiguration$EnhancerBySpringCGLIB$10d39711.optimizeElasticsearchClient(<generated>)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:566)

at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)

at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)

at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:638)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)

at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)

at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)

at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)

at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)

at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)

at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)

at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)

at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)

at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)

``

at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)

at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)

at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)

at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)

at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)

at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)

at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1067)

at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)

at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:996)

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:910)

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)

at org.eclipse.jetty.server.Server.start(Server.java:423)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)

``

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)

at org.eclipse.jetty.server.Server.doStart(Server.java:387)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)

at org.camunda.optimize.jetty.EmbeddedCamundaOptimize.startOptimize(EmbeddedCamundaOptimize.java:245)

at org.camunda.optimize.Main.main(Main.java:27)

Caused by: java.net.ConnectException: Connection refused

at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)

at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)

at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)

at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)

at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)

at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)

at java.base/java.lang.Thread.run(Thread.java:829)

01:13:23.175 [main] INFO o.c.o.s.e.OptimizeElasticsearchClientFactory - No Elasticsearch nodes available, waiting [30000] ms to retry connecting

2

Hi @Techie ,
Could you post what’s in your environment-config.yaml, specifically the elasticsearch configurations?

3

Env-config:

security:
auth:
superUserIds: []
api:
accessToken: ${OPTIMIZE_API_ACCESS_TOKEN:null}
container:
host: 0.0.0.0
ports:
http: 8090
https: 8091
accessUrl: null
engines:
‘camunda-bpm’:
name: default
defaultTenant:
id: null
name: null
rest: ‘http://localhost:8080/engine-rest’
importEnabled: true
eventImportEnabled: false
authentication:
enabled: false
password: ‘’
user: ‘’
webapps:
endpoint: ‘http://localhost:8080/camunda’
enabled: true
es:
connection:
nodes:
- host: ‘localhost’
httpPort: 9200
proxy:
enabled: false
host: null
port: null
sslEnabled: false
security:
username: null
password: null
ssl:
certificate: null
certificate_authorities: []
enabled: false
settings:
index:
prefix: ‘optimize’
number_of_replicas: 1
number_of_shards: 1
import:
data:
dmn:
enabled: true
user-task-worker:
enabled: true
metadata:
includeUserMetaData: true
cronTrigger: ‘0 */3 * * *’
identitySync:
includeUserMetaData: true
cronTrigger: ‘0 */2 * * *’
serialization:
engineDateFormat: yyyy-MM-dd’T’HH:mm:ss.SSSZ
plugin:
variableImport:
basePackages: []
engineRestFilter:
basePackages: [“com.example.optimize.enginerestplugin”]
authenticationExtractor:
basePackages: []
elasticsearchCustomHeader:
basePackages: []
email:
enabled: false
address: ‘’
hostname: ‘’
port: 587
authentication:
enabled: true
username: ‘’
password: ‘’
securityProtocol: ‘NONE’
webhookAlerting:
webhooks:
sharing:
enabled: true
historyCleanup:
cronTrigger: ‘0 1 * * *’
ttl: ‘P2Y’
processDataCleanup:
enabled: false
cleanupMode: ‘all’
perProcessDefinitionConfig:
decisionDataCleanup:
enabled: false
perDecisionDefinitionConfig:
ingestedEventCleanup:
enabled: false
locales:
availableLocales: [‘en’, ‘de’]
fallbackLocale: ‘en’
ui:
header:
textColor: ‘dark’
pathToLogoIcon: ‘logo/camunda_icon.svg’
backgroundColor: ‘#FFFFFF’
logoutHidden: false
eventBasedProcess:
authorizedUserIds: []
authorizedGroupIds: []
eventImport:
enabled: false
telemetry:
initializeTelemetry: false

4

Hi @Techie

es:
  connection:
    nodes:
      host: ‘localhost’

This section of your config probably overwrites the environement variable setting OPTIMIZE_ELASTICSEARCH_HOST to elasticsearch. If you would like to use this environment variable, the config should say something like host: ${OPTIMIZE_ELASTICSEARCH_HOST:localhost} which will only default to localhost if no environment variable is set.
The same applies to OPTIMIZE_CAMUNDABPM_REST_URL and the responding engine config section.
The rest of the config looks like the default to me, I would recommend you only leave those configuration sections in your environment-config file that differ from the default, the rest you can remove. Hope that helps!

1 Like
5

@Helene Thanks for the quick reply . I have modified my yaml as you suggested. This time I am getting error while connecting to camunda rest engine. This is the config i used:

Env-config:

engines:
‘camunda-bpm’:
name: default
defaultTenant:
id: null
name: null
rest: ${OPTIMIZE_CAMUNDABPM_REST_URL:http://localhost:8080/engine-rest}
importEnabled: true
eventImportEnabled: false
authentication:
enabled: true
password: ‘demo’
user: ‘demo’
webapps:
endpoint: ${OPTIMIZE_CAMUNDABPM_WEBAPPS_URL:http://localhost:8080/camunda}
enabled: true

es:
connection:
nodes:
- host: ${OPTIMIZE_ELASTICSEARCH_HOST:localhost}
httpPort: 9200
proxy:
enabled: false
host: null
port: null
sslEnabled: false
timeout: 10000
responseConsumerBufferLimitInMb: 100

docker-compose.yaml

version: ‘3’
services:
optimize:
image: registry.camunda.cloud/optimize-ee/optimize:3.7.2
container_name: optimize
hostname: optimize
depends_on:
- elasticsearch
environment:
- JAVA_OPTS=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
- OPTIMIZE_CAMUNDABPM_REST_URL=http://localhost:8080/engine-rest
- OPTIMIZE_CAMUNDABPM_WEBAPPS_URL:http://localhost:8080/camunda
- OPTIMIZE_ELASTICSEARCH_HOST=elasticsearch
- OPTIMIZE_ELASTICSEARCH_PORT=9200
volumes:
- ./environment-config.yaml:/optimize/config/environment-config.yaml
- ./OptimizeLicense.txt:/optimize/config/OptimizeLicense.txt
restart: always
mem_limit: 2g
networks:
- optimize

elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.0
container_name: elasticsearch
environment:
- cluster.routing.allocation.disk.threshold_enabled=true
- cluster.routing.allocation.disk.watermark.low=1024mb
- cluster.routing.allocation.disk.watermark.high=512mb
- cluster.routing.allocation.disk.watermark.flood_stage=256mb
- cluster.name=elasticsearch
- bootstrap.memory_lock=true
- discovery.type=single-node
- action.auto_create_index=false
- transport.host=0.0.0.0
- http.host=0.0.0.0
- “ES_JAVA_OPTS=-Xms512m -Xmx512m”
- client.transport.sniff=false
mem_limit: 1g
networks:
- optimize
networks:
optimize: {}

Error stack:

03:51:51.287 [EngineImportScheduler-1] ERROR o.c.o.s.i.e.f.instance.TenantFetcher - Error during fetching of entities. Please check the connection with [camunda-bpm]!
org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn’t respond. Can not verify this engine’s version

03:52:21.257 [EngineImportScheduler-1] ERROR o.c.o.rest.engine.EngineContext - Failed to validate engine camunda-bpm version with error message: Engine didn’t respond. Can not verify this engine’s version
org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn’t respond. Can not verify this engine’s version

6

Hi @Techie

I can’t see any engine setup in your docker compose, are you sure your engine is running?

Here you’re setting the engine rest URL to the Optimize container’s localhost, which is probably not the correct one. Though this depends on whether and where your engine is really running.

Have a look at this docker-compose example, especially the cambpm section and how the OPTIMIZE_CAMUNDABPM_REST_URL is set in accordance to that.

If you have a license I would also like to encourage you to reach out to our support team, they’ll be able to give you some more hands on support with your setup.

7