Camunda operate - Elasticsearch 'Connection refused'. Why?

mkelton · September 23, 2024, 6:31pm

Trying to bring up test Camunda 8 installation.
I have docker containers for Zeebe and Operate.

When I start Operate I get message:

Retry Operation Connect to Elasticsearch cluster [elasticsearch] at http://localhost:9200 failed: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused

Sure, Elasticsearch (http://localhost:9200) is running. Elasticsearch accepts connection from any host (network.host: 0.0.0.0).
What is wrong there? Why ‘refused’?
I tried as well Elasticsearch as docker container - same result.
No connection tries in Elasticsearch log so ‘refused’ is not really true.

GotnOGuts · September 23, 2024, 6:59pm

Configuring Operate to have an ElasticSearch at localhost /shouldn’t/ work. The only thing in the Operate container is Operate. ElasticSearch should be in its own container.

You’re on the right track with pointing Operate at the container for ElasticSearch, but you have to make sure that they both have the same networks in the Docker files. You can see this in the example Docker Compose | Camunda 8 Docs file where both opensearch and operate are on the camunda-platform network

You could try opening a terminal on the operate container and use wget or curl to check what it can reach (I think both are in the image)

mkelton · September 23, 2024, 7:39pm

Well, i modified CAMUNDA_OPERATE_ELASTICSEARCH_URL to point ‘http://host.docker.internal:9200’ what was reall address. Operate could connect to it but in next seconds i could not.

OK, I started Elasticsearch as docker container. Running.

|name|3cd25c51dc9d|
|---|---|
|cluster_name|elasticsearch|
|cluster_uuid|JTK252uLTmumTXDIU6q84w|
|version||
|number|7.17.24|
|build_flavor|default|
|build_type|docker|
|build_hash|fcf25fff740db6ab3ed5d145c58d70e4c3528ea7|
|build_date|2024-09-05T07:34:51.812485320Z|
|build_snapshot|false|
|lucene_version|8.11.3|
|minimum_wire_compatibility_version|6.8.0|
|minimum_index_compatibility_version|6.0.0-beta1|
|tagline|You Know, for Search|

But Operate says:
Connect to Elasticsearch cluster [elasticsearch] at http://localhost:9200 failed: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused

mkelton · September 23, 2024, 7:53pm

I’m not Docker expert but looks like this Operate image has totally bad configuration. Docker containter does not see ‘localhost’ address. The only one that is responding from inside containter is ‘http://host.docker.internal:9200’

Localhost:

$ wget -qO- http://localhost:9200
wget: can't connect to remote host: Connection refused

host.docker.internal:9200:

~ $ wget -qO- http://host.docker.internal:9200
{
  "name" : "3cd25c51dc9d",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "JTK252uLTmumTXDIU6q84w",
  "version" : {
    "number" : "7.17.24",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "fcf25fff740db6ab3ed5d145c58d70e4c3528ea7",
    "build_date" : "2024-09-05T07:34:51.812485320Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.3",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

But if I start Operate with variable ‘CAMUNDA_OPERATE_ELASTICSEARCH_URL=http://host.docker.internal:9200’ it connects initially to Elasticsearch and… ‘Connection refused’

GotnOGuts · September 23, 2024, 8:04pm

This is EXPECTED to not work.
There is no ElasticSearch / OpenSearch in the Operate container image.

I would not use host.docker.internal:9200 either, but rather point it to the OpenSearch container directly.

If it connects initially, then it should just work, but there might be something strange going on from trying to go out of the container environment and back in again, then on the reply going back outside again and back in.

mkelton · September 24, 2024, 8:03am

I do not get it. If there’s no connection to ‘localhost’ why some guy configured software to use this url?
‘OpenSearch container’? Camunda states it works with Elasticsearch. Looks like it does not. I tried two different versions of Elasticsearch (7 and 8), I installed it directly in OS, tried as docker contatainer. It does not work and this is not ES problem but Camunda one.

GotnOGuts · September 24, 2024, 3:22pm

I can’t speak to how you’ve got things configured.

Try the docker-compose file that I linked to earlier. In it, you can see that Operate is told to connect to the opensearch containter.

A lot of people struggle with making the transition to Docker, and they keep wanting to use localhost, because when they are working on their local machine, that’s what they are used to. Each Docker container is its own VM, with a distinct IP address. There is no Elasticsearch nor Opensearch in the Operate container (VM) - so it CANNOT be set to localhost.

I’m doing what I can to assist you - but I’m just another user of the software.

cpbpm · September 24, 2024, 4:23pm

Would you mind sharing few more details, so that we can help you to onboard on Camunda 8 journey.

What is your Host OS?
Do you like to try out Camunda 8 with docker-compose, it’s bundled for all the components. Make sure that you downloaed .env file which maps the hostname. By default it’s localhost.
You can update the hostname and keycloak hostname with system qualified name, and bring up the Camunda 8 platform.
If you download docker container for each image differently, it may work or may not work. Try to use the same version and repo mention in the docker-compose or docker-compose-core file. These are tested and it works as expected.

mkelton · September 25, 2024, 10:19am

Hi, thanks for reply.
My test environment is Windows 11 OS.
Journey with Camunda starts at the first level; download software. No clear at all where is the official repository. Camunda Download Hub | Camunda provides link to Camunda… 7. Not even single word about version 8.
So browsing official Camunda site I have no idea where to find some link to download software. No chance. So normally this is the point where most of people would finish Camunda journey; Camunda 8 is only delivered as SaS (no single word that in can be installed in datacenter)
I spent some time with google and found that Camunda offers ‘self-managed’ edition.
But where’s the repository? Google points me to Camunda 8 Self-Managed | Camunda 8 Docs page. But even there no info if I can istall it as docker image or better as standalone software (only Kubernetes). So, maybe camunda publish something on Github? Google says yes:
GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config
So it should be easy, just use docker-compose-core.yaml, right? No way, docker says this file has errors.
OK, let’s search for docker images on docker site. Bingo; there’s something https://hub.docker.com/search?q=camunda
Should be easy. So I download components, start containers but… there’s no communication between them. Looks like Zeebe is OK but Operate cannot communicate with Elasticsearch.

Anyway, what is the correct way to make this software is installed and working?
Regards

cpbpm · September 25, 2024, 11:02am

Sorry to hear that, your Camunda journey is not going well.

If you access camunda.com, has all the details. Looks like you missed the flow.

Camunda.com - Resources - Develop- Documentation - Camunda 8 Self Managed - Setup

Hope this helps.

mkelton · September 25, 2024, 12:06pm

Thank you but this is what I mentioned in my post.
That link takes me to Docker | Camunda 8 Docs and this site takes me to https://hub.docker.com/search?q=camunda
I downloaded that images but they do not work.

cpbpm · September 25, 2024, 12:53pm

I would advise you to follow the steps mentioned in the earlier post. If you select Docker compose link on the left side, it will show the details. In that page, you need to select the repository link, this will take into the page where all the information is available.

github.com

camunda/camunda-platform/blob/f13517b3450d8dbb9d0442f52615275ccea89e97/docker-compose.yaml

# While the Docker images themselves are supported for production usage,
# this docker-compose.yaml is designed to be used by developers to run
# an environment locally. It is not designed to be used in production.
# We recommend to use Kubernetes in production with our Helm Charts:
# https://docs.camunda.io/docs/self-managed/platform-deployment/kubernetes-helm/
# For local development, we recommend using KIND instead of `docker-compose`:
# https://docs.camunda.io/docs/self-managed/platform-deployment/helm-kubernetes/guides/local-kubernetes-cluster/

# This is a full configuration with Zeebe, Operate, Tasklist, Optimize, Identity, Keycloak, and Elasticsearch
# See docker-compose-core.yml for a lightweight configuration that does not include Optimize, Identity, and Keycloak.

services:
  init: # recommended config https://opensearch.org/docs/1.1/opensearch/install/important-settings/
    image: bash
    profiles: ["opensearch"]
    privileged: true
    user: root
    command: [ "sysctl", "-w", "vm.max_map_count=262144" ]

  zeebe: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#zeebe

This file has been truncated. show original

mkelton · September 27, 2024, 9:50am

I decided to switch to Kubernetes. I was able to install software.

nathan.loding · September 27, 2024, 6:50pm

Hi @mkelton - sorry you found your initial foray into installing Camunda 8 confusing. I think you have some very good feedback for our teams already, but if you don’t mind spending a few additional minutes answering some questions, I’d love to get even more feedback. There’s always room for improvement - clearly there’s room for improvement here! - and I’d love to get your thoughts on what would have been helpful. (If you’d prefer to answer privately, send me a DM/chat on the forum!)

If you had to name one thing that would have been the most helpful when starting out, what would that be?
You mentioned that you started in the Download Hub, but it only had Camunda 7. Now that you have a working installation, what would have been helpful to see on that page?
Did you have any prior experience with Camunda (either Camunda 7 or using an already installed/deployed version of Camunda 8)?
Did you have any prior experience with Docker (or containers) before trying to start Camunda 8?
In your initial post, were you trying to build your own Docker environment, or were you using the Docker Compose environment linked later in the thread?
The docker-compose-core.yaml file you link (here) works for me out of the box, but you do also need to include the .env file in the repository also. Do you recall if you had the .env file, or what the errors were?

It’s absolutely OK if you don’t want to spend additional time answering these questions - but I can promise this feedback is going directly into the hands of our documentation and product teams, and will be used to help improve the Camunda experience!

Also, I want to share that a revamped Download Hub page is scheduled to be released with version 8.6 on October 8, that will contain more information about Camunda 8 and how to run it. Perhaps that new page solves all the headaches?

mkelton · September 28, 2024, 6:38pm

Thanks for your reply; please give me some time to respond.
Best

system · October 5, 2024, 6:38pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.