Operate showing no process instances

Camunda 8 Topics
,
1

Hey,

I’m trying to get Operate working in my local docker compose development setup but process instances are not shown, neither on the dashboard nor on the processes tab (with “finished instances” enabled). Process definitions are shown correctly.

I’m using the latest docker compose file with the Camunda images on version 8.7.1 and Elasticsearch on 8.17.5 and added Kibana 8.17.5 to it to analyse the indices. My basic setup only uses the components defined in the docker-compose-core.yaml but I also tried to use the full setup and set resource authorizations in Identity for the Operate user. I have a separate compose file to deploy BPMNs to Zeebe and custom index templates to Elasticsearch.

Our application monitors Elasticsearch for updates on the process instances so I can see in the application and in Kibana that the process instances are created correctly. Also I can see that the “operate_” prefixed indices are created correctly in Elasticsearch.

The Operate logs contain some warnings concerning the connection to Elasticsearch during startup and Elasticsearch related deprecation warnings but nothing else. These warnings also happen when importing Zeebe indices, so the connection between Operate and Elasticsearch seems to work fine:

operate  |      org.elasticsearch.client.RestClient - request [POST http://elasticsearch:9200/zeebe-record-process/_search?routing=1&typed_keys=true&max_concurrent_shard_requests=5&request_cache=false&search_type=query_then_fetch&batched_reduce_size=512] returned 4 warnings: [299 Elasticsearch-8.17.5-55fce5db33c28431e3a504c47d359ddbeadde69d "Deprecated field [from] used, this field is unused and will be removed entirely"],[299 Elasticsearch-8.17.5-55fce5db33c28431e3a504c47d359ddbeadde69d "Deprecated field [to] used, this field is unused and will be removed entirely"],[299 Elasticsearch-8.17.5-55fce5db33c28431e3a504c47d359ddbeadde69d "Deprecated field [include_lower] used, this field is unused and will be removed entirely"],[299 Elasticsearch-8.17.5-55fce5db33c28431e3a504c47d359ddbeadde69d "Deprecated field [include_upper] used, this field is unused and will be removed entirely"]

In our test system the Camunda stack is deployed with helm using the chart version 12.0.0, having the same issue.

Thank you!

2

Try cleaning the elasticsearch and zeebe volumes and deleting their containers and recreating them, when this happened to me, just doing this solved it!

3

I tried that a couple of times, since that solved problems with upgrades before but it didn’t help.

4

It seems like the problem is caused by my custom index templates. When I prune all volumes and start the setup without pushing the templates, operate works fine. I’m now figuring out how to fix the templates.

5

Fixing the index templates fixed the problem. We use a value.variables field to filter our queries, so the field needs to be mapped as keyword.
Fetching the initial index template for the process instance creation index
http://localhost:9200/_index_template/zeebe-record_process-instance-creation_8.7.1
adding the necessary field and using the resulting template, enables operate to work properly.