I have written an api to query elasticsearch and delete deployment document and all the indexes document that contains data related to deployment , the documents have been deleted successfully but still the data of it is showing in operate and Tasklist.
I’m using self managed docker setup for camunda 8.5
Do we need to do any additional configuration in compose file so that it clear any cache or archives that may causing to display data in Operate and Tasklist.
Any idea or sources how to delete deployment data in elasticsearch so that it will be cleared in operate and tasklist as well
Hi @Mukesh_Reddy - I wouldn’t recommend doing manual deletions through Elasticsearch; that would be a fragile process, as the indexes and data within Elastic are subject to change between versions. The data stored in Elasticserach is also “cold” or “read only” data and not used by Zeebe, so you aren’t deleting processes within Zeebe. I would recommend following the procedures described here:
Hey @nathan.loding
Thank you for the response ,In my case we need to delete or cancel processInstances in batch before deleting a version without cancelling or completing , like how we used to do in camunda 7 , so I felt through elasticsearch queries we can achieve that but we are able to delete data in elasticsearch but still the data is present in operate and tasklist is there any way to achieve this .
@Mukesh_Reddy - there are multiple indices (I believe that is the correct term) inside Elasticsearch, and the different components use them differently. For instance, Optimize has its own importer that takes the data exported from Zeebe and re-imports it in a different structure that better suits the analytics Optimize performs. If you are manually removing data from Elasticsearch, you aren’t removing all the relevant records.
Again, I would caution against this approach as future releases may change those structures and your routine will break again; this is an unsupported approach. Additionally, if it’s a running process instance, removing it from Elasticsearch doesn’t remove it from Zeebe, which has it’s own internal datastore on each broker, resulting in orphaned data which can cause issues in the future.