Data Cleanup

[Rabeb_Abid]

Hi everyone,

I’m currently working with Camunda 8.7 (self-managed) and I’m looking for a proper way to automatically delete old completed process instances, along with their related Elasticsearch indexes.

Specifically, I want to:

1. Clean up Zeebe data from completed process instances to free up disk space.
2. Remove the corresponding Elasticsearch indexes (e.g., zeebe-*, operate-*) to reduce storage usage and improve search performance.

I understand that it’s possible to configure zeebe.data.retentionPeriod, but it seems this only affects Zeebe logs and not the data stored in Elasticsearch (used by Operate, Tasklist, Optimize, etc.).

So my question is:

What is the recommended approach to clean up both completed process instances in Zeebe and their corresponding Elasticsearch indexes?
Is there a best practice for this, or maybe ILM policy examples/scripts?

Thanks in advance for your help!

[jonathan.lukas]

Hello @Rabeb_Abid ,

data retention is a feature native to the platform.

Camunda uses the Elasticsearch ILM to remove old data.

This can be configured in:

• the Zeebe Elasticsearch exporter
• the Operate archiver
• the Tasklist archiver
• Optimize

If you deploy your platform using our helm chart, most of this is available directly from the values.yaml:

zeebe:
  retention:
    enabled: true
    minimumAge: 30d # elasticsearch formatted duration
operate:
  retention:
    enabled: true
    minimumAge: 30d # elasticsearch formatted duration
tasklist:
  retention:
    enabled: true
    minimumAge: 30d # elasticsearch formatted duration
optimize:
  env:
  - name: CAMUNDA_OPTIMIZE_HISTORY_CLEANUP_TTL
    value: P1Y # ISO 8601 expression
  - name: CAMUNDA_OPTIMIZE_HISTORY_CLEANUP_PROCESS_DATA_CLEANUP_ENABLED
    value: "true"
  - name: CAMUNDA_OPTIMIZE_HISTORY_CLEANUP_DECISION_DATA_CLEANUP_ENABLED
    value: "true"

I hope this helps

Jonathan