I’ve enabled auto deletion of instances whose journey is completed by configuring TTL of 3 days. Now auto delete is working fine but the problem is, there is huge data(around 1.4 crores instances) residing in the database to be deleted and I run auto delete continuously for 8 long hours but the deletion process is so slow that even 8 hours is not enough to do the cleanup.
Is there any way I can optimise the delete process either by making configurational changes in camunda or by writing own delete stored procedures.
Camunda history cleaning is performed through jobs and batch sizes.
In other words, the cleaning speed will depend on how many jobs you configure to run the cleaning, and the size of the batches that you will execute at a time in each job.
For example, if you set 2 jobs to perform cleaning, and set the batch size to 500, you will have 1000 cleaning executions in parallel.
But you need to be aware that the jobs used to execute the cleaning are the same jobs that the Camunda engine uses to execute time-triggered activities (timer events, activity retries…).
Cleaning speed also depends on your database configuration and resources.
Anyway, I suggest taking a look at this official Camunda documentation, which will help you better understand what I explained above, and also how to configure this.