Best way to Mass/Bulk Delete 6K Process Instances

Ashley · September 23, 2021, 5:49pm

Hey guys,

We had a mistake and accidentally created 6,000 instances of a process.

I’ve tried deleting the deployment but that never works unless there is a small amount of instances. It will not clean up 6,000 instances.

I’ve tried deleting the API Deployment Delete method to create a batch as in the post below, but that is not working. The batch never makes progress.

Deleting process instances asynchronously using processinstancequery - Process Engine - Camunda Platform Forum

Does anyone have a few moments to help a lady in distress?

hassang · September 23, 2021, 6:44pm

Hi @Ashley,

What version do you have?

You can try below REST call to do async deletion of running instances
https://docs.camunda.org/manual/7.15/reference/rest/process-instance/post-delete-historic-query-based/

And below REST call to do async deletion of historic process instances
https://docs.camunda.org/manual/7.15/reference/rest/history/process-instance/post-delete/

Ashley · September 23, 2021, 7:27pm

My apologies for not putting that I have the current v7.15.

As I mentioned in the original post, I tried the REST call method, but the batch never seems to make any progress and just seems to hang in limbo.

I do have to admit I did not use the “historic query based” option but have tried it now and it does create a batch as I mentioned in my original post but it never seems to make progress.

Here is a screenshot;

hassang · September 23, 2021, 7:43pm

Hi @Ashley,

To be honest, I never had the chance to try any of these calls but can you please make sure that job executor is activated in your case.

https://docs.camunda.org/manual/7.15/user-guide/process-engine/the-job-executor/#job-executor-activation

Ashley · September 23, 2021, 8:02pm

So on further testing, I thought maybe the batch was to big so I set the data of this history query to something earlier.

The batches seem to run at that point as you can see here;

However, when this batch runs and completes, I submit the exact same REST call and get the exact same results again. It does not seem to be deleting the instances even though it claims to.

Here are 2 completed jobs with exactly the same results.

Notice the Start Times are different but the TotalJobs is exactly the same.

So basically it shows to be deleting but is not.

hassang · September 23, 2021, 8:33pm

Hi @Ashley

Then can you try to set the removal time to absolute value (current date/time or around) to check whether below call works for historical instances.

https://docs.camunda.org/manual/7.15/reference/rest/history/process-instance/post-process-instance-set-removal-time/

Ashley · September 23, 2021, 9:45pm

Ok so setting the removal time worked to clear out the ones that were not deleted, but as I work my way up the dates deleting small batches we finally gets to the place in my original post where nothing happens.

Here is the next small batch of only 86 that should take a few seconds based on the 1000’s I’ve deleted already but it just sits and does nothing.

No Failed Jobs, No Updates, No Timeout, No Progress…

Artem_Smirnov · September 23, 2021, 11:10pm

Iam deleting lots of instances via Java API, it tooks like few minutes to async delete 10k of em.

Ashley · September 23, 2021, 11:33pm

Yes, I aware it can take a bit of time for large amounts. Some of my batches are only a single process instance.

@hassang

UPDATE:

I keep deleting small batches and narrowed it down to the single instance that is causing the problems.

It looks normal in the database and when I try to set the removeal time and delete the single instance it shows to work in the batch screen, but nothing happens.

Can I remove this record in the act_hi_procinst table manually? Would that cause orphan records?