Memory consumption when deleting history data

Camunda Platform 7 Topics Camunda Platform 7 Process Engine
#1

Hi,

we had an incident with our Camunda engine yesterday. Tomcat ran out of heap while deleting a huge amount of historic process instances.
Does anybody have experience what the expected behavior is? In our case the list contained about 37k entries. We saw Tomcat running OOM after executing about 30% of the deletions. There was nothing else going on on the engine at this point of time.

The code behind was basically very simple:

    instancesInScope = historyService.createHistoricProcessInstanceQuery().finished().finishedBefore(minDeletionDate).list()
    instancesInScope.each {
            historyService.deleteHistoricProcessInstance(it.id)
    }

Thanks,
Werner

#2

Hi @wke-rec,

we will have batch deletion of historic process instances in the next alpha release of 7.6.0. Corresponding method signature will be deleteHistoricProcessInstancesAsync. Would that help you?

Cheers,
Askar

#3

Hi Askar,

this is great news. We just have been caught by surprise by running into this issue. We can surely chunk this kind of work while waiting for the 7.6 release. Is there any number you can suggest knowing what is happening in the background? We have been thinking about no more than 5k deletions at a time.

Thanks,
Werner

#4

Hi @wke-rec,

we don’t have any recommendation for exact number. You will have to experiment a bit and find out a proper threshold on your own.

Cheers,
Askar