Unable to delete process instances

Hi,

I am trying to delete old instances which are filling up the cockpit.
I try via cockpit, via the rest API and via the rest api with historic instance to delete the instances.
They don’t want to be deleted? Even if I descend to the child level and try to delete the child proces first it won’t work. Deleting child processes gives me back the error that there is no activityinstanceid? Deleting the top level process instance gives back a stack trace with inconsistencies in the database:

Version of Camunda is 7.11 and it is running as spring boot application.
Database is sqlserver and driver is jtds

stack traces:

at ...	
Caused by: org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Exception while executing Batch Database Operations with message '
### Error flushing statements.  Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution (batch index #1) failed. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
### Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution (batch index #1) failed. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
java.sql.SQLException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_PARENT". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'PARENT_ID_'.
java.sql.SQLException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_PARENT". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'PARENT_ID_'.
java.sql.SQLException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_PROCINST". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'PROC_INST_ID_'.
'. Flush summary: 
 [
  DELETE ExecutionEntity[f63f2ceb-5a54-11eb-a4d3-005056a3d984]
  DELETE ExecutionEntity[f63f2ce3-5a54-11eb-a4d3-005056a3d984]
  DELETE ExecutionEntity[bf833843-5667-11eb-a4d3-005056a3d984]
  DELETE ExecutionEntity[ba46bbfb-5667-11eb-a4d3-005056a3d984]
  UPDATE HistoricActivityInstanceEventEntity[f63f2ce4-5a54-11eb-a4d3-005056a3d984]
  UPDATE HistoricProcessInstanceEventEntity[ba46bbfb-5667-11eb-a4d3-005056a3d984]
]
	at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationsException(EnginePersistenceLogger.java:693)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:362)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:325)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:297)
	at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:208)
	at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:137)
	at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:116)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:46)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:44)
	at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
	at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
	at org.camunda.bpm.engine.impl.RuntimeServiceImpl.deleteProcessInstance(RuntimeServiceImpl.java:219)
	at org.camunda.bpm.engine.rest.sub.runtime.impl.ProcessInstanceResourceImpl.deleteProcessInstance(ProcessInstanceResourceImpl.java:71)
	... 80 more
Caused by: org.apache.ibatis.exceptions.PersistenceException: 
### Error flushing statements.  Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution (batch index #1) failed. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
### Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution (batch index #1) failed. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
	at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:255)
	at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.flushOperations(DbSqlSession.java:90)
	at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:357)
	... 92 more
Caused by: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution (batch index #1) failed. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
	at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:146)
	at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:129)
	at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
	at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
	... 94 more
Caused by: java.sql.BatchUpdateException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "ACT_FK_EXE_SUPER". The conflict occurred in database "CamundaDB", table "dbo.ACT_RU_EXECUTION", column 'SUPER_EXEC_'.
	at net.sourceforge.jtds.jdbc.JtdsStatement.executeBatch(JtdsStatement.java:1069)
	at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:128)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
	at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
	... 98 more

2021-02-16 12:18:31.612  WARN 3704 --- [io-8111-exec-10] o.a.cxf.ws.addressing.soap.MAPCodec      : Response message does not contain WS-Addressing properties.  Not correlating response.
2021-02-16 12:18:31.612  WARN 3704 --- [io-8111-exec-10] o.apache.cxf.ws.addressing.ContextUtils  : WS-Addressing - failed to retrieve Message Addressing Properties from context
2021-02-16 12:18:42.340  WARN 3704 --- [nio-8111-exec-8] ExceptionHandler                         : org.camunda.bpm.engine.rest.exception.InvalidRequestException: Process instance with id f63f2cf6-5a54-11eb-a4d3-005056a3d984 does not exist
	at org.camunda.bpm.engine.rest.sub.runtime.impl.ProcessInstanceResourceImpl.deleteProcessInstance(ProcessInstanceResourceImpl.java:78)
	at jdk.internal.reflect.GeneratedMethodAccessor1259.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:183)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at be.aquafin.bpm.filter.CsrfEmptyFilter.doFilter(CsrfEmptyFilter.java:28)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at be.aquafin.bpm.filter.LoginAuthenticationFilter.lambda$doFilter$1(LoginAuthenticationFilter.java:84)
	at org.camunda.bpm.webapp.impl.security.SecurityActions.runWithAuthentications(SecurityActions.java:44)
	at be.aquafin.bpm.filter.LoginAuthenticationFilter.doFilter(LoginAuthenticationFilter.java:81)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:117)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.camunda.bpm.engine.exception.NotValidException: activityInstanceId is null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.camunda.bpm.engine.impl.util.EnsureUtil.generateException(EnsureUtil.java:381)
	at org.camunda.bpm.engine.impl.util.EnsureUtil.ensureNotNull(EnsureUtil.java:55)
	at org.camunda.bpm.engine.impl.util.EnsureUtil.ensureNotNull(EnsureUtil.java:46)
	at org.camunda.bpm.engine.impl.ProcessInstanceModificationBuilderImpl.cancelActivityInstance(ProcessInstanceModificationBuilderImpl.java:91)
	at org.camunda.bpm.engine.impl.cmd.AbstractDeleteProcessInstanceCmd$1.call(AbstractDeleteProcessInstanceCmd.java:101)
	at org.camunda.bpm.engine.impl.cmd.AbstractDeleteProcessInstanceCmd$1.call(AbstractDeleteProcessInstanceCmd.java:98)
	at org.camunda.bpm.engine.impl.interceptor.CommandContext.runWithoutAuthorization(CommandContext.java:482)
	at org.camunda.bpm.engine.impl.cmd.AbstractDeleteProcessInstanceCmd.deleteProcessInstance(AbstractDeleteProcessInstanceCmd.java:98)
	at org.camunda.bpm.engine.impl.cmd.DeleteProcessInstanceCmd.execute(DeleteProcessInstanceCmd.java:48)
	at org.camunda.bpm.engine.impl.cmd.DeleteProcessInstanceCmd.execute(DeleteProcessInstanceCmd.java:28)
	at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28)
	at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:107)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:46)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:44)
	at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
	at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
	at org.camunda.bpm.engine.impl.RuntimeServiceImpl.deleteProcessInstance(RuntimeServiceImpl.java:219)
	at org.camunda.bpm.engine.rest.sub.runtime.impl.ProcessInstanceResourceImpl.deleteProcessInstance(ProcessInstanceResourceImpl.java:71)
	... 80 more

2021-02-16 12:19:01.541  WARN 3704 --- [io-8111-exec-10] ExceptionHandler                         : org.camunda.bpm.engine.rest.exception.InvalidRequestException: Historic process instance with id f63f2cf6-5a54-11eb-a4d3-005056a3d984 does not exist
	at org.camunda.bpm.engine.rest.sub.history.impl.HistoricProcessInstanceResourceImpl.deleteHistoricProcessInstance(HistoricProcessInstanceResourceImpl.java:64)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:183)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at be.aquafin.bpm.filter.CsrfEmptyFilter.doFilter(CsrfEmptyFilter.java:28)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at be.aquafin.bpm.filter.LoginAuthenticationFilter.lambda$doFilter$1(LoginAuthenticationFilter.java:84)
	at org.camunda.bpm.webapp.impl.security.SecurityActions.runWithAuthentications(SecurityActions.java:44)
	at be.aquafin.bpm.filter.LoginAuthenticationFilter.doFilter(LoginAuthenticationFilter.java:81)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:117)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.camunda.bpm.engine.BadUserRequestException: Process instance is still running, cannot delete historic process instance: HistoricProcessInstanceEntity[businessKey={"projectCode" : "22696","lotNr" : "1","volgNr" : "021","functieCode" : "01","typeCode" : "AANN","bisstaatVolgNr" : "2"}, startUserId=null, superProcessInstanceId=ba46bbfb-5667-11eb-a4d3-005056a3d984, rootProcessInstanceId=ba46bbfb-5667-11eb-a4d3-005056a3d984, superCaseInstanceId=null, deleteReason=null, durationInMillis=null, startTime=Tue Jan 19 13:50:52 CET 2021, endTime=null, removalTime=null, endActivityId=null, startActivityId=StartEvent_1, id=f63f2cf6-5a54-11eb-a4d3-005056a3d984, eventType=null, executionId=null, processDefinitionId=BeheerBeslisProces:3:819b7a67-558a-11eb-b53e-7c5cf8d534ec, processInstanceId=f63f2cf6-5a54-11eb-a4d3-005056a3d984, tenantId=null]: instance.getEndTime() is null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.camunda.bpm.engine.impl.util.EnsureUtil.generateException(EnsureUtil.java:381)
	at org.camunda.bpm.engine.impl.util.EnsureUtil.ensureNotNull(EnsureUtil.java:55)
	at org.camunda.bpm.engine.impl.cmd.DeleteHistoricProcessInstancesCmd.execute(DeleteHistoricProcessInstancesCmd.java:83)
	at org.camunda.bpm.engine.impl.cmd.DeleteHistoricProcessInstancesCmd.execute(DeleteHistoricProcessInstancesCmd.java:42)
	at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28)
	at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:107)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:46)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:44)
	at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
	at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
	at org.camunda.bpm.engine.impl.HistoryServiceImpl.deleteHistoricProcessInstances(HistoryServiceImpl.java:154)
	at org.camunda.bpm.engine.impl.HistoryServiceImpl.deleteHistoricProcessInstance(HistoryServiceImpl.java:146)
	at org.camunda.bpm.engine.rest.sub.history.impl.HistoricProcessInstanceResourceImpl.deleteHistoricProcessInstance(HistoricProcessInstanceResourceImpl.java:57) ```
1 Like

Hi @speetrs

Can you take a look at the Style Guide? It’ll help you understand how to format code correctly and also help you create questions that are easy to understand and also easier to answer :slight_smile:

Niall,

Thanks!
I looked at the Style Guide and changed the post accordingly.
Hopefully, I will get some feedback.

Hi @speetrs

we had a similar Problem that thousands of incidents were piling up and it was impossible to delete the process definition by hand in the cockpit, with similar exceptions you describe here, if I remember correctly.

We did make a appropriate Rest-Call with curl to get the json of all the incidents we wanted to delete, the json into a little piece of code which produced a bash skript with a huge amount of rest-calls to delete the appropriate process instance. You can probably tweak this procedure to your will.

Even though it would be nicer to handle this in the cockpit. I hope this helps,
cnfg

1 Like

Hi @cnfg ,

Tank you for your reply.
My question is a bit long and also contains the rest calls to try to delete the instances.
So that didn’t work.

I found out the cause of the corrupt instances which contain approximately half of the variables.
The application makes use of the Java API and closes tasks of a user by searching for them.
If this is triggered 2 times in less then a second, then the task is not yet completed fully and is found the second time, I made the closing method synchronized which will prevent this from happening.

i will still need to figure out how to delete these corrupt instances, I am guessing this wilhave to be done in the db. All other API ways don’t seem to work. It is strange however that cockpit shows these instances withou being able to delete them in an API fashion.

Best regards,

Soooo how do we delete from ACT_RU_EXECUTION ??? Why on earth would it have 3 foreign key constraints on itself, and the same column, “ID_”?

image

You cannot delete from the table ACT_RU_EXECUTION easily, even if you have deleted from the other tables that reference it. Because the table has constraints on itself. And I have to delete from this this table manually using our DAO factory when our application starts up but before Camunda is booted, because for some reason Camunda does not clear previously running tasks / process instances on boot or provide a configuration setting to allow this, at least in camunda 7 (and possibly the latest version?)

When deleting from the table it gives an error because of the foreign key constraint ACT_FK_EXE_PARENT but not the one above it (ACT_FK_EXE_PROCINST) because of the on delete cascade on update cascade. Shouldn’t all of these constraints on ACT_RU_EXECUTION have this added on delete cascade on update cascade so we can delete from this table and it cascade through the child tables?

Can I add “on delete cascade” to these self-constraints, or will that cause problems?

Hi @Collin_Fox,

you can either use Delete Process Instance | docs.camunda.org

or RuntimeService (Camunda Platform Javadocs 7.18.13-ee)

to get rid of the entries together with their references.

You can find some background information here: Process Variables | docs.camunda.org

Hope this helps, Ingo

1 Like

Unfortunately, when I try to run this code before the process engine has been initialized, it does not work. I have tried both internally clearing the process instances as well as via APIs. It is relaunching the existing tasks before I can run Camunda code to delete process instances to try to prevent tasks from being relaunched that were running before my application was force-stopped / killed.

So, I have to resort to manually deleting from the ru tables (hopefully doing this prevents these lingering tasks from attempting to run after application boot…)

Is there a configuration setting I can use to prevent this relaunch on service boot?? Like clear the process instances on boot via a Camunda property/setting?

Hi @Collin_Fox,

in which state are the processes?

What happens when you restart the engine?

If the process instances continue with async continuation and jobs get executed, you can disable the job executor with jobExecutorActivate set to false.

Hope this helps, Ingo

When I set jobExecutorActivate to false, it is preventing new process instances from being created even those that are not “lingering” tasks from before the restart. Also, this setting does not clear the previous process instances. However, it does do the job of preventing the lingering tasks from running that were there before restarting the application/restarting Camunda.

Is there some other setting I need to tweak?