Hello Community
In our Camunda based application (deployed as containers in k8s) , we have a need to create the process definition dynamically based on the composition of the input request. We have achieved this via Fluent Builder API. Functionally, we can create the dynamic process definition and create process instance based on the definition.
But when the number of process definitions increase to a higher value (say 5000), Camunda pods gets killed by returning database connection error.
Caused by: org.apache.ibatis.exceptions.PersistenceException:
Error querying database. Cause: java.sql.SQLTransientConnectionException: HikariPool-2 - Connection is not available, request timed out after 30001ms.
The error may exist in org/camunda/bpm/engine/impl/mapping/entity/Job.xml
The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.selectNextJobsToExecute
The error occurred while executing a query
SQL: select RES.ID_, RES.REV_, RES.DUEDATE_, RES.PROCESS_INSTANCE_ID_, RES.EXCLUSIVE_ from ACT_RU_JOB RES where (RES.RETRIES_ > 0) and ( RES.DUEDATE_ is null or RES.DUEDATE_ <= ? ) and (RES.LOCK_OWNER_ is null or RES.LOCK_EXP_TIME_ < ?) and RES.SUSPENSION_STATE_ = 1 and ( ( RES.EXCLUSIVE_ = true and not exists( select J2.ID_ from ACT_RU_JOB J2 where J2.PROCESS_INSTANCE_ID_ = RES.PROCESS_INSTANCE_ID_ – from the same proc. inst. and (J2.EXCLUSIVE_ = true) – also exclusive and (J2.LOCK_OWNER_ is not null and J2.LOCK_EXP_TIME_ >= ?) – in progress ) ) or RES.EXCLUSIVE_ = false ) LIMIT ? OFFSET ?
We have explored all the database configurations from the Camunda side but still not able to resolve.
If we delete the Dynamic process definition, immediately after execution of the process instance. The system looks stable but we will loose the history view in the Camunda cockpit.
Does anyone know the limit of the number of process definitions?
Is there a way to overcome this connection error?
Thanks in advance
Mike