Camunda batch not running in PCF application because acquisition query is not as expected

Hi,
I have a Java SpringBoot application which is deployed in PCF and connects to Azure postgres database. The camunda batch is not working in this app.

When the same app is running in my local machine with the same configuration then the camunda batches are working fine.

So I checked the Camunda acquisition query used in local and PCF and there is a difference but I don’t know why. Could you please tell me how to fix this so that the acquisition query in PCF will be similar to the acquisition query running in my local machine?

The PCF query has an extra check (RES.DEPLOYMENT_ID_ is null or ( RES.DEPLOYMENT_ID_ IN ( ? ) ) ) which checks for the latest deployment ID alone. What should be changed in camunda configuration so that this extra check is not done? In property file I see that camunda deployment-aware is commented out so this should not be in use. Please see below logs where I added the complete query

act_hi_batch table:

id_ type_ total_jobs_ jobs_per_seed_ invocations_per_job_ seed_job_def_id_ monitor_job_def_id_ batch_job_def_id_ tenant_id_ create_user_id_ start_time_ end_time_ removal_time_
2a65dabb-5556-11ed-a49e-eeee0aff11dc instance-migration 1 100 1 2a65dabc-5556-11ed-a49e-eeee0aff11dc 2a65dabd-5556-11ed-a49e-eeee0aff11dc 2a65dabe-5556-11ed-a49e-eeee0aff11dc 2022-10-26 19:46:48.523
c229c8d8-5556-11ed-ae0d-d279f718261b instance-migration 1 100 1 c229c8d9-5556-11ed-ae0d-d279f718261b c229c8da-5556-11ed-ae0d-d279f718261b c229c8db-5556-11ed-ae0d-d279f718261b 2022-10-26 19:51:03.144 2022-10-26 19:51:04.342
logging.level:
  ROOT: OFF
  #org.camunda.bpm.engine.impl.persistence.entity: DEBUG
  org.camunda.bpm.engine.jobexecutor: DEBUG
  org.camunda.bpm.engine.impl.persistence.entity.JobEntity: DEBUG
  org.camunda.bpm.engine.cmd: INFO

When there is a pending batch and local app is started then this is the log (Working scenario):

2022-10-26 22:39:05.102 61532 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : ==>  Preparing: select * from ACT_RU_JOB where HANDLER_TYPE_ = ?
2022-10-26 22:39:05.112 61532 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : ==> Parameters: history-cleanup(String)
2022-10-26 22:39:05.135 61532 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : <==      Total: 1

2022-10-26 22:39:17.803 61532 --- [           main] org.camunda.bpm.engine.jobexecutor       : ENGINE-14014 Starting up the JobExecutor[org.camunda.bpm.engine.spring.components.jobexecutor.SpringJobExecutor].
2022-10-26 22:39:17.806 61532 --- [ingJobExecutor]] org.camunda.bpm.engine.jobexecutor       : ENGINE-14018 JobExecutor[org.camunda.bpm.engine.spring.components.jobexecutor.SpringJobExecutor] starting to acquire jobs

2022-10-26 22:39:17.834 61532 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : ==>  Preparing: select RES.ID_, RES.REV_, RES.DUEDATE_, RES.PROCESS_INSTANCE_ID_, RES.EXCLUSIVE_ from ACT_RU_JOB RES where (RES.RETRIES_ > 0) and ( RES.DUEDATE_ is null or RES.DUEDATE_ <= ? ) and (RES.LOCK_OWNER_ is null or RES.LOCK_EXP_TIME_ < ?) and RES.SUSPENSION_STATE_ = 1 and ( ( RES.EXCLUSIVE_ = true and not exists( select J2.ID_ from ACT_RU_JOB J2 where J2.PROCESS_INSTANCE_ID_ = RES.PROCESS_INSTANCE_ID_ -- from the same proc. inst. and (J2.EXCLUSIVE_ = true) -- also exclusive and (J2.LOCK_OWNER_ is not null and J2.LOCK_EXP_TIME_ >= ?) -- in progress ) ) or RES.EXCLUSIVE_ = false ) LIMIT ? OFFSET ?
2022-10-26 22:39:17.837 61532 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : ==> Parameters: 2022-10-26 22:39:17.83(Timestamp), 2022-10-26 22:39:17.83(Timestamp), 2022-10-26 22:39:17.83(Timestamp), 3(Integer), 0(Integer)
2022-10-26 22:39:17.852 61532 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : <==      Total: 2

When there is a pending batch and PCF app in test environment is started then this is the log (Not working scenario):

08:14:12.196: [APP/PROC/WEB.0] 2022-10-27 08:14:12.195 8 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : ==>  Preparing: select * from ACT_RU_JOB where HANDLER_TYPE_ = ?
08:14:12.223: [APP/PROC/WEB.0] 2022-10-27 08:14:12.222 8 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : ==> Parameters: history-cleanup(String)
08:14:12.240: [APP/PROC/WEB.0] 2022-10-27 08:14:12.239 8 --- [           main] o.c.b.e.i.p.e.J.selectJobsByHandlerType  : <==      Total: 1

08:14:22.519: [APP/PROC/WEB.0] 2022-10-27 08:14:22.518 8 --- [           main] org.camunda.bpm.engine.jobexecutor       : ENGINE-14014 Starting up the JobExecutor[org.camunda.bpm.engine.spring.components.jobexecutor.SpringJobExecutor].
08:14:22.531: [APP/PROC/WEB.0] 2022-10-27 08:14:22.527 8 --- [ingJobExecutor]] org.camunda.bpm.engine.jobexecutor       : ENGINE-14018 JobExecutor[org.camunda.bpm.engine.spring.components.jobexecutor.SpringJobExecutor] starting to acquire jobs

08:14:22.549: [APP/PROC/WEB.0] 2022-10-27 08:14:22.548 8 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : ==>  Preparing: select RES.ID_, RES.REV_, RES.DUEDATE_, RES.PROCESS_INSTANCE_ID_, RES.EXCLUSIVE_ from ACT_RU_JOB RES where (RES.RETRIES_ > 0) and ( RES.DUEDATE_ is null or RES.DUEDATE_ <= ? ) and (RES.LOCK_OWNER_ is null or RES.LOCK_EXP_TIME_ < ?) and RES.SUSPENSION_STATE_ = 1 and (RES.DEPLOYMENT_ID_ is null or ( RES.DEPLOYMENT_ID_ IN ( ? ) ) ) and ( ( RES.EXCLUSIVE_ = true and not exists( select J2.ID_ from ACT_RU_JOB J2 where J2.PROCESS_INSTANCE_ID_ = RES.PROCESS_INSTANCE_ID_ -- from the same proc. inst. and (J2.EXCLUSIVE_ = true) -- also exclusive and (J2.LOCK_OWNER_ is not null and J2.LOCK_EXP_TIME_ >= ?) -- in progress ) ) or RES.EXCLUSIVE_ = false ) LIMIT ? OFFSET ?
08:14:22.553: [APP/PROC/WEB.0] 2022-10-27 08:14:22.552 8 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : ==> Parameters: 2022-10-27 08:14:22.541(Timestamp), 2022-10-27 08:14:22.541(Timestamp), 38ddcea6-556d-11ed-896a-d279f718261b(String), 2022-10-27 08:14:22.541(Timestamp), 3(Integer), 0(Integer)
08:14:22.558: [APP/PROC/WEB.0] 2022-10-27 08:14:22.558 8 --- [ingJobExecutor]] o.c.b.e.i.p.e.J.selectNextJobsToExecute  : <==      Total: 0

Hi, the issue is resolved after I had explicitly set the deployment-aware flag to false in application yml file.

I need to find a way to check the deploymentAware flag value of camunda to avoid these kind of issues.

I tried this How to set jobExecutor Deployment Aware to false globally in camunda - #4 by aravindhrs but the status value is null