History queries high CPU usage on historyLevel=none

Camunda 7 Topics
1

Hi!

We are hitting high cpu usage levels for our camunda app (embedded engine 7.16) and Postgres database 13.6, and, while history is fully disabled (property value is 0 in act_ge_property for historyLevel, and no new rows in act_hi_* since we disabled it), we still use 20% of cpu for a simple select res.* from act_hi_job_log.

We also are using camunda-ui, and we are using java api to delete and restart incidents. What can be the cause of history table usage when the history is disabled? We cannot trace it, would be glad if anyone advised us on that.

2

Would you please provide more details about your environment.

  1. Is your environment is standalone or containerized?
  2. Is engine and postgres running on same server?
  3. Is this dev or prod server?
  4. Have you applied any performance tuning for your environment? [ Link for tuning: Performance tuning Camunda 7 | Camunda Platform 8 Docs ]
3

Hi!

  1. We have several replicas of embedded engine apps running in containers on Kubernetes. They form a homogeneous cluster.
  2. Postgres is bare metal and running elsewhere
  3. This is on prod. Unfortunately could not reproduce on dev, where we see no queries to history table (just as we would expect, because we have set history level to none)
  4. Yes, we have our job executor already tuned, jvm is running with default settings, memory and cpu limits for containers are enough, everything else looks okay. Most of all - our greatest optimisation was setting history level to zero, because we have lots of short-term processes, external history-like features and we did not need camunda history, while it generated quite a lot of cpu work and took space. So we turned the level from default (3) to none (0) and truncated the biggest history tables, leaving data only in some for debugging purposes if turning history off goes wrong.
4

Hi!

  1. We have several replicas of embedded engine apps running in containers on Kubernetes. They form a homogeneous cluster.
  2. Postgres is bare metal and running elsewhere
  3. This is on prod. Unfortunately could not reproduce on dev, where we see no queries to history table (just as we would expect, because we have set history level to none)
  4. Yes, we have our job executor already tuned, jvm is running with default settings, memory and cpu limits for containers are enough, everything else looks okay. Most of all - our greatest optimisation was setting history level to zero, because we have lots of short-term processes, external history-like features and we did not need camunda history, while it generated quite a lot of cpu work and took space. So we turned the level from default (3) to none (0) and truncated the biggest history tables, leaving data only in some for debugging purposes if turning history off goes wrong.
5

Did you check the Camunda DB isolation level? What is currently set to.

How are you accessing the postgres db from Camunda Cluster?