How to improve the performance of Zeebe, the configuration file is as follows

How to improve the performance of Zeebe, the configuration file is as follows

# Zeebe Standalone Broker configuration file (with embedded gateway)

# This file is based on broker.standalone.yaml.template but stripped down to contain only a limited
# set of configuration options. These are a good starting point to get to know Zeebe.
# For advanced configuration options, have a look at the templates in this folder.

# !!! Note that this configuration is not suitable for running a standalone gateway. !!!
# If you want to run a standalone gateway node, please have a look at gateway.yaml.template

# ----------------------------------------------------
# Byte sizes
# For buffers and others must be specified as strings and follow the following
# format: "10U" where U (unit) must be replaced with KB = Kilobytes, MB = Megabytes or GB = Gigabytes.
# If unit is omitted then the default unit is simply bytes.
# Example:
# sendBufferSize = "16MB" (creates a buffer of 16 Megabytes)
#
# Time units
# Timeouts, intervals, and the likes, must be specified either in the standard ISO-8601 format used
# by java.time.Duration, or as strings with the following format: "VU", where:
#   - V is a numerical value (e.g. 1, 5, 10, etc.)
#   - U is the unit, one of: ms = Millis, s = Seconds, m = Minutes, or h = Hours
#
# Paths:
# Relative paths are resolved relative to the installation directory of the broker.
# ----------------------------------------------------

zeebe:
  broker:
    gateway:
      # Enable the embedded gateway to start on broker startup.
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_GATEWAY_ENABLE.
      enable: true

      network:
        # Sets the port the embedded gateway binds to.
        # This setting can also be overridden using the environment variable ZEEBE_BROKER_GATEWAY_NETWORK_PORT.
        port: 26500

      security:
        # Enables TLS authentication between clients and the gateway
        # This setting can also be overridden using the environment variable ZEEBE_BROKER_GATEWAY_SECURITY_ENABLED.
        enabled: false

    network:
      # Controls the default host the broker should bind to. Can be overwritten on a
      # per binding basis for client, management and replication
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_NETWORK_HOST.
      host: 0.0.0.0

    data:
      # Specify a directory in which data is stored.
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_DATA_DIRECTORY.
      directory: data
      # The size of data log segment files.
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_DATA_LOGSEGMENTSIZE.
      logSegmentSize: 512MB
      # How often we take snapshots of streams (time unit)
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_DATA_SNAPSHOTPERIOD.
      snapshotPeriod: 15m

    cluster:
      # Specifies the Zeebe cluster size.
      # This can also be overridden using the environment variable ZEEBE_BROKER_CLUSTER_CLUSTERSIZE.
      clusterSize: 1
      # Controls the replication factor, which defines the count of replicas per partition.
      # This can also be overridden using the environment variable ZEEBE_BROKER_CLUSTER_REPLICATIONFACTOR.
      replicationFactor: 1
      # Controls the number of partitions, which should exist in the cluster.
      # This can also be overridden using the environment variable ZEEBE_BROKER_CLUSTER_PARTITIONSCOUNT.
      partitionsCount: 8

    threads:
      # Controls the number of non-blocking CPU threads to be used.
      # WARNING: You should never specify a value that is larger than the number of physical cores
      # available. Good practice is to leave 1-2 cores for ioThreads and the operating
      # system (it has to run somewhere). For example, when running Zeebe on a machine
      # which has 4 cores, a good value would be 2.
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_THREADS_CPUTHREADCOUNT
      cpuThreadCount: 10
      # Controls the number of io threads to be used.
      # This setting can also be overridden using the environment variable ZEEBE_BROKER_THREADS_IOTHREADCOUNT
      ioThreadCount: 2
    backpressure:
      enabled: false
    exporters:
      hazelcast:
        className: io.zeebe.hazelcast.exporter.HazelcastExporter
        jarPath: exporters/zeebe-hazelcast-exporter-1.0.0-jar-with-dependencies.jar
        args:
        # Hazelcast port
          port: 5701
          enabledValueTypes: ""
          enabledRecordTypes: ""
          name: "zeebe"
          capacity: 100000
          timeToLiveInSeconds: 0
          format: "protobuf"

Please see the other forum post, performance optimization is never an isolated endeavor, you have to know what you want to optimize for.

This is comparable to Camunda Platform, where we had this performance tuning guide: Camunda Best Practices - Performance Tuning Camunda. While Camunda Cloud works pretty different, this should still give you an idea about the methodology (what’s my goal, generate the right load and measure, adjust one parameter at a time).

But I could imagine that you face some very basic issue that might be easy to solve even without tuning any Zeebe configs, if you share some details in the other forum post (When the concurrent number reaches 10, the response time is very slow. Is there any configuration that Zeebe can optimize? - #2 by berndruecker), that might help.

Best
Bernd

2 Likes

When the process has parallel tasks, the performance decreases more。

When 100 threads are concurrent per second and the interface is called through Zeebe, the process has only one node and the task is only to call the interface. The throughput will drop by 30-40%, the CPU will occupy almost 50%, and there is still a lot of free memory.
The first figure is calling the interface separately, and the second figure is calling through Zeebe


Unfortunately, I can say much without more information - could you probably share some pieces about your architecture/stack and probably some code? Without knowing how you run Zeebe, how you produce load, and how your workers work (sync/async, threading, …) I could only do super wild guesses. Sorry!

There are factors like L2 cache and number of threads available to each node that impact performance. You really need to go down into the details to performance tune Zeebe.

Have a look at the video and the spreadsheet linked in this Forum post: New Zeebe Performance Benchmarking Tool

This gives you a good overview of the factors that impact performance.

Josh