I used the helm chart zeebe-cluster-helm to build the zeebe cluster, the cluster itself works fine, I can deploy workflow and create instance inside it.
I make the gateway service nodeport type.
so I can visit the metrics endpoint http://gateway:port/metrics and below is the only zeebe metrics I can see. zeebe_long_polling_queued_current
But actually there should be much more metrics like below.
zeebe_stream_processor_events_total
zeebe_running_workflow_instances_total
zeebe_pending_jobs_total
zeebe_incident_events_total
I also tried to refer to the zeebe/gateway.yaml.template at main · camunda/zeebe · GitHub and add the environment variable ZEEBE_GATEWAY_MONITORING_ENABLED to true in the helm chart and redeploy the chart.
but it only add one more metrics ‘zeebe_gateway_topology_partition_roles’
So what I want is how to display all the metrics like zeebe_stream_processor_events_total,zeebe_running_workflow_instances_total,zeebe_pending_jobs_total,zeebe_incident_events_total in the endpoint gateway:port/metrics for the zeebe cluster.
one more thing I want to mention is that if I start a standalone zeebe broker with builtin gateway in the port 9600(without docker or k8s), then I can visit the host:9600/metrics to get all the zeebe metrics.
thanks @Zelldon but how can I get all zeebe metrics in the gateway endpoint, or in the best practice how shall these metrics exposed since the broker itself is not exposed to outside, right?
Thanks for you reply.
my zeebe broker service name is myzeebe2-zeebe
and my broker has two replica, one endpoint domain name is: myzeebe2-zeebe-0.myzeebe2-zeebe ,another one is: myzeebe2-zeebe-1.myzeebe2-zeebe
according to my testing, I found only the myzeebe2-zeebe-1.myzeebe2-zeebe:9600/metrics can return all the zeebe metrics like ‘zeebe_running_workflow_instances_total’, myzeebe2-zeebe-0.myzeebe2-zeebe:9600/metrics only return very small amount metrics like 'zeebe_snapshot_file_size_megabytes_bucket’
So I can not use the myzeebe2-zeebe:9600/metrics to get the metrics, I have to directly access the myzeebe2-zeebe-1.myzeebe2-zeebe
So I was wondering how is it designed? I have not found too much information on the official document.