I am running camunda on a Kubernetes cluster, I am facing a serious issue, the broker somehow is not working and is unable to be connected.
Here are the logs from my pod :
2022-05-05 03:10:25.010 [] [main] INFO
io.atomix.raft.partition.impl.RaftPartitionServer - RaftPartitionServer{raft-partition-partition-1} - Starting server for partition PartitionId{id=1, group=raft-partition}
2022-05-05 03:10:25.179 [Broker-0-SnapshotStore-1] [Broker-0-zb-fs-workers-0] WARN
io.camunda.zeebe.snapshots.impl.FileBasedSnapshotStore - Expected snapshot /usr/local/zeebe/data/raft-partition/partitions/1/snapshots/10379670-6-17224943-17224939 to have checksum 4260498054914973801, but the actual checksum is 763173917; the snapshot is most likely corrupted. The startup will fail if there is no other valid snapshot and the log has been compacted.
2022-05-05 03:10:26.288 [] [main] ERROR
io.camunda.zeebe.broker.system - Bootstrap Broker-0 [7/9]: zeebe partitions failed with unexpected exception.
java.lang.IllegalStateException: Expected to find a snapshot at index >= log's first index 10343772, but found snapshot 0. A previous snapshot is most likely corrupted.
at io.atomix.raft.impl.RaftContext.verifySnapshotLogConsistent(RaftContext.java:221) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.RaftContext.<init>(RaftContext.java:198) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:258) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:232) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.buildServer(RaftPartitionServer.java:191) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.initServer(RaftPartitionServer.java:157) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.start(RaftPartitionServer.java:115) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartition.open(RaftPartition.java:120) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartitionGroup.lambda$join$6(RaftPartitionGroup.java:188) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?]
at java.util.HashMap$KeySpliterator.forEachRemaining(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
at io.atomix.raft.partition.RaftPartitionGroup.join(RaftPartitionGroup.java:190) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.primitive.partition.impl.DefaultPartitionService.start(DefaultPartitionService.java:63) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.partitioning.PartitionManagerImpl.start(PartitionManagerImpl.java:128) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.partitionsStep(Broker.java:356) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.lambda$initStart$6(Broker.java:188) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:64) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:62) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:47) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:141) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) [zeebe-util-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.start(Broker.java:121) [zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:74) [camunda-cloud-zeebe-1.2.8.jar:1.2.8]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332) [spring-boot-2.5.5.jar:2.5.5]
at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:58) [camunda-cloud-zeebe-1.2.8.jar:1.2.8]
2022-05-05 03:10:26.303 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [1/6]: leader management request handler
2022-05-05 03:10:26.310 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [2/6]: disk space monitor
2022-05-05 03:10:26.311 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [3/6]: cluster services
2022-05-05 03:10:26.312 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [4/6]: subscription api
2022-05-05 03:10:26.313 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [5/6]: command api transport and handler
2022-05-05 03:10:26.433 [] [Thread-13] INFO
io.atomix.cluster.messaging.impl.NettyMessagingService - Stopped
2022-05-05 03:10:26.434 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 [6/6]: Migrated Startup Steps
2022-05-05 03:10:26.437 [Broker-0-Startup] [Broker-0-zb-actors-1] INFO
io.camunda.zeebe.broker.system - Shutdown Cluster Services (Creation)
2022-05-05 03:10:26.438 [Broker-0-Startup] [Broker-0-zb-actors-1] INFO
io.atomix.cluster.messaging.impl.DefaultClusterCommunicationService - Stopped
2022-05-05 03:10:26.440 [] [atomix-cluster-0] INFO
io.atomix.cluster.messaging.impl.DefaultClusterEventService - Stopped
2022-05-05 03:10:26.442 [] [atomix-cluster-0] INFO
io.atomix.cluster.protocol.SwimMembershipProtocol - 0 - Member deactivated: Member{id=0, address=emil-bp-zeebe-0.emil-bp-zeebe.default.svc.cluster.local:26502, properties={brokerInfo=EADJAAAAAwAAAAAAAQAAAAMAAAADAAAAAAABCgAAAGNvbW1hbmRBcGk9AAAAZW1pbC1icC16ZWViZS0wLmVtaWwtYnAtemVlYmUuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDoyNjUwMQUAAAwAAAUAAAAxLjIuOAUAAA==}}
2022-05-05 03:10:26.444 [] [atomix-cluster-0] INFO
io.atomix.cluster.protocol.SwimMembershipProtocol - Stopped
2022-05-05 03:10:26.445 [] [atomix-cluster-0] INFO
io.atomix.cluster.discovery.BootstrapDiscoveryProvider - Local node Member{id=0, address=emil-bp-zeebe-0.emil-bp-zeebe.default.svc.cluster.local:26502, properties={brokerInfo=EADJAAAAAwAAAAAAAQAAAAMAAAADAAAAAAABCgAAAGNvbW1hbmRBcGk9AAAAZW1pbC1icC16ZWViZS0wLmVtaWwtYnAtemVlYmUuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDoyNjUwMQUAAAwAAAUAAAAxLjIuOAUAAA==}} left the bootstrap servide
2022-05-05 03:10:26.448 [] [atomix-cluster-0] INFO
io.atomix.cluster.impl.DefaultClusterMembershipService - Stopped cluster membership service for member Member{id=0, address=emil-bp-zeebe-0.emil-bp-zeebe.default.svc.cluster.local:26502, properties={brokerInfo=EADJAAAAAwAAAAAAAQAAAAMAAAADAAAAAAABCgAAAGNvbW1hbmRBcGk9AAAAZW1pbC1icC16ZWViZS0wLmVtaWwtYnAtemVlYmUuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDoyNjUwMQUAAAwAAAUAAAAxLjIuOAUAAA==}}
2022-05-05 03:10:28.577 [] [Thread-18] INFO
io.atomix.cluster.messaging.impl.NettyMessagingService - Stopped
2022-05-05 03:10:28.581 [] [atomix-cluster-0] INFO
io.atomix.cluster.AtomixCluster - Stopped
2022-05-05 03:10:28.583 [Broker-0-Startup] [Broker-0-zb-actors-1] INFO
io.camunda.zeebe.broker.system - Shutdown monitoring services
2022-05-05 03:10:28.585 [] [main] INFO
io.camunda.zeebe.broker.system - Closing Broker-0 succeeded. Closed 6 steps in 2282 ms.
2022-05-05 03:10:28.586 [] [main] ERROR
io.camunda.zeebe.broker.system - Failed to start broker 0!
java.lang.IllegalStateException: Expected to find a snapshot at index >= log's first index 10343772, but found snapshot 0. A previous snapshot is most likely corrupted.
at io.atomix.raft.impl.RaftContext.verifySnapshotLogConsistent(RaftContext.java:221) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.RaftContext.<init>(RaftContext.java:198) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:258) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:232) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.buildServer(RaftPartitionServer.java:191) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.initServer(RaftPartitionServer.java:157) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.start(RaftPartitionServer.java:115) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartition.open(RaftPartition.java:120) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartitionGroup.lambda$join$6(RaftPartitionGroup.java:188) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?]
at java.util.HashMap$KeySpliterator.forEachRemaining(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
at io.atomix.raft.partition.RaftPartitionGroup.join(RaftPartitionGroup.java:190) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.primitive.partition.impl.DefaultPartitionService.start(DefaultPartitionService.java:63) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.partitioning.PartitionManagerImpl.start(PartitionManagerImpl.java:128) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.partitionsStep(Broker.java:356) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.lambda$initStart$6(Broker.java:188) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:64) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:62) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:47) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:141) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) [zeebe-util-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.start(Broker.java:121) [zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:74) [camunda-cloud-zeebe-1.2.8.jar:1.2.8]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332) [spring-boot-2.5.5.jar:2.5.5]
at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:58) [camunda-cloud-zeebe-1.2.8.jar:1.2.8]
2022-05-05 03:10:28.597 [] [main] INFO
org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-05-05 03:10:28.620 [] [main] ERROR
org.springframework.boot.SpringApplication - Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:794) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) [spring-boot-2.5.5.jar:2.5.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332) [spring-boot-2.5.5.jar:2.5.5]
at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:58) [camunda-cloud-zeebe-1.2.8.jar:1.2.8]
Caused by: io.camunda.zeebe.util.exception.UncheckedExecutionException: Failed to start broker
at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:151) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.start(Broker.java:121) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:74) ~[camunda-cloud-zeebe-1.2.8.jar:1.2.8]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) ~[spring-boot-2.5.5.jar:2.5.5]
... 5 more
Caused by: java.lang.IllegalStateException: Expected to find a snapshot at index >= log's first index 10343772, but found snapshot 0. A previous snapshot is most likely corrupted.
at io.atomix.raft.impl.RaftContext.verifySnapshotLogConsistent(RaftContext.java:221) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.RaftContext.<init>(RaftContext.java:198) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:258) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.impl.DefaultRaftServer$Builder.build(DefaultRaftServer.java:232) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.buildServer(RaftPartitionServer.java:191) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.initServer(RaftPartitionServer.java:157) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.impl.RaftPartitionServer.start(RaftPartitionServer.java:115) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartition.open(RaftPartition.java:120) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.raft.partition.RaftPartitionGroup.lambda$join$6(RaftPartitionGroup.java:188) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?]
at java.util.HashMap$KeySpliterator.forEachRemaining(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
at io.atomix.raft.partition.RaftPartitionGroup.join(RaftPartitionGroup.java:190) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.atomix.primitive.partition.impl.DefaultPartitionService.start(DefaultPartitionService.java:63) ~[zeebe-atomix-cluster-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.partitioning.PartitionManagerImpl.start(PartitionManagerImpl.java:128) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.partitionsStep(Broker.java:356) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.lambda$initStart$6(Broker.java:188) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:64) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:62) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:93) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:47) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:141) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.Broker.start(Broker.java:121) ~[zeebe-broker-1.2.8.jar:1.2.8]
at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:74) ~[camunda-cloud-zeebe-1.2.8.jar:1.2.8]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) ~[spring-boot-2.5.5.jar:2.5.5]
... 5 more
And here is my describe of the pod:
Name: emil-bp-zeebe-0
Namespace: default
Priority: 0
Node: ip-10-0-5-222.eu-central-1.compute.internal/10.0.5.222
Start Time: Thu, 05 May 2022 05:10:03 +0200
Labels: app.kubernetes.io/component=broker
app.kubernetes.io/instance=emil-bp
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=zeebe-cluster-helm
controller-revision-hash=emil-bp-zeebe-f7767fc6f
statefulset.kubernetes.io/pod-name=emil-bp-zeebe-0
Annotations: kubernetes.io/psp: eks.privileged
Status: Running
IP: 10.0.5.15
IPs:
IP: 10.0.5.15
Controlled By: StatefulSet/emil-bp-zeebe
Containers:
zeebe-cluster-helm:
Container ID: docker://80c5b6b76d780e5d4112694d59b2cb6d41115de10bbb048fd5457d650b80a75e
Image: camunda/zeebe:1.2.8
Image ID: docker-pullable://camunda/zeebe@sha256:bb1807fd2a2310c73d660f32a3e57f4496457fed6bd26ad8778944df6ea0460f
Ports: 9600/TCP, 26501/TCP, 26502/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
State: Running
Started: Thu, 05 May 2022 05:10:10 +0200
Ready: False
Restart Count: 0
Limits:
cpu: 1
memory: 4Gi
Requests:
cpu: 500m
memory: 2Gi
Readiness: http-get http://:9600/ready delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
ZEEBE_BROKER_CLUSTER_CLUSTERNAME: emil-bp-zeebe
ZEEBE_LOG_LEVEL: info
ZEEBE_BROKER_CLUSTER_PARTITIONSCOUNT: 1
ZEEBE_BROKER_CLUSTER_CLUSTERSIZE: 3
ZEEBE_BROKER_CLUSTER_REPLICATIONFACTOR: 3
ZEEBE_BROKER_THREADS_CPUTHREADCOUNT: 2
ZEEBE_BROKER_THREADS_IOTHREADCOUNT: 2
ZEEBE_BROKER_GATEWAY_ENABLE: false
ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_CLASSNAME: io.camunda.zeebe.exporter.ElasticsearchExporter
ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_URL: http://elasticsearch-master:9200
ZEEBE_BROKER_NETWORK_COMMANDAPI_PORT: 26501
ZEEBE_BROKER_NETWORK_INTERNALAPI_PORT: 26502
ZEEBE_BROKER_NETWORK_MONITORINGAPI_PORT: 9600
K8S_POD_NAME: emil-bp-zeebe-0 (v1:metadata.name)
JAVA_TOOL_OPTIONS: -XX:MaxRAMPercentage=25.0 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/zeebe/data -XX:ErrorFile=/usr/local/zeebe/data/zeebe_error%p.log -XX:+ExitOnOutOfMemoryError
Mounts:
/exporters from exporters (rw)
/usr/local/bin/startup.sh from config (rw,path="startup.sh")
/usr/local/zeebe/config/application.yaml from config (rw,path="application.yaml")
/usr/local/zeebe/data from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7sjmz (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: data-emil-bp-zeebe-0
ReadOnly: false
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: emil-bp-zeebe-cluster-helm
Optional: false
exporters:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-7sjmz:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 29m default-scheduler Successfully assigned default/emil-bp-zeebe-0 to ip-10-0-5-222.eu-central-1.compute.internal
Normal Pulled 29m kubelet Container image "camunda/zeebe:1.2.8" already present on machine
Normal Created 29m kubelet Created container zeebe-cluster-helm
Normal Started 29m kubelet Started container zeebe-cluster-helm
Warning Unhealthy 29m kubelet Readiness probe failed: HTTP probe failed with statuscode: 503
Warning Unhealthy 4m20s (x151 over 29m) kubelet Readiness probe failed: Get "http://10.0.5.15:9600/ready": dial tcp 10.0.5.15:9600: connect: connection refused
Will appreciate the fast reply as it is super critical .
thank you in advance .