Hi,
i’m trying to install camunda 8 with hazelcast exporter via helm chart. I’m following the guide from the documentation, but it seems to be ineffective. This is the values.yaml i’m using:
Hi @gmosti
Could you please explain what you mean by “but it seems to be ineffective.”? Do you see some errors during startup? Have you checked the Note in the documentation regarding pods with a read-only root file system?
Hi @gmosti - the extraInitContainers and env sections at the bottom of your values.yaml file need to be under the zeebe key. See here and here for reference!
I think the documentation could be improved. Do you have any suggestions on what we could add to the docs to clarify this? Do you think just having zeebe: at the top of the code block would be enough?
Hi, thanks for your responses. @Alex_Voloshyn exporters are not under /exporters and variables are not setted but the reaseon is the one explained by @nathan.loding
@nathan.loding yes i think it could help, for the rest seems clear to me. I’ve corrected the yaml but the zeebe pod doesn’t start:
camunda-zeebe-0 0/1 Init:CreateContainerConfigError 0 103s
This the last events describing the pod :
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 57s default-scheduler Successfully assigned default/camunda-zeebe-0 to 10.0.10.225
Normal SuccessfulAttachVolume 35s attachdetach-controller AttachVolume.Attach succeeded for volume "csi-32dbacf8-beaf-453c-a5da-11a2d1301614"
Normal Pulled 9s (x3 over 23s) kubelet Container image "busybox:1.35" already present on machine
Warning Failed 9s (x3 over 23s) kubelet Error: container has runAsNonRoot and image will run as root (pod: "camunda-zeebe-0_default(b3aa2512-829f-424d-9d63-977a1c2a1e2f)", container: init-exporters-hazelcast)
Here the yaml i’m using now, i’ve inserted imagePullPolicy :
Name: camunda-zeebe-0
Namespace: default
Priority: 0
Service Account: default
Node: 10.0.10.225/10.0.10.225
Start Time: Fri, 23 Feb 2024 11:22:21 +0100
Labels: app=camunda-platform
app.kubernetes.io/component=zeebe-broker
app.kubernetes.io/instance=camunda
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=camunda-platform
app.kubernetes.io/part-of=camunda-platform
app.kubernetes.io/version=latest
apps.kubernetes.io/pod-index=0
controller-revision-hash=camunda-zeebe-6589cbcf5c
helm.sh/chart=camunda-platform-9.1.0
statefulset.kubernetes.io/pod-name=camunda-zeebe-0
Annotations: <none>
Status: Pending
IP: 10.0.10.71
IPs:
IP: 10.0.10.71
Controlled By: StatefulSet/camunda-zeebe
Init Containers:
init-exporters-hazelcast:
Container ID:
Image: busybox:1.35
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
wget --no-check-certificate https://repo1.maven.org/maven2/io/zeebe/hazelc
ast/zeebe-hazelcast-exporter/1.4.0/zeebe-hazelcast-exporter-1.4.0-jar-with-depen
dencies.jar -O /exporters/zeebe-hazelcast-exporter.jar; ls -al /exporters
State: Waiting
Reason: CreateContainerConfigError
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/exporters/ from exporters (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-b9qfv (
ro)
Containers:
zeebe:
Container ID:
Image: camunda/zeebe:8.4.1
Image ID:
Ports: 9600/TCP, 26501/TCP, 26502/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 960m
memory: 1920Mi
Requests:
cpu: 800m
memory: 1200Mi
Readiness: http-get http://:9600/ready delay=30s timeout=1s period=30s #suc
cess=1 #failure=5
Environment:
LC_ALL: C.UTF-8
K8S_NAME: camunda-zeebe-0 (
v1:metadata.name)
K8S_SERVICE_NAME: camunda-zeebe
K8S_NAMESPACE: default (v1:metad
ata.namespace)
ZEEBE_BROKER_NETWORK_ADVERTISEDHOST: $(K8S_NAME).$(K8S
_SERVICE_NAME).$(K8S_NAMESPACE).svc
ZEEBE_BROKER_CLUSTER_INITIALCONTACTPOINTS: $(K8S_SERVICE_NAM
E)-0.$(K8S_SERVICE_NAME).$(K8S_NAMESPACE).svc:26502,
ZEEBE_BROKER_CLUSTER_CLUSTERNAME: camunda-zeebe
ZEEBE_LOG_LEVEL: info
ZEEBE_BROKER_CLUSTER_PARTITIONSCOUNT: 1
ZEEBE_BROKER_CLUSTER_CLUSTERSIZE: 1
ZEEBE_BROKER_CLUSTER_REPLICATIONFACTOR: 1
ZEEBE_BROKER_THREADS_CPUTHREADCOUNT: 3
ZEEBE_BROKER_THREADS_IOTHREADCOUNT: 3
ZEEBE_BROKER_GATEWAY_ENABLE: false
ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_CLASSNAME: io.camunda.zeebe.
exporter.ElasticsearchExporter
ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_URL: http://camunda-el
asticsearch:9200
ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_INDEX_PREFIX: zeebe-record
ZEEBE_BROKER_NETWORK_COMMANDAPI_PORT: 26501
ZEEBE_BROKER_NETWORK_INTERNALAPI_PORT: 26502
ZEEBE_BROKER_NETWORK_MONITORINGAPI_PORT: 9600
K8S_POD_NAME: camunda-zeebe-0 (
v1:metadata.name)
JAVA_TOOL_OPTIONS: -XX:+HeapDumpOnOu
tOfMemoryError -XX:HeapDumpPath=/usr/local/zeebe/data -XX:ErrorFile=/usr/local/z
eebe/data/zeebe_error%p.log -XX:+ExitOnOutOfMemoryError
ZEEBE_BROKER_EXPORTERS_HAZELCAST_JARPATH: /exporters/zeebe-
hazelcast-exporter.jar
ZEEBE_BROKER_EXPORTERS_HAZELCAST_CLASSNAME: io.zeebe.hazelcas
t.exporter.HazelcastExporter
ZEEBE_HAZELCAST_REMOTE_ADDRESS: camunda-hazelcast
Mounts:
/exporters from exporters (rw)
/tmp from tmp (rw)
/usr/local/bin/startup.sh from config (rw,path="startup.sh")
/usr/local/zeebe/data from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-b9qfv (
ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in
the same namespace)
ClaimName: data-camunda-zeebe-0
ReadOnly: false
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: camunda-zeebe
Optional: false
exporters:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
tmp:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-b9qfv:
Type: Projected (a volume that contains injected data fro
m multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists fo
r 300s
node.kubernetes.io/unreachable:NoExecute op=Exists
for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulled 50s (x19641 over 2d22h) kubelet Container image "busybox:1.3
5" already present on machine
I’ve no result in logs :
kubectl logs -f camunda-zeebe-0 -c init-exporters-hazelcast
Error from server (BadRequest): container "init-exporters-hazelcast" in pod "camunda-zeebe-0" is waiting to start: CreateContainerConfigError
kubectl logs -f camunda-zeebe-0
Defaulted container "zeebe" out of: zeebe, init-exporters-hazelcast (init)
Error from server (BadRequest): container "zeebe" in pod "camunda-zeebe-0" is waiting to start: PodInitializing
@gmosti - that’s strange, I was expecting something in the logs! If you remove the exporter (remove the extraInitContainers and env sections you added), does Zeebe start normally?
Hi @gmosti - I just noticed that you specified an image repository with the value of camunda/camunda-bpm-platform … that repository has images for Camunda 7, I think. I think the Camunda 8 repository is camunda/camunda-platform.
The image: section should not be needed if you are using Camunda’s hosted images. That option is available for customers who have their own private registry that they maintain.
