We are running Camunda8 Self-Managed (Helm v14.1.0, Camunda 8.9) in Kubernetes (GKE) in GCP (Google Cloud).
Several Camunda8 environments (“dev”, “test”) are deployed in separate namespaces.
The platform runs on private IPs only and is connected to our corporate network via VPN.
Our corporate DNS server resolves all our domains to one single IP (this is hardcoded and beyond my access):
dev.<internal-domain> → <fixed-ip>
dev-grpc.<internal-domain> → <fixed-ip>
test.<internal-domain> → <fixed-ip>
test-grpc.<internal-domain> → <fixed-ip>
Using Ingress (ingress-nginx), this was easy business. We had a central nginx ingress controller deployed with load balancer IP <fixed-ip> in a dedicated namespace and the C8 helm charts added ingress resources in their respective namespaces as configured with the chart.
With Gateway API things are not as simple anymore. To achieve the same thing I need to run a central Gateway with IP <fixed-ip> in a dedicated namespace and point all routes created by the Camunda8 Helm chart to that gateway. The chart appears to support that:
The controllerNamespace property has no effect on the HTTP and GRPC routes created, they lack the namespace property in parentRef.
Having TLS enabled on the central gateway, I need to set tls.enabled=true in the Camunda8 Helm chart to set the sectionName in parentRef to https (instead of http).
The GRPC route in Orchestration gets created with parentRef.sectionName=grpcs, even though in the shared setup we’ll very likely only have one listener listening on *.<internal-domain> supporting both HTTP and GRPCs routes (thus the sectionNameneeds to be https).
All routes point to a gateway named <deployment-name>-camunda-platform and there is no way to configure this, which means my central gateway (despite potentially serving other services, too) needs to be named that way.
So what did we do?
After deploying with the exact configuration shown above we patch the routes:
kubectl get httproute \
-n "$camunda_namespace" -l app=camunda-platform -o name 2>/dev/null | \
while read route; do
kubectl patch "$route" -n "$camunda_namespace" --type=json \
-p='[{"op":"add","path":"/spec/parentRefs/0/namespace","value":"$gateway_namespace"}]'
done
kubectl get grpcroute \
-n "$camunda_namespace" -l app=camunda-platform -o name 2>/dev/null | \
while read route; do
kubectl patch "$route" -n "$camunda_namespace" --type=json \
-p='[{"op":"add","path":"/spec/parentRefs/0/namespace","value":"$gateway_namespace"},{"op":"replace","path":"/spec/parentRefs/0/sectionName","value":"https"}]'
done
After that, everything works like a charm! I think the Helm chart needs some improvements here, do you agree? Do I miss something?
You’ve identified some valid limitations with the current Gateway API implementation in the Camunda 8 Helm chart for shared gateway scenarios. I found the following relevant resources:
On top of that I discovered that the URL rewrite from /identity (exact) to /identity/ does not work as desired and leaves us with a blank page if /identity is called because the resources can’t be loaded:
Another issue when using Gateway API instead of Ingress:
In the Camunda8 Release Info and in Console Configuration, all URLs are pointing to localhost because the Helm chart constructs them using the following pattern:
{{- if .Values.global.ingress.enabled -}}
...
{{- else -}}
{{- printf "http://localhost:..." -}}
{{- end -}}
A check for .Values.global.gateway.enabled is missing.
Regarding 6168: While a mixed TLS/non-TLS setup would cause similar issues it is not what we do and why I brought the issue up. My point is different, I’ll try to elaborate.
What the helm chart currently does, if you let it create the gateways:
It creates an “https Gateway” with section name “https”, protocol HTTPS, port 443 listening to my.domain
It creates a “grpcs Gateway” with section name “grcps”, protocol HTTPS, port 443 listening to my-grpc.domain (this is an https listener as well, just like the one from #1)
It creates HTTPRoutes with section name “https” and GRPCRoutes with section name “grpcs”
With a shared gateway:
I need to listen to *.domain to cover all environments (which might not be known beforehad). Listening to *-grpc.domain is not possible.
I cannot create two sections “https” and “grpcs” with the very same parameters: protocol HTTPS, port 443 listening to *.domain
-> I need to run the GRPCRoute on the “https” section/listener. That is why we patch it.
Maybe introduce a .Values.global.gateway.listener or .Values.global.gateway.sectionName setting next to .Values.global.gateway.controllerNamespace? Or name the section “https” (or “http” depending on the TLS setting) in the GRPC Gateway created by the chart?
That is possible as global.host and grpc.host are known and different.
In our shared gateway it is one gateway with one listener on a wildcard host (in other words: no listener called “grpcs”). So the need for the patch is the same I just didn’t explain the root cause right,
I’m surprised it can work on the same url for both grpcs and https. a request comes in to the gateway, and it needs to somehow identify which port on the pod it should send the request to. I figured it used the URL to switch between those. If it’s protocol-aware as well, then I wonder if only your GatewayClass supports it this way, or if most GatewayClasses are intended to work this way.
@Jesse_Simpson It is not that https and grpcs work on the same url. The gateway has a listener on a wildcard domain (*.<domain>) and the routes define the actual urls (e.g. dev.<domain> and dev-grpc.<domain>).
That way you can have dev, test1, test2, int, … on the same Gateway under a single IP.
I think I understand why you want the listeners and sectionName options to be configurable. But if these things are configurable (for the purpose of supporting one Gateway with multiple camunda installations), it feels a bit like the Helm chart isn’t doing any simplification in setting this up. Is supplying the extraListeners you want and sectionName you want in values.yaml much different from defining the Gateway and HTTPRoute inside global.extraManifests (or outside the helm chart) ?
Any outside solution means that we need to check if/how the helm chart potentially changes with updates (change of deployment name e.g.). Possible, not desirable though.
I feel like if the chart supports “createGateway=false” then these things should to be configurable as you cannot foresee how the provided gateway could look like. However, the chart itself needs to stay maintainable, I’m aware of that!
I feel like if the chart supports “createGateway=false” then these things should to be configurable as you cannot foresee how the provided gateway could look like.
Yeah they definitely have to be configurable since we do have this option. The createGatewayResource option was needed because the Gateway spec is written with different roles in mind. a Cluster Operator creates the Gateway, an Application Developer creates HttpRoutes+GRPCRoutes. So the helm chart needs to account for these 2 roles existing with different access permissions.
the (GRPC|HTTP)Routes reference parts of the Gateway. So the sectionName needs to be configurable.
We also have the global.gateway.external option for people who want to do all the managing of their own routes and gateways, but as camunda gains or consolidates components, it’d be a burden for chart users to maintain those routes.
You can file these sorts of requests (especially if they have your degree of detail) directly in github issues. There is some value in me creating issues directly, since I can validate the problem and ensure all the information is present for any one of our team to work on, but there’s less of a guarantee I’ll see it in the forum.
The forum is a bit less visible to engineers, but you might also get quicker responses from other chart users who have ran similar errors.
Hints: Use the Ask AI feature in 
