Optimize configuration using custom environment-config.yaml file in Kubernetes

Camunda 8 Topics Discussion & Questions
,
1

Hi Community ! Some advanced help is required if possible.
We want to deploy Camunda Platform and Optimize with Kubernetes. Camunda Platform is up and running, rest endpoints are reachable and all fine.

During Optimize installation, we customized the environment-config.yaml file due to requiring more environment variables than provided. We now need to mount this configuration file into the Optimize pod to apply it. The official documentation says: Replacing the `ABSOLUTE_PATH_ON_HOST_TO_CONFIGURATION_FILE
as in here We've moved! | docs.camunda.org

However we keep getting the following error:

 [EngineImportScheduler-1] ERROR o.c.o.s.i.e.f.instance.TenantFetcher - Error during fetching of entities. Please check the connection with [camunda-bpm]!

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didnt respond. Can not verify this engine's version

With this error and with the fact we can reach the engine endpoints with for example Postman, we believe this error indicates that optimize cannot find our custom configuration very likely the custom config is mounted but optimize does not read it. We are using a ConfigMap to try to mount it and pass it to optimize, but with no success, again we strongly believe is the reading part, because it is mounted. Here is our yaml (anonymized) for Kubernetes.

apiVersion: v1
kind: Pod
metadata:
  name: optimize
  namespace: camunda-optimize
spec:
  hostAliases:
  - ip: "some.numbers.here"
    hostnames:
    - "camunda.mycompany"
  containers:
  - name: optimize-container
    image: registry.camunda.cloud/optimize-ee/optimize:3.6.0
    env:
      - name: OPTIMIZE_CAMUNDABPM_REST_URL
        value: "https://engine/engine-rest"
      - name: OPTIMIZE_CAMUNDABPM_WEBAPPS_URL
        value: "https://engine/camunda"
      - name: OPTIMIZE_ELASTICSEARCH_HOST
        value: "some.local"
      - name: OPTIMIZE_ELASTICSEARCH_HTTP_PORT
        value: "9200"
      - name: OPTIMIZE_ELASTICSEARCH_USERNAME
        value: "<user>"
      - name: OPTIMIZE_ELASTICSEARCH_PASSWORD
        value: "<pass>"
      - name: OPTIMIZE_ELASTICSEARCH_SSL_CERTIFICATE
        value: "/optimize/config/certs/elastic.crt"
      - name: OPTIMIZE_ELASTICSEARCH_SSL_ENABLED
        value: "true"
    ports:
      - containerPort: 8090
        hostPort: 8090
      - containerPort: 8091
        hostPort: 8091
    volumeMounts:
    - name: config
      mountPath: "/optimize/config"
      readOnly: true
    - name: elastic-certs
      mountPath: "/optimize/config/certs"
      readOnly: true
  volumes:
  - name: config
    configMap:
      name: optimize-environment-config
      items:
      - key: "environment-config.yaml"
        path: "environment-config.yaml"
  - name: elastic-certs
    configMap:
      name: elastic-certs
      items:
      - key: "elastic.crt"
        path: "elastic.crt"
      - key: "elastic-ca.crt"
        path: "elastic-ca.crt"
  imagePullSecrets:
  - name: registry-camunda-cloud-secret

If our guess is correct, so the new configuration is mounted but optimize cannot read it, so how do we make optimize read it using the ConfigMaps ?
If not, does anyone have an idea what could be the issue ?

Thanks in advance

2

Can your optimize server resolve https://engine/ ?
It might be something simple like not being able to figure out where the service ingress is running…

3

Hi @g.manzano!
Can you show me what the environment-config you’re trying to mount looks like? If for example you’ve overwritten the engine configuration section without the correct environment variables the variables will not work properly, so I’m wondering if that could be the issue here.

1 Like
4

Hi @Helene thanks for replying.

For more comfort. I am uploading the anonymized environment-config.yaml file as text file. The extension can be changed.
Main takeaway of the file is, to read the env variables, following is used:

${PLACE_HOLDER_ENV_VARIABLE:'some.default.value.if.variable.not.provided'}

Here the assumption was this is allowed to do. If this is not allowed or if there is an specific order (first the default value, then the placeholder), could that be part of the documentation ?.

Do you see anything unusual with this, together with the other yaml file so to more or less give us some input ?

Cheers !

environment-config - anonimyzed.txt (14.4 KB)

5

Hi @g.manzano,
Yeah your assumption there is correct, the environment-config looks fine to me as well. Is the line you posted above regarding the engine connection the only error you’re getting, and could you post more of the log where you’re getting the error? The stacktrace might give some more info where the issue lies. Thanks!

6

Hi @Helene
I think we know what the problem is.

Context.
Engine setup: Spring boot app with embedded engine keycloak for API authentication.
Optimize configuration is pointing at this engine.

Problem (our thesis):
optimize is trying to reach camunda, but was not sending any jwt token/ bearer token with it (the logs show the message: no JWT token found).
Camunda therefore had an empty bearer token that it forwarded to keycloak.
keycloak says: I cannot resolve the groups of an empty bearer token and I should return “not allowed to camunda”
Camunda in turn declinee the request by optimize

So this problem is documented here Optimize Login with Bearer Token and here Custom authentication

So I guess, the solution is to do as recommended in the other tickets ?
Any additional input is very welcome ? And thanks for having a look at my post !

7

Hi @g.manzano,

Glad to hear you’ve already identified the issue, thanks for the detailed reply! If you haven’t already, it sounds to me like you should also look into implementing the Engine Rest Filter to add the relevant infos to Optimize’s requests to the engine.
For the SSO plugin that’s also mentioned in the other forum post you linked, you may want to checkout this Keycloak SSO example for reference.
Let me know if you have any other issues with this!

8