I saw articles related to multi-tenancy in camunda 8, I took the latest pull from the main branch yesterday(10th October 2023) of camunda platform (GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config), but when i started the applications and checked the identity application , there is no options for tenants as showed in the article (https://camunda.com/blog/2023/10/camunda-8-3-scaling-automation-maximize-value/#multitenancy"), Is the tenants feature not yet released?
Hi @savi003 - did you enable multi-tenancy?
Hi @nathan.loding , No I didn’t know, I had to enable it, I am running camunda through docker, where is that option in case of docker?
There are environment variables for each of the services that need to be enabled. Full documentation in our docs is coming very soon, but you can find the environment variables by searching for where that value is used in the Helm charts: https://github.com/search?q=repo%3Acamunda%2Fcamunda-platform-helm%20.Values.global.multitenancy.enabled&type=code
If you set those environment variables, it should enable multi-tenancy for you!
@nathan.loding I’m not using helm in for local development, I’m using this repo GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config to run the services on docker in my local machine, is multitenancy available here?
@savi003 - those exact same Docker images are used for the Kubernetes cluster created by the Helm charts. The Helm chart values map to environment variables, so by searching for where the Helm charts are using a specific settings you can find the related environment variables. If you look at the example docker-compose.yml, it is setting those same environment variables.
@nathan.loding In this repo of helm I searched for multitenancy https://github.com/search?q=repo%3Acamunda%2Fcamunda-platform-helm%20multitenancy&type=code and i am seeing the configurations related to multitenancy
but when i searched for multitenancy in this repo which I’m using to run the services in my local machine GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config I’m not seeing any configurations related to multitenancy
https://github.com/search?q=repo%3Acamunda%2Fcamunda-platform%20multitenancy&type=code
@savi003 - that is correct. The environment variables are used by the applications, not by Docker, and not all environment variables are used in that Docker configuration.
@nathan.loding so i f want to enable multitenancy in my local with GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config what configurations do i have to make in the docker-compose.yaml (https://github.com/camunda/camunda-platform/blob/97cc173c306195eb3359e5b147b647b66f1dc60e/docker-compose.yaml) file?
@savi003 - you need to add all the environment variables referenced in that Helm search. Then, when you rebuild the container, the applications will see those variables and the feature will be enabled.
@nathan.loding Could you please provide me with a sample?
I do not have a sample Docker configuration for multi-tenancy at this time; you just need to add those environment variables for each application, the same way the environment variables are currently used in the docker-compose.yaml file.
For instance, under services.zeebe.environment in the docker-compose.yaml file, you need to add:
ZEEBE_GATEWAY_MULTITENANCY_ENABLED=true
ZEEBE_BROKER_GATEWAY_MULTITENANCY_ENABLED=true
You can find all the environment variables from the Helm charts in the link I provided.
@nathan.loding I have added the environment variables in helm to the docker-compose file, now I’m getting an error in zeebe sayings authorized tenants is null
2023-10-14 07:33:26 2023-10-14 02:03:26.233 [] [grpc-executor-0] [] ERROR
2023-10-14 07:33:26 io.camunda.zeebe.gateway - Expected to handle gRPC request, but an unexpected error occurred
2023-10-14 07:33:26 java.lang.NullPointerException: Cannot invoke "java.util.List.contains(Object)" because "authorizedTenants" is null
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.RequestMapper.ensureTenantIdSet(RequestMapper.java:334) ~[zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.RequestMapper.lambda$ensureTenantIdsSet$0(RequestMapper.java:354) ~[zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(Unknown Source) ~[?:?]
2023-10-14 07:33:26 at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) ~[?:?]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.RequestMapper.ensureTenantIdsSet(RequestMapper.java:354) ~[zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.RequestMapper.toActivateJobsRequest(RequestMapper.java:220) ~[zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.EndpointManager.mapToBrokerRequest(EndpointManager.java:464) ~[zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.EndpointManager.activateJobs(EndpointManager.java:183) [zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.GatewayGrpcService.activateJobs(GatewayGrpcService.java:63) [zeebe-gateway-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.camunda.zeebe.gateway.protocol.GatewayGrpc$MethodHandlers.invoke(GatewayGrpc.java:2022) [zeebe-gateway-protocol-impl-8.3.0.jar:8.3.0]
2023-10-14 07:33:26 at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) [grpc-stub-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) [grpc-api-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) [grpc-api-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:351) [grpc-core-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:860) [grpc-core-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) [grpc-core-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) [grpc-core-1.58.0.jar:1.58.0]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source) [?:?]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
2023-10-14 07:33:26 at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
this is my modified docker-compose file
# While the Docker images themselves are supported for production usage,
# this docker-compose.yaml is designed to be used by developers to run
# an environment locally. It is not designed to be used in production.
# We recommend to use Kubernetes in production with our Helm Charts:
# https://docs.camunda.io/docs/self-managed/platform-deployment/kubernetes-helm/
# For local development, we recommend using KIND instead of `docker-compose`:
# https://docs.camunda.io/docs/self-managed/platform-deployment/helm-kubernetes/guides/local-kubernetes-cluster/
# This is a full configuration with Zeebe, Operate, Tasklist, Optimize, Identity, Keycloak, and Elasticsearch
# See docker-compose-core.yml for a lightweight configuration that does not include Optimize, Identity, and Keycloak.
services:
zeebe: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#zeebe
image: camunda/zeebe:${CAMUNDA_PLATFORM_VERSION}
container_name: zeebe
ports:
- "26500:26500"
- "9600:9600"
environment: # https://docs.camunda.io/docs/self-managed/zeebe-deployment/configuration/environment-variables/
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_MODE=${ZEEBE_AUTHENTICATION_MODE}
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_IDENTITY_ISSUERBACKENDURL=http://keycloak:8080/auth/realms/camunda-platform
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_IDENTITY_AUDIENCE=zeebe-api
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_CLASSNAME=io.camunda.zeebe.exporter.ElasticsearchExporter
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_URL=http://elasticsearch:9200
# default is 1000, see here: https://github.com/camunda/zeebe/blob/16b482d9205a15c0ec07ae4d86dc87f6ba9e79db/exporters/elasticsearch-exporter/src/main/java/io/camunda/zeebe/exporter/ElasticsearchExporterConfiguration.java#L259
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_BULK_SIZE=1
# allow running with low disk space
- ZEEBE_BROKER_DATA_DISKUSAGECOMMANDWATERMARK=0.998
- ZEEBE_BROKER_DATA_DISKUSAGEREPLICATIONWATERMARK=0.999
- ZEEBE_GATEWAY_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- ZEEBE_BROKER_GATEWAY_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- "JAVA_TOOL_OPTIONS=-Xms512m -Xmx512m"
restart: always
healthcheck:
test: [ "CMD-SHELL", "timeout 10s bash -c ':> /dev/tcp/127.0.0.1/9600' || exit 1" ]
interval: 30s
timeout: 5s
retries: 5
start_period: 30s
volumes:
- zeebe:/usr/local/zeebe/data
networks:
- camunda-platform
depends_on:
- elasticsearch
- identity
operate: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#operate
image: camunda/operate:${CAMUNDA_PLATFORM_VERSION}
container_name: operate
ports:
- "8081:8080"
environment: # https://docs.camunda.io/docs/self-managed/operate-deployment/configuration/
- CAMUNDA_OPERATE_ZEEBE_GATEWAYADDRESS=zeebe:26500
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_OPERATE_ELASTICSEARCH_URL=http://elasticsearch:9200
- CAMUNDA_OPERATE_ZEEBEELASTICSEARCH_URL=http://elasticsearch:9200
# For more information regarding configuration with Identity see:
# https://docs.camunda.io/docs/self-managed/operate-deployment/authentication/#identity
- SPRING_PROFILES_ACTIVE=identity-auth
- CAMUNDA_OPERATE_IDENTITY_BASEURL=http://identity:8084
- CAMUNDA_OPERATE_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_OPERATE_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_OPERATE_IDENTITY_CLIENTID=operate
- CAMUNDA_OPERATE_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPERATE_IDENTITY_AUDIENCE=operate-api
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI=http://${HOST}:18080/auth/realms/camunda-platform
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI=http://${HOST}:18080/auth/realms/camunda-platform/protocol/openid-connect/certs
- CAMUNDA_OPERATE_IDENTITY_RESOURCEPERMISSIONSENABLED=${RESOURCE_AUTHORIZATIONS_ENABLED}
- CAMUNDA_OPERATE_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
- zeebe
- identity
- elasticsearch
tasklist: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#tasklist
image: camunda/tasklist:${CAMUNDA_PLATFORM_VERSION}
container_name: tasklist
ports:
- "8082:8080"
environment: # https://docs.camunda.io/docs/self-managed/tasklist-deployment/configuration/
- CAMUNDA_TASKLIST_ZEEBE_GATEWAYADDRESS=zeebe:26500
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_TASKLIST_ELASTICSEARCH_URL=http://elasticsearch:9200
- CAMUNDA_TASKLIST_ZEEBEELASTICSEARCH_URL=http://elasticsearch:9200
# For more information regarding configuration with Identity see:
# https://docs.camunda.io/docs/self-managed/tasklist-deployment/authentication/#identity
- SPRING_PROFILES_ACTIVE=identity-auth
- CAMUNDA_TASKLIST_IDENTITY_BASEURL=http://identity:8084
- CAMUNDA_TASKLIST_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_TASKLIST_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_TASKLIST_IDENTITY_CLIENTID=tasklist
- CAMUNDA_TASKLIST_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_TASKLIST_IDENTITY_AUDIENCE=tasklist-api
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI=http://${HOST}:18080/auth/realms/camunda-platform
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI=http://${HOST}:18080/auth/realms/camunda-platform/protocol/openid-connect/certs
- CAMUNDA_TASKLIST_IDENTITY_RESOURCE_PERMISSIONS_ENABLED=${RESOURCE_AUTHORIZATIONS_ENABLED}
- CAMUNDA_TASKLIST_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
zeebe:
condition: service_started
elasticsearch:
condition: service_healthy
identity:
condition: service_healthy
connectors: # https://docs.camunda.io/docs/components/integration-framework/connectors/out-of-the-box-connectors/available-connectors-overview/
image: camunda/connectors-bundle:${CAMUNDA_CONNECTORS_VERSION}
container_name: connectors
ports:
- "8085:8080"
environment:
- ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS=zeebe:26500
- ZEEBE_CLIENT_SECURITY_PLAINTEXT=true
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_OPERATE_CLIENT_KEYCLOAK-URL=http://keycloak:8080
- CAMUNDA_OPERATE_CLIENT_CLIENT-ID=connectors
- CAMUNDA_OPERATE_CLIENT_CLIENT-SECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPERATE_CLIENT_KEYCLOAK-REALM=camunda-platform
- CAMUNDA_OPERATE_CLIENT_URL=http://operate:8080
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
env_file: connector-secrets.txt
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
- zeebe
- operate
- identity
optimize: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#optimize
image: camunda/optimize:${CAMUNDA_OPTIMIZE_VERSION}
container_name: optimize
ports:
- "8083:8090"
environment: # https://docs.camunda.io/docs/self-managed/optimize-deployment/setup/installation/#available-environment-variables
- OPTIMIZE_ELASTICSEARCH_HOST=elasticsearch
- OPTIMIZE_ELASTICSEARCH_HTTP_PORT=9200
- SPRING_PROFILES_ACTIVE=ccsm
- CAMUNDA_OPTIMIZE_ZEEBE_ENABLED=true
- CAMUNDA_OPTIMIZE_ENTERPRISE=false
- CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_OPTIMIZE_IDENTITY_CLIENTID=optimize
- CAMUNDA_OPTIMIZE_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPTIMIZE_IDENTITY_AUDIENCE=optimize-api
- CAMUNDA_OPTIMIZE_SECURITY_AUTH_COOKIE_SAME_SITE_ENABLED=false
- CAMUNDA_OPTIMIZE_UI_LOGOUT_HIDDEN=true
- CAMUNDA_OPTIMIZE_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8090/api/readyz" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
volumes:
- "./.optimize/environment-config.yaml:/optimize/config/environment-config.yaml"
restart: on-failure
networks:
- camunda-platform
depends_on:
- identity
- elasticsearch
identity: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#identity
container_name: identity
image: camunda/identity:${CAMUNDA_PLATFORM_VERSION}
ports:
- "8084:8084"
environment: # https://docs.camunda.io/docs/self-managed/identity/deployment/configuration-variables/
SERVER_PORT: 8084
IDENTITY_RETRY_DELAY_SECONDS: 30
KEYCLOAK_URL: http://keycloak:8080/auth
IDENTITY_AUTH_PROVIDER_BACKEND_URL: http://keycloak:8080/auth/realms/camunda-platform
IDENTITY_DATABASE_HOST: postgres
IDENTITY_DATABASE_PORT: 5432
IDENTITY_DATABASE_NAME: bitnami_keycloak
IDENTITY_DATABASE_USERNAME: bn_keycloak
IDENTITY_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
KEYCLOAK_INIT_OPERATE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_OPERATE_ROOT_URL: http://${HOST}:8081
KEYCLOAK_INIT_TASKLIST_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_TASKLIST_ROOT_URL: http://${HOST}:8082
KEYCLOAK_INIT_OPTIMIZE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_OPTIMIZE_ROOT_URL: http://${HOST}:8083
KEYCLOAK_INIT_WEBMODELER_ROOT_URL: http://${HOST}:8070
KEYCLOAK_INIT_CONNECTORS_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_CONNECTORS_ROOT_URL: http://${HOST}:8085
KEYCLOAK_INIT_ZEEBE_NAME: zeebe
KEYCLOAK_USERS_0_USERNAME: "demo"
KEYCLOAK_USERS_0_PASSWORD: "demo"
KEYCLOAK_USERS_0_FIRST_NAME: "demo"
KEYCLOAK_USERS_0_EMAIL: "demo@acme.com"
KEYCLOAK_USERS_0_ROLES_0: "Identity"
KEYCLOAK_USERS_0_ROLES_1: "Optimize"
KEYCLOAK_USERS_0_ROLES_2: "Operate"
KEYCLOAK_USERS_0_ROLES_3: "Tasklist"
KEYCLOAK_USERS_0_ROLES_4: "Web Modeler"
KEYCLOAK_CLIENTS_0_NAME: zeebe
KEYCLOAK_CLIENTS_0_ID: ${ZEEBE_CLIENT_ID}
KEYCLOAK_CLIENTS_0_SECRET: ${ZEEBE_CLIENT_SECRET}
KEYCLOAK_CLIENTS_0_TYPE: M2M
KEYCLOAK_CLIENTS_0_PERMISSIONS_0_RESOURCE_SERVER_ID: zeebe-api
KEYCLOAK_CLIENTS_0_PERMISSIONS_0_DEFINITION: write:*
RESOURCE_PERMISSIONS_ENABLED: ${RESOURCE_AUTHORIZATIONS_ENABLED}
healthcheck:
test: [ "CMD", "wget", "-q", "--tries=1", "--spider", "http://localhost:8082/actuator/health" ]
interval: 5s
timeout: 15s
retries: 30
start_period: 60s
restart: on-failure
volumes:
- keycloak-theme:/app/keycloak-theme
networks:
- camunda-platform
- identity-network
depends_on:
keycloak:
condition: service_healthy
postgres: # https://hub.docker.com/_/postgres
container_name: postgres
image: postgres:${POSTGRES_VERSION}
environment:
POSTGRES_DB: bitnami_keycloak
POSTGRES_USER: bn_keycloak
POSTGRES_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
restart: on-failure
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}" ]
interval: 10s
timeout: 5s
retries: 5
volumes:
- postgres:/var/lib/postgresql/data
networks:
- identity-network
keycloak: # https://hub.docker.com/r/bitnami/keycloak
container_name: keycloak
image: bitnami/keycloak:${KEYCLOAK_SERVER_VERSION}
volumes:
- keycloak-theme:/opt/bitnami/keycloak/themes/identity
ports:
- "18080:8080"
environment:
KEYCLOAK_HTTP_RELATIVE_PATH: /auth
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: admin
restart: on-failure
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:8080/auth" ]
interval: 30s
timeout: 15s
retries: 5
start_period: 30s
networks:
- camunda-platform
- identity-network
depends_on:
- postgres
elasticsearch: # https://hub.docker.com/_/elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION}
container_name: elasticsearch
ports:
- "9200:9200"
- "9300:9300"
environment:
- bootstrap.memory_lock=true
- discovery.type=single-node
- xpack.security.enabled=false
# allow running with low disk space
- cluster.routing.allocation.disk.threshold_enabled=false
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
restart: always
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:9200/_cat/health | grep -q green" ]
interval: 30s
timeout: 5s
retries: 3
volumes:
- elastic:/usr/share/elasticsearch/data
networks:
- camunda-platform
kibana:
image: docker.elastic.co/kibana/kibana:${ELASTIC_VERSION}
container_name: kibana
ports:
- 5601:5601
volumes:
- kibana:/usr/share/kibana/data
networks:
- camunda-platform
depends_on:
- elasticsearch
profiles:
- kibana
volumes:
zeebe:
elastic:
postgres:
keycloak-theme:
kibana:
networks:
# Note there are two bridge networks: One for Camunda Platform and one for Identity.
# Identity and Keycloak are part of both as they need to be accessible by platform components.
camunda-platform:
identity-network:
Hi @savi003 - you’re right, there’s one more environment variable that needs to be set and I just learned about it today! As I mentioned earlier we have lots of changes coming to our documentation around 8.3 and multi-tenancy, so this will be included in a future (very soon!) release of the docs.
You also need to set ZEEBE_GATEWAY_SECURITY_AUTHENTICATION_MODE to identity in the Zeebe Gateway configuration. See the docs here. Multi-tenancy has a dependency on Identity for authorization, so the Gateway needs to be set to use Identity.
# .env
ZEEBE_AUTHENTICATION_MODE=identity
# docker-compose.yml
services:
zeebe:
# ...
environment:
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_MODE=${ZEEBE_AUTHENTICATION_MODE}
@nathan.loding , now zeebe is working, but I’m getting errors in another services. I will wait until the full documentation is released. Thankyou for your help.
What errors are you getting? Much of the docs have now been released!
in operate
2023-10-18 21:28:52 Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Expected Identity to provide authorized tenants, see cause for details
2023-10-18 21:28:52 at java.util.concurrent.CompletableFuture.reportGet(Unknown Source) ~[?:?]
2023-10-18 21:28:52 at java.util.concurrent.CompletableFuture.get(Unknown Source) ~[?:?]
2023-10-18 21:28:52 at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:52) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-18 21:28:52 ... 37 more
2023-10-18 21:28:52 Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Expected Identity to provide authorized tenants, see cause for details
2023-10-18 21:28:52 at io.grpc.Status.asRuntimeException(Status.java:537) ~[grpc-api-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481) ~[grpc-stub-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-18 21:28:52 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
2023-10-18 21:28:52 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
2023-10-18 21:28:52 at java.lang.Thread.run(Unknown Source) ~[?:?]
in tasklist
2023-10-18 21:34:58
2023-10-18 21:34:58 _______ ___ ___ ___ _____ _ ___ _ ___ ___ ___ _____
2023-10-18 21:34:58 |_ / __| __| _ ) __| |_ _/_\ / __| |/ / | |_ _/ __|_ _|
2023-10-18 21:34:58 / /| _|| _|| _ \ _| | |/ _ \\__ \ ' <| |__ | |\__ \ | |
2023-10-18 21:34:58 /___|___|___|___/___| |_/_/ \_\___/_|\_\____|___|___/ |_|
2023-10-18 21:34:58
2023-10-18 21:34:58
2023-10-18 21:34:58 8.3.0
2023-10-18 21:34:58
2023-10-18 21:34:58
2023-10-18 21:34:58 2023-10-18 16:04:58.775 INFO 1 --- [kground-preinit] o.h.v.i.u.Version : HV000001: Hibernate Validator 8.0.1.Final
2023-10-18 21:34:59 2023-10-18 16:04:58.997 INFO 1 --- [ main] i.c.t.Application : Starting Application using Java 17.0.8.1 with PID 1 (/app/classes started by tasklist in /)
2023-10-18 21:34:59 2023-10-18 16:04:59.001 INFO 1 --- [ main] i.c.t.Application : The following 1 profile is active: "identity-auth"
2023-10-18 21:35:01 2023-10-18 16:05:01.349 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'io.camunda.tasklist.webapp.security.CustomMethodSecurityExpressionHandler' of type [io.camunda.tasklist.webapp.security.CustomMethodSecurityExpressionHandler] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-10-18 21:35:01 2023-10-18 16:05:01.351 INFO 1 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'io.camunda.tasklist.webapp.security.MethodSecurityConfig' of type [io.camunda.tasklist.webapp.security.MethodSecurityConfig$$SpringCGLIB$$0] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-10-18 21:35:01 2023-10-18 16:05:01.755 INFO 1 --- [ main] o.s.b.w.e.t.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2023-10-18 21:35:01 2023-10-18 16:05:01.773 INFO 1 --- [ main] o.a.c.h.Http11NioProtocol : Initializing ProtocolHandler ["http-nio-8080"]
2023-10-18 21:35:01 2023-10-18 16:05:01.777 INFO 1 --- [ main] o.a.c.c.StandardService : Starting service [Tomcat]
2023-10-18 21:35:01 2023-10-18 16:05:01.779 INFO 1 --- [ main] o.a.c.c.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.12]
2023-10-18 21:35:01 2023-10-18 16:05:01.902 INFO 1 --- [ main] o.a.c.c.C.[.[.[/] : Initializing Spring embedded WebApplicationContext
2023-10-18 21:35:01 2023-10-18 16:05:01.904 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2840 ms
2023-10-18 21:35:02 2023-10-18 16:05:02.128 WARN 1 --- [ main] i.c.t.e.ElasticsearchConnector : Username and/or password for are empty. Basic authentication for elasticsearch is not used.
2023-10-18 21:35:02 2023-10-18 16:05:02.621 WARN 1 --- [ main] i.c.t.e.ElasticsearchConnector : Elasticsearch cluster is not accessible
2023-10-18 21:35:02 2023-10-18 16:05:02.766 INFO 1 --- [ main] i.c.t.z.ZeebeConnector : Use plaintext connection to zeebe
2023-10-18 21:35:03 2023-10-18 16:05:03.295 INFO 1 --- [ main] o.e.c.EhcacheManager : Cache 'tokenCache' created in EhcacheManager.
2023-10-18 21:35:03 2023-10-18 16:05:03.518 WARN 1 --- [ main] .k.a.t.GraphQLJavaToolsAutoConfiguration : GraphQL introspection query disabled! This puts your server in contravention of the GraphQL specification and expectations of most clients, so use this option with caution
2023-10-18 21:35:04 2023-10-18 16:05:04.824 WARN 1 --- [ main] o.s.b.a.e.EndpointId : Endpoint ID 'usage-metrics' contains invalid characters, please migrate to a valid format.
2023-10-18 21:35:04 2023-10-18 16:05:04.862 INFO 1 --- [ main] g.k.a.w.s.GraphQLWebAutoConfiguration : Enabling cors filter
2023-10-18 21:35:04 2023-10-18 16:05:04.966 INFO 1 --- [ main] i.c.t.WebappModuleConfiguration : Starting module: webapp
2023-10-18 21:35:05 2023-10-18 16:05:05.016 INFO 1 --- [ main] i.c.t.s.SchemaStartup : SchemaStartup started.
2023-10-18 21:35:05 2023-10-18 16:05:05.016 INFO 1 --- [ main] i.c.t.s.SchemaStartup : SchemaStartup: validate schema.
2023-10-18 21:35:05 2023-10-18 16:05:05.326 INFO 1 --- [ main] i.c.t.s.SchemaStartup : SchemaStartup: schema won't be created, it either already exist, or schema creation is disabled in configuration.
2023-10-18 21:35:05 2023-10-18 16:05:05.326 INFO 1 --- [ main] i.c.t.s.SchemaStartup : SchemaStartup: migrate schema.
2023-10-18 21:35:05 2023-10-18 16:05:05.333 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-2_flownode-instance-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.419 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-6_variable-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.421 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.2.0-1_user_add_role.json
2023-10-18 21:35:05 2023-10-18 16:05:05.422 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.1.0-1_task-variable_script.json
2023-10-18 21:35:05 2023-10-18 16:05:05.423 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.4.0-0_user_script.json
2023-10-18 21:35:05 2023-10-18 16:05:05.424 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-0_process_form.json
2023-10-18 21:35:05 2023-10-18 16:05:05.425 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.4.0-1_user_delete_fields_script.json
2023-10-18 21:35:05 2023-10-18 16:05:05.426 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.2-0_task_add_follow_up.json
2023-10-18 21:35:05 2023-10-18 16:05:05.427 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-5_process-instance-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.427 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-1_metric-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.428 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.0-1_process_add_version.json
2023-10-18 21:35:05 2023-10-18 16:05:05.429 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.3.0-0_task_clean_assignee.json
2023-10-18 21:35:05 2023-10-18 16:05:05.430 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-7_draft-task-variable-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.430 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.2.0-0_user-remove-role.json
2023-10-18 21:35:05 2023-10-18 16:05:05.431 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-8_task-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.431 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.0-0_add_sequence_import.json
2023-10-18 21:35:05 2023-10-18 16:05:05.432 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.3-0_task_candidate_users.json
2023-10-18 21:35:05 2023-10-18 16:05:05.433 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-4_process-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.433 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.0-0_process_script.json
2023-10-18 21:35:05 2023-10-18 16:05:05.434 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-3_form-default-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.434 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 1.1.0-0_variable_script.json
2023-10-18 21:35:05 2023-10-18 16:05:05.435 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.2.2-1_task_add_due_.json
2023-10-18 21:35:05 2023-10-18 16:05:05.435 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Read step 8.3.0-9_task-variable-tenantId.json
2023-10-18 21:35:05 2023-10-18 16:05:05.516 WARN 1 --- [ main] o.e.c.RestClient : request [POST http://elasticsearch:9200/tasklist-migration-steps-repository-1.1.0_/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true&ignore_throttled=false&search_type=query_then_fetch&batched_reduce_size=512] returned 1 warnings: [299 Elasticsearch-8.8.2-98e1271edf932a480e4262a471281f1ee295ce6b "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-10-18 21:35:05 2023-10-18 16:05:05.557 WARN 1 --- [ main] o.e.c.RestClient : request [POST http://elasticsearch:9200/tasklist-migration-steps-repository-1.1.0_/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true&ignore_throttled=false&scroll=60000ms&search_type=query_then_fetch&batched_reduce_size=512] returned 1 warnings: [299 Elasticsearch-8.8.2-98e1271edf932a480e4262a471281f1ee295ce6b "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-10-18 21:35:05 2023-10-18 16:05:05.612 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Add new step ProcessorStep [content={"set": {"field": "bpmnProcessId", "value": ""}}, appliedDate=null, indexName=process, isApplied=false, version=8.2.0, order=0, createdDate=2023-10-18T16:05:05.612531698Z] to repository.
2023-10-18 21:35:05 2023-10-18 16:05:05.701 INFO 1 --- [ main] i.c.t.s.m.e.ElasticsearchStepsRepository : Step ProcessorStep [content={"set": {"field": "bpmnProcessId", "value": ""}}, appliedDate=null, indexName=process, isApplied=false, version=8.2.0, order=0, createdDate=2023-10-18T16:05:05.612531698Z] saved.
2023-10-18 21:35:05 2023-10-18 16:05:05.737 INFO 1 --- [ migration_1] i.c.t.s.m.e.ElasticSearchMigrator : Check if index web-session needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.738 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : Check if index form needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.738 INFO 1 --- [ migration_2] i.c.t.s.m.e.ElasticSearchMigrator : Check if index metric needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.738 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : Check if index user needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.739 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : Check if index variable needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.744 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for user, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.744 INFO 1 --- [ migration_1] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for web-session, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.745 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : Check if index migration-steps-repository needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.745 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for form, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.745 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : Check if index process needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.745 INFO 1 --- [ migration_2] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for metric, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.746 INFO 1 --- [ migration_2] i.c.t.s.m.e.ElasticSearchMigrator : Check if index import-position needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.746 INFO 1 --- [ migration_1] i.c.t.s.m.e.ElasticSearchMigrator : Check if index flownode-instance needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.747 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for variable, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.748 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : Check if index process-instance needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.751 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for process, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.752 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for migration-steps-repository, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.753 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : Check if index task-variable needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.753 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for process-instance, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.754 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : Check if index draft-task-variable needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.754 INFO 1 --- [ migration_1] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for flownode-instance, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.751 INFO 1 --- [ migration_2] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for import-position, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.752 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : Check if index task needs to migrate.
2023-10-18 21:35:05 2023-10-18 16:05:05.763 INFO 1 --- [ migration_3] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for task, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.763 INFO 1 --- [ migration_4] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for task-variable, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.768 INFO 1 --- [ migration_5] i.c.t.s.m.e.ElasticSearchMigrator : No migration needed for draft-task-variable, no previous indices found.
2023-10-18 21:35:05 2023-10-18 16:05:05.768 INFO 1 --- [ main] i.c.t.s.SchemaStartup : SchemaStartup: finished.
2023-10-18 21:35:05 2023-10-18 16:05:05.772 INFO 1 --- [ main] i.c.t.ImportModuleConfiguration : Starting module: importer
2023-10-18 21:35:05 2023-10-18 16:05:05.774 INFO 1 --- [ main] i.c.t.ArchiverModuleConfiguration : Starting module: archiver
2023-10-18 21:35:05 2023-10-18 16:05:05.808 INFO 1 --- [ main] i.c.t.w.StartupBean : INIT: DONE
2023-10-18 21:35:06 2023-10-18 16:05:06.179 INFO 1 --- [ main] i.c.t.z.ImportPositionHolderAbstract : INIT: Start import position updater...
2023-10-18 21:35:06 2023-10-18 16:05:06.484 WARN 1 --- [-worker-ELG-1-4] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.632 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.658 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.676 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.697 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.717 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.744 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.763 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.787 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.809 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.830 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.855 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.875 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.896 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.917 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.938 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.965 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:06 2023-10-18 16:05:06.989 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:07 2023-10-18 16:05:07.012 WARN 1 --- [ault-executor-0] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:07 2023-10-18 16:05:07.032 WARN 1 --- [ault-executor-2] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:07 2023-10-18 16:05:07.053 WARN 1 --- [ault-executor-2] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-18 21:35:07 2023-10-18 16:05:07.071 WARN 1 --- [ault-executor-2] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will
The “request’s security level…” warning message is pretty common and not related to multi-tenancy. I do not get that error with Operate when I used the docker-compose.yaml file you provided. Does that error occur on startup, or does it happen when you try to do something within Operate?
# While the Docker images themselves are supported for production usage,
# this docker-compose.yaml is designed to be used by developers to run
# an environment locally. It is not designed to be used in production.
# We recommend to use Kubernetes in production with our Helm Charts:
# https://docs.camunda.io/docs/self-managed/platform-deployment/kubernetes-helm/
# For local development, we recommend using KIND instead of `docker-compose`:
# https://docs.camunda.io/docs/self-managed/platform-deployment/helm-kubernetes/guides/local-kubernetes-cluster/
# This is a full configuration with Zeebe, Operate, Tasklist, Optimize, Identity, Keycloak, and Elasticsearch
# See docker-compose-core.yml for a lightweight configuration that does not include Optimize, Identity, and Keycloak.
services:
zeebe: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#zeebe
image: camunda/zeebe:${CAMUNDA_PLATFORM_VERSION}
container_name: zeebe
ports:
- "26500:26500"
- "9600:9600"
environment: # https://docs.camunda.io/docs/self-managed/zeebe-deployment/configuration/environment-variables/
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_MODE=${ZEEBE_AUTHENTICATION_MODE}
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_IDENTITY_ISSUERBACKENDURL=http://keycloak:8080/auth/realms/camunda-platform
- ZEEBE_BROKER_GATEWAY_SECURITY_AUTHENTICATION_IDENTITY_AUDIENCE=zeebe-api
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_CLASSNAME=io.camunda.zeebe.exporter.ElasticsearchExporter
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_URL=http://elasticsearch:9200
# default is 1000, see here: https://github.com/camunda/zeebe/blob/cd443f118a5c622b2902589b6d75ea9fde3ca0af/exporters/elasticsearch-exporter/src/main/java/io/camunda/zeebe/exporter/ElasticsearchExporterConfiguration.java#L259
- ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_ARGS_BULK_SIZE=1
# allow running with low disk space
- ZEEBE_BROKER_DATA_DISKUSAGECOMMANDWATERMARK=0.998
- ZEEBE_BROKER_DATA_DISKUSAGEREPLICATIONWATERMARK=0.999
- ZEEBE_GATEWAY_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- ZEEBE_BROKER_GATEWAY_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- "JAVA_TOOL_OPTIONS=-Xms512m -Xmx512m"
restart: always
healthcheck:
test: [ "CMD-SHELL", "timeout 10s bash -c ':> /dev/tcp/127.0.0.1/9600' || exit 1" ]
interval: 30s
timeout: 5s
retries: 5
start_period: 30s
volumes:
- zeebe:/usr/local/zeebe/data
networks:
- camunda-platform
depends_on:
- elasticsearch
- identity
operate: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#operate
image: camunda/operate:${CAMUNDA_PLATFORM_VERSION}
container_name: operate
ports:
- "8081:8080"
environment: # https://docs.camunda.io/docs/self-managed/operate-deployment/configuration/
- CAMUNDA_OPERATE_ZEEBE_GATEWAYADDRESS=zeebe:26500
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_OPERATE_ELASTICSEARCH_URL=http://elasticsearch:9200
- CAMUNDA_OPERATE_ZEEBEELASTICSEARCH_URL=http://elasticsearch:9200
# For more information regarding configuration with Identity see:
# https://docs.camunda.io/docs/self-managed/operate-deployment/authentication/#identity
- SPRING_PROFILES_ACTIVE=identity-auth
- CAMUNDA_OPERATE_IDENTITY_BASEURL=http://identity:8084
- CAMUNDA_OPERATE_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_OPERATE_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_OPERATE_IDENTITY_CLIENTID=operate
- CAMUNDA_OPERATE_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPERATE_IDENTITY_AUDIENCE=operate-api
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI=http://${HOST}:18080/auth/realms/camunda-platform
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI=http://${HOST}:18080/auth/realms/camunda-platform/protocol/openid-connect/certs
- CAMUNDA_OPERATE_IDENTITY_RESOURCEPERMISSIONSENABLED=${RESOURCE_AUTHORIZATIONS_ENABLED}
- camunda.operate.multiTenancy.enabled=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
- zeebe
- identity
- elasticsearch
tasklist: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#tasklist
image: camunda/tasklist:${CAMUNDA_PLATFORM_VERSION}
container_name: tasklist
ports:
- "8082:8080"
environment: # https://docs.camunda.io/docs/self-managed/tasklist-deployment/configuration/
- CAMUNDA_TASKLIST_ZEEBE_GATEWAYADDRESS=zeebe:26500
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_TASKLIST_ELASTICSEARCH_URL=http://elasticsearch:9200
- CAMUNDA_TASKLIST_ZEEBEELASTICSEARCH_URL=http://elasticsearch:9200
# For more information regarding configuration with Identity see:
# https://docs.camunda.io/docs/self-managed/tasklist-deployment/authentication/#identity
- SPRING_PROFILES_ACTIVE=identity-auth
- CAMUNDA_TASKLIST_IDENTITY_BASEURL=http://identity:8084
- CAMUNDA_TASKLIST_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_TASKLIST_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_TASKLIST_IDENTITY_CLIENTID=tasklist
- CAMUNDA_TASKLIST_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_TASKLIST_IDENTITY_AUDIENCE=tasklist-api
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI=http://${HOST}:18080/auth/realms/camunda-platform
- SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI=http://${HOST}:18080/auth/realms/camunda-platform/protocol/openid-connect/certs
- CAMUNDA_TASKLIST_IDENTITY_RESOURCE_PERMISSIONS_ENABLED=${RESOURCE_AUTHORIZATIONS_ENABLED}
- CAMUNDA_TASKLIST_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
zeebe:
condition: service_started
elasticsearch:
condition: service_healthy
identity:
condition: service_healthy
connectors: # https://docs.camunda.io/docs/components/integration-framework/connectors/out-of-the-box-connectors/available-connectors-overview/
image: camunda/connectors-bundle:${CAMUNDA_CONNECTORS_VERSION}
container_name: connectors
ports:
- "8085:8080"
environment:
- ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS=zeebe:26500
- ZEEBE_CLIENT_SECURITY_PLAINTEXT=true
- ZEEBE_CLIENT_ID=${ZEEBE_CLIENT_ID}
- ZEEBE_CLIENT_SECRET=${ZEEBE_CLIENT_SECRET}
- ZEEBE_TOKEN_AUDIENCE=zeebe-api
- ZEEBE_AUTHORIZATION_SERVER_URL=http://keycloak:8080/auth/realms/camunda-platform/protocol/openid-connect/token
- CAMUNDA_OPERATE_CLIENT_KEYCLOAK-URL=http://keycloak:8080
- CAMUNDA_OPERATE_CLIENT_CLIENT-ID=connectors
- CAMUNDA_OPERATE_CLIENT_CLIENT-SECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPERATE_CLIENT_KEYCLOAK-REALM=camunda-platform
- CAMUNDA_OPERATE_CLIENT_URL=http://operate:8080
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
env_file: connector-secrets.txt
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health/readiness" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
networks:
- camunda-platform
depends_on:
- zeebe
- operate
- identity
optimize: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#optimize
image: camunda/optimize:${CAMUNDA_OPTIMIZE_VERSION}
container_name: optimize
ports:
- "8083:8090"
environment: # https://docs.camunda.io/docs/self-managed/optimize-deployment/setup/installation/#available-environment-variables
- OPTIMIZE_ELASTICSEARCH_HOST=elasticsearch
- OPTIMIZE_ELASTICSEARCH_HTTP_PORT=9200
- SPRING_PROFILES_ACTIVE=ccsm
- CAMUNDA_OPTIMIZE_ZEEBE_ENABLED=true
- CAMUNDA_OPTIMIZE_ENTERPRISE=false
- CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_URL=http://${HOST}:18080/auth/realms/camunda-platform
- CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_BACKEND_URL=http://keycloak:8080/auth/realms/camunda-platform
- CAMUNDA_OPTIMIZE_IDENTITY_CLIENTID=optimize
- CAMUNDA_OPTIMIZE_IDENTITY_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPTIMIZE_IDENTITY_AUDIENCE=optimize-api
- CAMUNDA_OPTIMIZE_SECURITY_AUTH_COOKIE_SAME_SITE_ENABLED=false
- CAMUNDA_OPTIMIZE_UI_LOGOUT_HIDDEN=true
- CAMUNDA_OPTIMIZE_MULTITENANCY_ENABLED=${MULTITENANCY_ENABLED}
- management.endpoints.web.exposure.include=health
- management.endpoint.health.probes.enabled=true
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8090/api/readyz" ]
interval: 30s
timeout: 1s
retries: 5
start_period: 30s
volumes:
- "./.optimize/environment-config.yaml:/optimize/config/environment-config.yaml"
restart: on-failure
networks:
- camunda-platform
depends_on:
- identity
- elasticsearch
identity: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#identity
container_name: identity
image: camunda/identity:${CAMUNDA_PLATFORM_VERSION}
ports:
- "8084:8084"
environment: # https://docs.camunda.io/docs/self-managed/identity/deployment/configuration-variables/
SERVER_PORT: 8084
IDENTITY_RETRY_DELAY_SECONDS: 30
KEYCLOAK_URL: http://keycloak:8080/auth
IDENTITY_AUTH_PROVIDER_BACKEND_URL: http://keycloak:8080/auth/realms/camunda-platform
IDENTITY_DATABASE_HOST: postgres
IDENTITY_DATABASE_PORT: 5432
IDENTITY_DATABASE_NAME: bitnami_keycloak
IDENTITY_DATABASE_USERNAME: bn_keycloak
IDENTITY_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
KEYCLOAK_INIT_OPERATE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_OPERATE_ROOT_URL: http://${HOST}:8081
KEYCLOAK_INIT_TASKLIST_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_TASKLIST_ROOT_URL: http://${HOST}:8082
KEYCLOAK_INIT_OPTIMIZE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_OPTIMIZE_ROOT_URL: http://${HOST}:8083
KEYCLOAK_INIT_WEBMODELER_ROOT_URL: http://${HOST}:8070
KEYCLOAK_INIT_CONNECTORS_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
KEYCLOAK_INIT_CONNECTORS_ROOT_URL: http://${HOST}:8085
KEYCLOAK_INIT_ZEEBE_NAME: zeebe
KEYCLOAK_USERS_0_USERNAME: "demo"
KEYCLOAK_USERS_0_PASSWORD: "demo"
KEYCLOAK_USERS_0_FIRST_NAME: "demo"
KEYCLOAK_USERS_0_EMAIL: "demo@acme.com"
KEYCLOAK_USERS_0_ROLES_0: "Identity"
KEYCLOAK_USERS_0_ROLES_1: "Optimize"
KEYCLOAK_USERS_0_ROLES_2: "Operate"
KEYCLOAK_USERS_0_ROLES_3: "Tasklist"
KEYCLOAK_USERS_0_ROLES_4: "Web Modeler"
KEYCLOAK_CLIENTS_0_NAME: zeebe
KEYCLOAK_CLIENTS_0_ID: ${ZEEBE_CLIENT_ID}
KEYCLOAK_CLIENTS_0_SECRET: ${ZEEBE_CLIENT_SECRET}
KEYCLOAK_CLIENTS_0_TYPE: M2M
KEYCLOAK_CLIENTS_0_PERMISSIONS_0_RESOURCE_SERVER_ID: zeebe-api
KEYCLOAK_CLIENTS_0_PERMISSIONS_0_DEFINITION: write:*
RESOURCE_PERMISSIONS_ENABLED: ${RESOURCE_AUTHORIZATIONS_ENABLED}
MULTITENANCY_ENABLED: ${MULTITENANCY_ENABLED}
healthcheck:
test: [ "CMD", "wget", "-q", "--tries=1", "--spider", "http://localhost:8082/actuator/health" ]
interval: 5s
timeout: 15s
retries: 30
start_period: 60s
restart: on-failure
volumes:
- keycloak-theme:/app/keycloak-theme
networks:
- camunda-platform
- identity-network
depends_on:
keycloak:
condition: service_healthy
postgres: # https://hub.docker.com/_/postgres
container_name: postgres
image: postgres:${POSTGRES_VERSION}
environment:
POSTGRES_DB: bitnami_keycloak
POSTGRES_USER: bn_keycloak
POSTGRES_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
restart: on-failure
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}" ]
interval: 10s
timeout: 5s
retries: 5
volumes:
- postgres:/var/lib/postgresql/data
networks:
- identity-network
keycloak: # https://hub.docker.com/r/bitnami/keycloak
container_name: keycloak
image: bitnami/keycloak:${KEYCLOAK_SERVER_VERSION}
volumes:
- keycloak-theme:/opt/bitnami/keycloak/themes/identity
ports:
- "18080:8080"
environment:
KEYCLOAK_HTTP_RELATIVE_PATH: /auth
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: admin
restart: on-failure
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:8080/auth" ]
interval: 30s
timeout: 15s
retries: 5
start_period: 30s
networks:
- camunda-platform
- identity-network
depends_on:
- postgres
elasticsearch: # https://hub.docker.com/_/elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION}
container_name: elasticsearch
ports:
- "9200:9200"
- "9300:9300"
environment:
- bootstrap.memory_lock=true
- discovery.type=single-node
- xpack.security.enabled=false
# allow running with low disk space
- cluster.routing.allocation.disk.threshold_enabled=false
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
restart: always
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:9200/_cat/health | grep -q green" ]
interval: 30s
timeout: 5s
retries: 3
volumes:
- elastic:/usr/share/elasticsearch/data
networks:
- camunda-platform
kibana:
image: docker.elastic.co/kibana/kibana:${ELASTIC_VERSION}
container_name: kibana
ports:
- 5601:5601
volumes:
- kibana:/usr/share/kibana/data
networks:
- camunda-platform
depends_on:
- elasticsearch
profiles:
- kibana
volumes:
zeebe:
elastic:
postgres:
keycloak-theme:
kibana:
networks:
# Note there are two bridge networks: One for Camunda Platform and one for Identity.
# Identity and Keycloak are part of both as they need to be accessible by platform components.
camunda-platform:
identity-network:
this is my docker-compose.yaml file
i have created an environment variable in .env file, so that i can manage it easily
MULTITENANCY_ENABLED=true
on startup, operate is showing error and then shutting down, the error is
2023-10-19 07:16:28 2023-10-19 01:46:28.703 INFO 7 --- [ main] i.c.o.z.PartitionHolder : Partition ids can't be fetched from Zeebe. Try next round (60).
2023-10-19 07:16:29 2023-10-19 01:46:29.703 WARN 7 --- [ main] i.c.z.c.i.ZeebeCallCredentials : The request's security level does not guarantee that the credentials will be confidential.
2023-10-19 07:16:29 2023-10-19 01:46:29.709 ERROR 7 --- [ault-executor-0] i.c.z.c.i.o.OAuthCredentialsProvider : Failed while fetching credentials:
2023-10-19 07:16:29
2023-10-19 07:16:29 java.nio.file.AccessDeniedException: /usr/local/operate/.camunda
2023-10-19 07:16:29 at sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at sun.nio.fs.UnixFileSystemProvider.createDirectory(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.nio.file.Files.createDirectory(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.nio.file.Files.createAndCheckIsDirectory(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.nio.file.Files.createDirectories(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsCache.ensureCacheFileExists(OAuthCredentialsCache.java:147) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsCache.writeCache(OAuthCredentialsCache.java:74) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsProvider.lambda$shouldRetryRequest$0(OAuthCredentialsProvider.java:110) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsCache.withCache(OAuthCredentialsCache.java:111) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.oauth.OAuthCredentialsProvider.shouldRetryRequest(OAuthCredentialsProvider.java:106) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.RetriableClientFutureImpl.onError(RetriableClientFutureImpl.java:48) [zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481) [grpc-stub-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) [grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
2023-10-19 07:16:29 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
2023-10-19 07:16:29 at java.lang.Thread.run(Unknown Source) [?:?]
2023-10-19 07:16:29
2023-10-19 07:16:29 2023-10-19 01:46:29.710 WARN 7 --- [ main] i.c.o.z.PartitionHolder : Error occurred when requesting partition ids from Zeebe client: Expected Identity to provide authorized tenants, see cause for details
2023-10-19 07:16:29
2023-10-19 07:16:29 io.camunda.zeebe.client.api.command.ClientStatusException: Expected Identity to provide authorized tenants, see cause for details
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.transformExecutionException(ZeebeClientFutureImpl.java:116) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:54) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 at io.camunda.operate.zeebe.PartitionHolder.getPartitionIdsFromZeebe(PartitionHolder.java:102) ~[operate-common-8.3.0.jar!/:?]
2023-10-19 07:16:29 at io.camunda.operate.zeebe.PartitionHolder.getPartitionIdsWithWaitingTimeAndRetries(PartitionHolder.java:65) ~[operate-common-8.3.0.jar!/:?]
2023-10-19 07:16:29 at io.camunda.operate.zeebe.PartitionHolder.getPartitionIds(PartitionHolder.java:50) ~[operate-common-8.3.0.jar!/:?]
2023-10-19 07:16:29 at io.camunda.operate.zeebeimport.RecordsReaderHolder.getAllRecordsReaders(RecordsReaderHolder.java:51) ~[operate-importer-8.3.0.jar!/:?]
2023-10-19 07:16:29 at io.camunda.operate.zeebeimport.ZeebeImporter.scheduleReaders(ZeebeImporter.java:55) ~[operate-importer-8.3.0.jar!/:?]
2023-10-19 07:16:29 at io.camunda.operate.zeebeimport.ZeebeImporter.startImportingData(ZeebeImporter.java:48) ~[operate-importer-8.3.0.jar!/:?]
2023-10-19 07:16:29 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
2023-10-19 07:16:29 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMethod.invoke(InitDestroyAnnotationBeanPostProcessor.java:457) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:401) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:219) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325) ~[spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) [spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323) [spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) [spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973) [spring-beans-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:942) [spring-context-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608) [spring-context-6.0.12.jar!/:6.0.12]
2023-10-19 07:16:29 at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) [spring-boot-3.1.4.jar!/:3.1.4]
2023-10-19 07:16:29 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-3.1.4.jar!/:3.1.4]
2023-10-19 07:16:29 at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439) [spring-boot-3.1.4.jar!/:3.1.4]
2023-10-19 07:16:29 at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-3.1.4.jar!/:3.1.4]
2023-10-19 07:16:29 at io.camunda.operate.Application.main(Application.java:67) [classes!/:?]
2023-10-19 07:16:29 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
2023-10-19 07:16:29 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [operate-webapp-8.3.0-exec.jar:?]
2023-10-19 07:16:29 at org.springframework.boot.loader.Launcher.launch(Launcher.java:95) [operate-webapp-8.3.0-exec.jar:?]
2023-10-19 07:16:29 at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [operate-webapp-8.3.0-exec.jar:?]
2023-10-19 07:16:29 at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) [operate-webapp-8.3.0-exec.jar:?]
2023-10-19 07:16:29 Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Expected Identity to provide authorized tenants, see cause for details
2023-10-19 07:16:29 at java.util.concurrent.CompletableFuture.reportGet(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.util.concurrent.CompletableFuture.get(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:52) ~[zeebe-client-java-8.3.0.jar!/:8.3.0]
2023-10-19 07:16:29 ... 37 more
2023-10-19 07:16:29 Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Expected Identity to provide authorized tenants, see cause for details
2023-10-19 07:16:29 at io.grpc.Status.asRuntimeException(Status.java:537) ~[grpc-api-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481) ~[grpc-stub-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ~[grpc-core-1.58.0.jar!/:1.58.0]
2023-10-19 07:16:29 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
2023-10-19 07:16:29 at java.lang.Thread.run(Unknown Source) ~[?:?]
2023-10-19 07:16:29
2023-10-19 07:16:29 2023-10-19 01:46:29.712 INFO 7 --- [ main] i.c.o.z.PartitionHolder : Partition ids can't be fetched from Zeebe. Return empty partition ids list.
2023-10-19 07:16:29 2023-10-19 01:46:29.712 ERROR 7 --- [ main] i.c.o.z.PartitionHolder : Operate needs an established connection to Zeebe in order to retrieve Zeebe Partitions. Shutting down...
2023-10-19 07:16:29 2023-10-19 01:46:29.727 INFO 7 --- [ main] i.c.o.w.StartupBean : Shutdown elasticsearch clients.
2023-10-19 07:16:29 2023-10-19 01:46:29.735 INFO 7 --- [ main] i.c.o.w.z.o.OperationExecutor : Shutdown OperationExecutor
Multi-tenancy has a dependency on Identity for authorization, so the Gateway needs to be set to use Identity.
@nathan.loding - one related question - Since Identity is available only with proprietary license, does this mean multi-tenancy feature can be enabled only in enterprise deployments?