Custom OIDC Provider configuration with self hosted service (with docker compose)

Camunda 8 Topics
1

Hello,

I’m trying to set up a self hosted environment of Camunda for development, but run into some issue/question.
(Using the camunda/camunda-platform github docker-compose for it (version is 8.5).)

I know that Keycloak can be excluded when I use my own OIDC provider. But is Identity needed?
I’ve found different statements about it, in forum topics:

Identity 8.4.3 - Failing to connect to an OpenID Connect provider - Microsoft Entra ID This one’s written in Febr 22: “When using a custom OIDC provider, Identity isn’t needed at all. You don’t need to start that container/pod.”

Identity: What is the purpose? May 28. Here it states: “I think the simplest way to think of Identity is that it acts as the glue between an OIDC provider (Keycloak by default, but supports others) and the rest of the Camunda stack.”

So is it needed or not?
If needed, how can i configure it in the docker-compose file, so that it would use my own provider, instead of Keycloak.

I’ve tried to paste and fill these environments somewhere in the docker-compose (Connect to an OpenID Connect provider | Camunda 8 Docs).
But it’s not clear. In which service(s) do I need to use them? If I copied them into the identity service, it still tried to connect to keycloak, no option to change it it seems.

CAMUNDA_IDENTITY_TYPE=GENERIC
CAMUNDA_IDENTITY_BASE_URL=<IDENTITY_URL>
CAMUNDA_IDENTITY_ISSUER=<URL_OF_ISSUER>
CAMUNDA_IDENTITY_ISSUER_BACKEND_URL=<URL_OF_ISSUER> // this is used for container to container communication
CAMUNDA_IDENTITY_CLIENT_ID=<Client ID from Step 2>
CAMUNDA_IDENTITY_CLIENT_SECRET=<Client secret from Step 2>
CAMUNDA_IDENTITY_AUDIENCE=<Audience from Step 2>
IDENTITY_INITIAL_CLAIM_NAME=<Initial claim name  if not using the default "oid">
IDENTITY_INITIAL_CLAIM_VALUE=<Initial claim value>

When I tried the Camunda stack without the Identity and Keycloak, the closest point that I could reach is if I tried to enter the Operate page, my custom OIDC’s login page came in. I logged in, but after it called back the http://localhost:8081/identity-callback url, Operate gave me a nopermission page. In the log it throws a 404 error, without an error body. It’s unclear to me what gave this error 404.

I feel that the docker-compose part of this custom OIDC provider is not documented well enough.
Please help me with some instruction or docs on how to connect it correctly.

Thanks in advance,
Bence

2

Hi @Benciky, welcome to the forums! Both the forum comments you linked are from me, so this is a good chance to clarify. In the first link, that comment concludes with:

In the upcoming 8.5 release, the Identity service will have new features that further enable custom integrations, and the image may be needed then. But not for 8.4!

When the “bring your own provider” was first introduced it didn’t require Identity. However, the architecture changed in 8.5 and now Identity is required. Identity is what enables this for most of the other services, so you only need to configure the OIDC provider in Identity (in other words, the environment variables you included are for the Identity application). Web Modeler and Connectors do require some additional configuration, which is documented on the same page a little further down.

1 Like
3

Hi @nathan.loding!

Thanks for the fast and understandable reply.
I’ve retried as you and the docs mentioned, but the there’s one thing i still don’t understand.

The Identity service gives this error:
Failure #2. Unable to connect to Keycloak.

Keycloak is excluded from the services in my docker-compose.

I’ve copied and filled in the environments into Identity service, stated in this page:

This is how my docker compose idenity service is configured now:

identity: # https://docs.camunda.io/docs/self-managed/platform-deployment/docker/#identity
    container_name: identity
    image: camunda/identity:8.5.0
    ports:
      - "8084:8084"
    environment: # https://docs.camunda.io/docs/self-managed/identity/deployment/configuration-variables/
      SERVER_PORT: 8084
      IDENTITY_RETRY_DELAY_SECONDS: 30
      CAMUNDA_IDENTITY_TYPE: GENERIC
      CAMUNDA_IDENTITY_BASE_URL: http://identity:8084
      CAMUNDA_IDENTITY_ISSUER: http://localhost:8085
      CAMUNDA_IDENTITY_ISSUER_BACKEND_URL: http://host.docker.internal:8085
      CAMUNDA_IDENTITY_CLIENT_ID: identity
      CAMUNDA_IDENTITY_CLIENT_SECRET: ${LOGIN_ADMIN_SECRET:?error}
      CAMUNDA_IDENTITY_AUDIENCE:
      IDENTITY_INITIAL_CLAIM_NAME:
      IDENTITY_INITIAL_CLAIM_VALUE:   
      # KEYCLOAK_URL: http://keycloak:8080/auth
      # IDENTITY_AUTH_PROVIDER_BACKEND_URL: http://keycloak:8080/auth/realms/camunda-platform
      # IDENTITY_DATABASE_HOST: postgres
      # IDENTITY_DATABASE_PORT: 5432
      # IDENTITY_DATABASE_NAME: bitnami_keycloak
      # IDENTITY_DATABASE_USERNAME: bn_keycloak
      # IDENTITY_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"
      # KEYCLOAK_INIT_OPERATE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
      # KEYCLOAK_INIT_OPERATE_ROOT_URL: http://localhost:8081
      # KEYCLOAK_INIT_TASKLIST_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
      # KEYCLOAK_INIT_TASKLIST_ROOT_URL: http://localhost:8082
      # KEYCLOAK_INIT_OPTIMIZE_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
      # KEYCLOAK_INIT_OPTIMIZE_ROOT_URL: http://localhost:8083
      # KEYCLOAK_INIT_WEBMODELER_ROOT_URL: http://localhost:8070
      # KEYCLOAK_INIT_CONNECTORS_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
      # KEYCLOAK_INIT_CONNECTORS_ROOT_URL: http://localhost:8085
      # KEYCLOAK_INIT_ZEEBE_NAME: zeebe
      # KEYCLOAK_USERS_0_USERNAME: "demo"
      # KEYCLOAK_USERS_0_PASSWORD: "demo"
      # KEYCLOAK_USERS_0_FIRST_NAME: "demo"
      # KEYCLOAK_USERS_0_EMAIL: "demo@acme.com"
      # KEYCLOAK_USERS_0_ROLES_0: "Identity"
      # KEYCLOAK_USERS_0_ROLES_1: "Optimize"
      # KEYCLOAK_USERS_0_ROLES_2: "Operate"
      # KEYCLOAK_USERS_0_ROLES_3: "Tasklist"
      # KEYCLOAK_USERS_0_ROLES_4: "Web Modeler"
      # KEYCLOAK_CLIENTS_0_NAME: zeebe
      # KEYCLOAK_CLIENTS_0_ID: zeebe
      # KEYCLOAK_CLIENTS_0_SECRET: ${LOGIN_ADMIN_SECRET:?error}
      # KEYCLOAK_CLIENTS_0_TYPE: M2M
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_0_RESOURCE_SERVER_ID: zeebe-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_0_DEFINITION: write:*
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_1_RESOURCE_SERVER_ID: operate-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_1_DEFINITION: write:*
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_2_RESOURCE_SERVER_ID: tasklist-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_2_DEFINITION: write:*
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_3_RESOURCE_SERVER_ID: optimize-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_3_DEFINITION: write:*
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_4_RESOURCE_SERVER_ID: tasklist-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_4_DEFINITION: read:*
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_5_RESOURCE_SERVER_ID: operate-api
      # KEYCLOAK_CLIENTS_0_PERMISSIONS_5_DEFINITION: read:*
      # Set to 'true' to enable multi-tenancy across all components
      # This requires use of identity for authentication
      #
      #  ZEEBE_AUTHENTICATION_MODE=identity
      #
      MULTITENANCY_ENABLED: false
      # Set to 'true' to enable resource based authorizations for users and groups
      # This can be used to limit access for users or groups to view/update specific
      # processes and decisions in Operate and Tasklist
      RESOURCE_PERMISSIONS_ENABLED: false
    healthcheck:
      test: [ "CMD", "wget", "-q", "--tries=1", "--spider", "http://localhost:8082/actuator/health" ]
      interval: 5s
      timeout: 15s
      retries: 30
      start_period: 60s
    restart: on-failure
    # volumes:
      # - keycloak-theme:/app/keycloak-theme
    networks:
      - camunda-platform
      # - identity-network
    # depends_on:
      # keycloak:
        # condition: service_healthy

Thanks,
Bence

4

@Benciky - it’s possible the error code is mistakenly saying “Keycloak” when it really means “OIDC Provider” … looking at your configuration, you have the issuer set to http://localhost:8085, but in Docker networking localhost is still referencing the local container, and you don’t have an OIDC provider configured inside the Identity container; you should probably be using host.docker.internal like you are on the backend URL. Your audience and claim variables aren’t set, which may cause issues depending on your configuration (I believe audience is always required).

5

@nathan.loding

Sadly it doesn’t solve the problem.

SERVER_PORT: 8084
IDENTITY_RETRY_DELAY_SECONDS: 30
CAMUNDA_IDENTITY_TYPE: GENERIC
CAMUNDA_IDENTITY_BASE_URL: http://identity:8084
CAMUNDA_IDENTITY_ISSUER: http://host.docker.internal:8085
CAMUNDA_IDENTITY_ISSUER_BACKEND_URL: http://host.docker.internal:8085
CAMUNDA_IDENTITY_CLIENT_ID: identity
CAMUNDA_IDENTITY_CLIENT_SECRET: ${LOGIN_ADMIN_SECRET:?error}
CAMUNDA_IDENTITY_AUDIENCE: identity
IDENTITY_INITIAL_CLAIM_NAME: oid 
IDENTITY_INITIAL_CLAIM_VALUE: admin

It seems that whatever i try, it’s not reaching out to my oidc.
I’ve tried to use my oidc with Operate, and it has reached out to it, with this configuration:

- CAMUNDA_OPERATE_IDENTITY_BASEURL=http://identity:8084
- CAMUNDA_OPERATE_IDENTITY_ISSUER_URL=http://localhost:8085
- CAMUNDA_OPERATE_IDENTITY_ISSUER_BACKEND_URL=http://host.docker.internal:8085

It seems to me that the problem lies somewhere else.

6

Hi @Benciky - I took this back to the engineering team and we found some missing details in our documentation :frowning: (the team is working on updating the docs right now!). You need one additional environment variable for Identity: SPRING_PROFILES_ACTIVE=oidc. Try adding that and let me know if that works or not!

7

Hello @nathan.loding!

Thanks for the feedback.
I’ve tried and it works with the SPRING_PROFILES_ACTIVE=oidc environment. I had to make some change in my oidc provider, but Identity’s trying to use that now.
However there’s some other problem and question.

In this picture it shows that postgres only needed by KeyCloak. https://helm.camunda.io/imgs/camunda-platform-8-self-managed-architecture-diagram-combined-ingress.png
Does Identity still need postgres, even if custom oidc is used?
It seems because if I comment out or doesn’t configure these environment variables, then Identity fails to init:

IDENTITY_DATABASE_HOST: postgres
IDENTITY_DATABASE_PORT: 5432
IDENTITY_DATABASE_NAME: bitnami_keycloak
IDENTITY_DATABASE_USERNAME: bn_keycloak
IDENTITY_DATABASE_PASSWORD: "#3]O?4RGj)DE7Z!9SA5"

The other problem is that after my oidc’s auth step, it should call back Identity at http://localhost:8084/auth/login-callback. But Identity gives an error:

ERROR 1 --- [nio-8084-exec-5] o.apache.coyote.http11.Http11Processor   : Error processing request

org.apache.coyote.http11.HeadersTooLargeException: An attempt was made to write more data to the response headers than there was room available in the buffer. Increase maxHttpHeaderSize on the connector or write less data into the response headers.
        at org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Http11OutputBuffer.java:463) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.java:419) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.java:406) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.http11.Http11OutputBuffer.sendHeader(Http11OutputBuffer.java:368) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor.java:1049) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:377) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.Response.action(Response.java:210) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.Response.sendHeaders(Response.java:447) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:287) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:246) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.catalina.connector.Response.finishResponse(Response.java:416) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:373) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.19.jar!/:na]
        at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]

Could you please ask the engineering team if there are any environment value or configuration available for the Identity app to configure the maxHttpHeaderSize?
Or is this error an indication of an another problem?

Thanks in advance,
Bence

8

@Benciky - Identity does require Postgres without Keycloak, because it needs a data store to store the role/claim/etc. mappings and credentials. I’ll ask the teams to double check that documentation and diagrams are accurate!

I’m not sure on the callback issue, I’ll see what I can find out!

1 Like
9

@nathan.loding
Thanks for the reply.
I understand that Keycloak need postgres, but does Identity needs it, if I use my own oidc instead of Keycloak?

Thanks for asking around.

10

@Benciky - oops, typo! :joy: I just edited the previous reply, it should have been “without” instead of “with”. Yes, Identity requires Postgres without Keycloak to store all the mappings between claims and roles and credentials etc.

11

@Benciky - I chatted with the Identity engineers, and there isn’t a direct environment variable to configure this as it isn’t something we’ve encountered before or had a request for. Would you be willing to share some details - either here in the thread, or by sending me a private chat on the forum - about your OIDC provider and how you have it configured? Identity is configured for an 8kb header size, and the engineers are curious what is sending a header larger than that, why it’s sending a large header, and if this is something they need to make more easily configurable. The more details the better!

That said, because Identity is built on Spring Boot, you should be able to provide an application.yaml file to the application inside the container and configure the header size as described here: https://www.baeldung.com/spring-boot-max-http-header-size

12

Hi, I am having same issue with OIDC Provider setup. I have been using Helm Chart to setup Identity. I am beginner of kubernetes. Could you please guide me how can I set this variable in helm values ?

13

@kvkirankumar - the application.yml file isn’t part of the Helm values directly (though some values are starting with 8.5). There’s a couple different ways to solve this problem. This guide should help:

1 Like
14

Hello @nathan.loding,
I am doing same configuration with docker compose but with Azure Entra ID. In the documentation, there is nothing about zeebe. Does it use the same app registration as operate, tasklist, optimize? What do I need to add/remove in the environment variable of zeebe?
Is it possible to have sample of docker compose in the documentation, same as helm, please. So we just need to add the client ID, tenant ID and client Secret at the right place and that’s it.
The documentation is again really not easy to follow, with really few explanation and it’s difficult to understand what to add, what to remove where.

15

Hi @kamilleB - I would open a new topic with your questions, since they aren’t related to a self-hosted OIDC provider like this topic.

1 Like
16

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.