Camunda can't take new token from Keycloak. Unable to query users

I have configured extension GitHub - camunda-community-hub/camunda-bpm-identity-keycloak: Camunda Keycloak Identity Provider Plugin within K8s cluster.
It works well, but sometimes camunda can’t take new token when previous has expired. (Or keycloak doesn’t give new one).
Temporary solution - delete pod with camunda and deployment will create new one. After this camunda works well.
Below you can find configuration and logs.

Camunda pod logs:

13-May-2021 09:53:41.309 WARNING [http-nio-8080-exec-3] org.camunda.commons.logging.BaseLogger.logWarn ENGINE-REST-HTTP500 org.camunda.bpm.engine.impl.identity.IdentityProviderException: Unable to query users
    at org.camunda.bpm.extension.keycloak.KeycloakIdentityProviderSession.requestUsersWithoutGroupId(KeycloakIdentityProviderSession.java:314)
    at org.camunda.bpm.extension.keycloak.KeycloakIdentityProviderSession.findUserByQueryCriteria(KeycloakIdentityProviderSession.java:138)
    at org.camunda.bpm.extension.keycloak.KeycloakUserQuery.executeList(KeycloakUserQuery.java:36)
    at org.camunda.bpm.engine.impl.AbstractQuery.evaluateExpressionsAndExecuteList(AbstractQuery.java:216)
    at org.camunda.bpm.engine.impl.AbstractQuery.executeSingleResult(AbstractQuery.java:238)
    at org.camunda.bpm.engine.impl.AbstractQuery.execute(AbstractQuery.java:194)
    at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28)
    at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110)
    at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
    at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
    at org.camunda.bpm.engine.impl.AbstractQuery.executeResult(AbstractQuery.java:159)
    at org.camunda.bpm.engine.impl.AbstractQuery.singleResult(AbstractQuery.java:135)
    at org.camunda.bpm.webapp.impl.security.auth.AuthenticationService.createAuthenticate(AuthenticationService.java:63)
    at org.camunda.bpm.webapp.impl.security.auth.UserAuthenticationResource.doLogin(UserAuthenticationResource.java:95)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138)
    at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:526)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:415)
    at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:376)
    at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:378)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:347)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:320)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135)
    at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356)
    at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215)
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.engine.rest.filter.CacheControlFilter.doFilter(CacheControlFilter.java:45)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.engine.rest.filter.EmptyBodyFilter.doFilter(EmptyBodyFilter.java:101)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.webapp.impl.security.filter.headersec.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:87)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.webapp.impl.security.filter.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:175)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.webapp.impl.security.filter.SecurityFilter.doFilterSecure(SecurityFilter.java:71)
    at org.camunda.bpm.webapp.impl.security.filter.SecurityFilter.doFilter(SecurityFilter.java:55)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.camunda.bpm.webapp.impl.security.auth.AuthenticationFilter$1.execute(AuthenticationFilter.java:62)
    at org.camunda.bpm.webapp.impl.security.auth.AuthenticationFilter$1.execute(AuthenticationFilter.java:60)
    at org.camunda.bpm.webapp.impl.security.SecurityActions.runWithAuthentications(SecurityActions.java:44)
    at org.camunda.bpm.webapp.impl.security.auth.AuthenticationFilter.doFilter(AuthenticationFilter.java:60)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: keycloakjar.org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 Unauthorized
    at keycloakjar.org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:81)
    at keycloakjar.org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:123)
    at keycloakjar.org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102)
    at keycloakjar.org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
    at keycloakjar.org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785)
    at keycloakjar.org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743)
    at keycloakjar.org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677)
    at keycloakjar.org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:586)
    at org.camunda.bpm.extension.keycloak.KeycloakIdentityProviderSession.requestUserById(KeycloakIdentityProviderSession.java:389)
    at org.camunda.bpm.extension.keycloak.KeycloakIdentityProviderSession.requestUsersWithoutGroupId(KeycloakIdentityProviderSession.java:262)

Login behaivor:

Camunda-identity-service client is configured according to guide. Can’t attach one more screenshot since I am new user.
Thank you in advance for response. Let me know if I need to share more info about my settings.

Hi there @11125! Thank you for the insightful post. @VonDerBeck is the maintainer of Keycloak and I’m sure he would be most happy to help you with this.

Additionally, would you be able to open an issue in the repository so that he can triage it appropriately?

Thank you so much!

I have created an issue in the repository. Thank you for advise.

1 Like

Hi there @11125,

first of all I would suggest checking your access token lifespan compared to the SSO session timeout. See Session Idle Expiration Issue · Issue #45 · camunda/camunda-bpm-identity-keycloak · GitHub.

In the long run it might be helpful for the plugin to implement an internal retry including reauthentication in order to treat such missconfigurations - cloud setup and configuration can be very confusing, and it might be better to be prepared against this kind of errors.

1 Like

don’t you mind if i will close this issue and move to github? I have already answered there - Camunda can’t take new token from Keycloak. Unable to query users · Issue #62 · camunda/camunda-bpm-identity-keycloak · GitHub

1 Like

That’s perfect :+1:

2 Likes