Optimize can not verify Engine's version

Hi,
I am creating a poc to evaluate Optimize.
I am using Optimize 3.7.2 through docker image and Camunda Platform version 7.16.0-ee.
Optimize is not able to connect to Camunda engine somehow and failing to do so.
It is trying to access https://workflow-engine/engine-rest/engine/default/version.
I have verified using curl that this URL is accessible through Optimize’s container from CLI and it returns the version

{“version”:“7.16.0-ee”}

Logs as follows,

[EngineImportScheduler-1] TRACE o.c.o.r.e.PlatformEngineContextFactory - sending request to [https://workflow-engine/engine-rest/engine/default/version] with body []

08:32:12.569 [EngineImportScheduler-1] ERROR o.c.o.rest.engine.EngineContext - Failed to validate engine workflow-engine-qa version with error message: Engine didn't respond. Can not verify this engine's version

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn't respond. Can not verify this engine's version

at org.camunda.optimize.service.util.EngineVersionChecker.checkEngineVersionSupport(EngineVersionChecker.java:50)
[EngineImportScheduler-1] ERROR o.c.o.rest.engine.EngineContext - Failed to validate engine workflow-engine-qa version with error message: Engine didn't respond. Can not verify this engine's version.

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn't respond. Can not verify this engine's version

at org.camunda.optimize.service.util.EngineVersionChecker.checkEngineVersionSupport(EngineVersionChecker.java:50)

at org.camunda.optimize.rest.engine.EngineContext.getEngineClient(EngineContext.java:104)

at org.camunda.optimize.service.importing.engine.fetcher.EngineEntityFetcher.getEngineClient(EngineEntityFetcher.java:33)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.performTenantRequest(TenantFetcher.java:58)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.lambda$fetchTenants$0(TenantFetcher.java:44)

at org.camunda.optimize.service.importing.engine.fetcher.instance.RetryBackoffEngineEntityFetcher.fetchWithRetry(RetryBackoffEngineEntityFetcher.java:44)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.fetchTenants(TenantFetcher.java:44)

at org.camunda.optimize.service.importing.engine.mediator.TenantImportMediator.importNextPage(TenantImportMediator.java:38)

at org.camunda.optimize.service.importing.BackoffImportMediator.importNextPageRetryOnError(BackoffImportMediator.java:98)

at org.camunda.optimize.service.importing.BackoffImportMediator.runImport(BackoffImportMediator.java:46)

at org.camunda.optimize.service.importing.AbstractImportScheduler.lambda$executeImportRound$2(AbstractImportScheduler.java:99)

at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)

at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)

at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)

at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)

at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)

at org.camunda.optimize.service.importing.AbstractImportScheduler.executeImportRound(AbstractImportScheduler.java:108)

at org.camunda.optimize.service.importing.engine.EngineImportScheduler.runImportRound(EngineImportScheduler.java:57)

at org.camunda.optimize.service.importing.AbstractImportScheduler.runImportRound(AbstractImportScheduler.java:65)

at org.camunda.optimize.service.importing.AbstractImportScheduler.run(AbstractImportScheduler.java:35)

at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)

at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:829)

08:32:06.061 [EngineImportScheduler-1] ERROR o.c.o.s.i.e.f.instance.TenantFetcher - Error during fetching of entities. Please check the connection with [workflow-engine-qa]!

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn't respond. Can not verify this engine's version

at org.camunda.optimize.service.util.EngineVersionChecker.checkEngineVersionSupport(EngineVersionChecker.java:50)

at org.camunda.optimize.rest.engine.EngineContext.getEngineClient(EngineContext.java:104)

at org.camunda.optimize.service.importing.engine.fetcher.EngineEntityFetcher.getEngineClient(EngineEntityFetcher.java:33)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.performTenantRequest(TenantFetcher.java:58)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.lambda$fetchTenants$0(TenantFetcher.java:44)

at org.camunda.optimize.service.importing.engine.fetcher.instance.RetryBackoffEngineEntityFetcher.fetchWithRetry(RetryBackoffEngineEntityFetcher.java:44)

at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.fetchTenants(TenantFetcher.java:44)

at org.camunda.optimize.service.importing.engine.mediator.TenantImportMediator.importNextPage(TenantImportMediator.java:38)

at org.camunda.optimize.service.importing.BackoffImportMediator.importNextPageRetryOnError(BackoffImportMediator.java:98)

at org.camunda.optimize.service.importing.BackoffImportMediator.runImport(BackoffImportMediator.java:46)

at org.camunda.optimize.service.importing.AbstractImportScheduler.lambda$executeImportRound$2(AbstractImportScheduler.java:99)

at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)

at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)

at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)

at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)

at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)

at org.camunda.optimize.service.importing.AbstractImportScheduler.executeImportRound(AbstractImportScheduler.java:108)

at org.camunda.optimize.service.importing.engine.EngineImportScheduler.runImportRound(EngineImportScheduler.java:57)

at org.camunda.optimize.service.importing.AbstractImportScheduler.runImportRound(AbstractImportScheduler.java:65)

at org.camunda.optimize.service.importing.AbstractImportScheduler.run(AbstractImportScheduler.java:35)

at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)

at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:829)

I am not sure how to troubleshoot further.

Hello @hrishi_joshi, thank you for reaching out to us. Is your engine container up and running?

Hi @andromachirozaki ,
Yes, and I have verified the connection using curl from Optimize’s container.

Hello! @hrishi_joshi would it be possible to share with me the curl you send and response which is being returned? This error message:

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn't respond. Can not verify this engine's version

occurs when the connection is refused from the engine or the engine is not running

Hi,
Engine config is as follows,

engines:
  # An alias of the engine, which will be used for internal purposes like
  # logging and displaying which data belong to which engine.
 
  'workflow-engine-local':
    # The process engines name on the platform, this is the unique engine identifier on the platforms REST API.
    name: default
    # A default tenant to the be injected on data from this engine where no tenant is configured in the engine itself.
    # This property is only relevant in the context of a `One Process Engine Per Tenant`.
    # For details consult the Multi-Tenancy documentation.
    defaultTenant:
      # the id used for this default tenant on persisted entities
      id: null
      # the name used for this tenant when displayed in the UI
      name: null
    # A base URL that will be used for connections to the Camunda Engine REST API.
    rest: 'https://workflow-engine/engine-rest'
    # Determines whether this instance of Optimize should import definition & historical data from this engine.
    importEnabled: true
    # Determines whether this instance of Optimize should convert historical data to event data
    # usable for event based processes.
    eventImportEnabled: false
    authentication:
      # Toggles basic authentication on or off. When enabling basic
      # authentication, please be aware that you also need to adjust the values
      # of the user and password.
      # Also note, when enabled, it is required that the user has
      # * READ & READ_HISTORY permission on the Process and Decision Definition resources
      # * READ permission on the Authorization, Group, User, Deployment & Tenant resources
      # to enable users to log in and Optimize to import the engine data.
      enabled: true
      #When basic authentication is enabled, this password is used to
      #authenticate against the engine.
      password: 'a'
      #When basic authentication is enabled, this user is used to authenticate
      #against the engine.
      user: 'admin'
    # The webapps configuration allows Optimize to directly link
    # to the other Camunda Web Applications, e.g. to jump from
    # Optimize directly to a dedicated process instance in Cockpit
    webapps:
      # Defines the endpoint where to find the camunda webapps
      endpoint: 'https://workflow-engine/camunda'
      # Enables/disables linking to other Camunda Web Applications
      enabled: true 

below are some more logs I found

12:48:24.314 [qtp2002790740-14] WARN o.c.o.s.s.a.PlatformAuthenticationService - Could not log you in.

Error messages from engines:

workflow-engine-local: Could not authenticated against engine [workflow-engine-local]. Connection to engine was refused! Please check if the engine is still running.


12:48:24.475 [qtp2002790740-14] DEBUG o.c.o.r.p.NotAuthorizedExceptionMapper - Mapping NotAuthorizedException

12:48:37.631 [qtp2002790740-12] DEBUG o.c.o.r.s.SingleSignOnRequestFilter - Received new request.

12:48:37.631 [qtp2002790740-12] DEBUG o.c.o.r.s.AuthenticationCookieFilter - Authenticating null

12:48:37.632 [qtp2002790740-12] DEBUG o.c.o.r.s.AuthenticationCookieFilter - No pre-authenticated principal found in request

12:48:37.687 [qtp2002790740-12] TRACE o.c.o.r.e.PlatformEngineContextFactory - sending request to [https://workflow-engine/engine-rest/engine/default/version] with body []

12:48:37.770 [qtp2002790740-12] ERROR o.c.o.rest.engine.EngineContext - Failed to validate engine workflow-engine-local version with error message: Engine didn't respond. Can not verify this engine's version

org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn't respond. Can not verify this engine's version

at org.camunda.optimize.service.util.EngineVersionChecker.checkEngineVersionSupport(EngineVersionChecker.java:50)

at org.camunda.optimize.rest.engine.EngineContext.getEngineClient(EngineContext.java:104)

at org.camunda.optimize.service.security.EngineAuthenticationProvider.performAuthenticationCheck(EngineAuthenticationProvider.java:39)

at org.camunda.optimize.service.security.authentication.PlatformAuthenticationService.lambda$authenticateUser$0(PlatformAuthenticationService.java:102)

Thank you for the answer! Could you tell me a bit more about your set up? How did you run the engine and how did you run Optimize?

Additionally, could you perhaps share with me what happens when you visit the following url:
http://localhost:8080/camunda

Thank you!

Hi Andrea,
We are running Engine in our test environment and now trying to setup Optimize.
I am able to login to Engine and see processes deployed in Cockpit and execute workflows.
I have verified that the engine’s REST API is accessible from Optimize’s container.
Below is the screenshot from Engine. I am a bit clueless and not sure what to check further

Hey @hrishi_joshi ,

Unfortunately, if the engine is reachable from inside the Optimize container, I am not sure what else the problem could be. Another assumption is that perhaps the Optimize app is looking for the engine in a different port from where it is mounted in this container. You could try contacting our support in case you need to debug the application with someone from our team.

I hope you found this helpful.

Kind regards,
Andromachi

Do we need to configure the certificate to access the camunda-engine ?
I am getting below exception -

16:47:29.273 [EngineImportScheduler-1] ERROR o.c.o.s.i.e.f.instance.TenantFetcher - Error during fetching of entities. Please check the connection with [camunda-bpm]!
org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Engine didn’t respond. Can not verify this engine’s version
at org.camunda.optimize.service.util.EngineVersionChecker.checkEngineVersionSupport(EngineVersionChecker.java:50)
at org.camunda.optimize.rest.engine.EngineContext.getEngineClient(EngineContext.java:104)
at org.camunda.optimize.service.importing.engine.fetcher.EngineEntityFetcher.getEngineClient(EngineEntityFetcher.java:33)
at org.camunda.optimize.service.importing.engine.fetcher.instance.TenantFetcher.performTenantRequest(TenantFetcher.java:58)

I think the issue was related to SSL setup. When I setup the keystore and truststore for Optimize correctly using javax.ssl arguments. It started working. The error logs are misleading though.

Hi @hrishi_joshi ,
just fyi, we recently improved the logging for these kinds of errors with OPT-6211, this change is included in the 3.8.3 patch release.

Thanks @Helene for the information and improving logging.
Is this patch for Camunda Platform 8?

Yes, you should be able to update Optimize to the patch release if you’re using C8 as well.

Okay, so the patch release 3.8.3 is compatible with both C7 and C8.
Thanks @Helene