BPMN process details not loading in the Operate UI. Host [_] is not valid

[SubhamPramanik]

Post upgrading Zeebe to 1.1.0 along with Operate, we started noticing that Zeebe Operate UI is not showing the instance history and diagram.

The following API is failing with 404:
http://HOST/api/processes/2251799813686414/xml

Error in Operate service:

2021-10-05 13:32:07.774  INFO 8 --- [nio-8080-exec-7] o.a.c.h.Http11Processor                  : The host [_] is not valid                                          │
│  Note: further occurrences of request parsing errors will be logged at DEBUG level.                                                                                 │
│ java.lang.IllegalArgumentException: null                                                                                                                            │
│     at org.apache.tomcat.util.http.parser.Host.parse(Host.java:78) ~[tomcat-embed-core-9.0.48.jar!/:?]                                                              │
│     at org.apache.tomcat.util.http.parser.Host.parse(Host.java:45) ~[tomcat-embed-core-9.0.48.jar!/:?]                                                              │
│     at org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:298) [tomcat-embed-core-9.0.48.jar!/:?]                                                 │
│     at org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:807) [tomcat-embed-core-9.0.48.jar!/:?]                                         │
│     at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:358) [tomcat-embed-core-9.0.48.jar!/:?]                                                │
│     at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.48.jar!/:?]                                          │
│     at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) [tomcat-embed-core-9.0.48.jar!/:?]                                   │
│     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723) [tomcat-embed-core-9.0.48.jar!/:?]                                       │
│     at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.48.jar!/:?]                                           │
│     at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]                                                                                      │
│     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]                                                                                     │
│     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.48.jar!/:?]                                        │
│     at java.lang.Thread.run(Unknown Source) [?:?]

Screenshot:

Let me know of what other details you would require.

[sdorokhova]

Hi @SubhamPramanik ,
that’s an interesting one. How does your host name looks like? Anything special? Does it contain underscores? Could you show network tab of your browser dev tools (details of failing queries)?
The problem always occur for the same queries? Or it happens from time to time?

How do you run Operate? It’s not in Camunda Cloud, but rather a standalone installation? Anything special about network/firewalls?

Best regards,
Svetlana

[SubhamPramanik]

Hey @svetlana

Thank you for your response. Here’s the information.

1. There’s nothing interesting in the hostname. It’ pretty generic. No special characters. Only alphabets. It looks like: zeebe.xxxxx.xxxxxxxxx, I can share the host name (or HAR) in private if you would like to test it out for yourself.

2. There’s only 1 call that fails which I shared above. And it’s common for all the workflow instances but the process id might vary. Here are some of the screenshots:

3. We’re running Operate in our K8s cluster with a single node Zeebe configuration. Nothing special about the firewalls or network. We have nginx routing the traffic. Operate ingress annotations:

ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: false 

Regards,
Subham