Cannot Run Java Worker (status code 404 invalid content-type)

MrDeeb · March 24, 2025, 4:28am

Hello all,

I’m trying to develop a worker using Java with Camunda SaaS following Camunda 8 - Develop Workers (Java) course but I’m stuck at Prepare Environment step as the app won’t start due to the error:

io.camunda.zeebe.client.api.command.ClientStatusException: HTTP status code 404
invalid content-type: text/plain; charset=utf-

What I Have Tried

I have tried updating zeebe-client-java maven dependency to the latest version.
I also double-checked the credentials and followed the exact steps in the course (which is merely downloading the project then entering the credentials in properties file)
I made sure the cluster is up and running using zbctl status.

About The Error

The HTTP status code 404 error seems trivial but I’m not sure what exactly that is not found since the credentials are correct and the cluster is running.
Also, it says invalid content-type, should I change it somehow?
Same error occurs in Camunda 8 - Develop Workers (Spring Zeebe) course

Logs

[main] INFO com.camunda.academy.OrderApplication - Starting: 1 process instances for process: orderProcess
[main] INFO com.camunda.academy.OrderApplication - Generating Order(4eb67ca)
io.camunda.zeebe.client.api.command.ClientStatusException: HTTP status code 404
invalid content-type: text/plain; charset=utf-8
headers: Metadata(:status=404,date=Mon, 24 Mar 2025 03:53:07 GMT,content-type=text/plain; charset=utf-8,strict-transport-security=max-age=63072000; includeSubDomains,content-length=21)
DATA-----------------------------
default backend - 404
DATA-----------------------------

	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.transformExecutionException(ZeebeClientFutureImpl.java:116)
	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:54)
	at com.camunda.academy.OrderApplication.startProcessInstances(OrderApplication.java:101)
	at com.camunda.academy.OrderApplication.main(OrderApplication.java:56)
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNIMPLEMENTED: HTTP status code 404
invalid content-type: text/plain; charset=utf-8
headers: Metadata(:status=404,date=Mon, 24 Mar 2025 03:53:07 GMT,content-type=text/plain; charset=utf-8,strict-transport-security=max-age=63072000; includeSubDomains,content-length=21)
DATA-----------------------------
default backend - 404
DATA-----------------------------

	at java.base/java.util.concurrent.CompletableFuture.wrapInExecutionException(CompletableFuture.java:345)
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:440)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2117)
	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:52)
	... 2 more
Caused by: io.grpc.StatusRuntimeException: UNIMPLEMENTED: HTTP status code 404
invalid content-type: text/plain; charset=utf-8
headers: Metadata(:status=404,date=Mon, 24 Mar 2025 03:53:07 GMT,content-type=text/plain; charset=utf-8,strict-transport-security=max-age=63072000; includeSubDomains,content-length=21)
DATA-----------------------------
default backend - 404
DATA-----------------------------

	at io.grpc.Status.asRuntimeException(Status.java:532)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564)
	at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)

Process finished with exit code 0

Alex_Voloshyn · March 24, 2025, 8:02am

Hi @MrDeeb
I would make another test and try connecting to the gRPC gateway with another tool like Postman. If it also works then it might be something code-related.
What is returned by “java -version”?

Regards,
Alex

MrDeeb · March 24, 2025, 8:53am

Hi @Alex_Voloshyn

Connecting to the gRPC Gateway Using Postman

I’m quite new to gRPC so I’m not sure if I’m doing this properly but here’s how I did it:

I used ZEEBE_GRPC_ADDRESS from my cluster credentials as the URL in Postman.
I used gateway.proto as the service definition in Postman.
I used the Topology method for simplicity.

Here are the results of what I tried:

Got Error: Invalid protocol "grpcs://", try "grpc://" instead probably because I need to authorize my request. I tried different Auth. types in Postman with cluster credentials but couldn’t bypass this error.
Using grpc instead resulted in an internal error: Received RST_STREAM with code 2 triggered by internal client error: Protocol error.

I think I just need to authorize my request with the grpcs request,
If this is the correct question, can you tell me which Auth. type should I use in Postman?

What is returned by “java-version”?

java version "21.0.3" 2024-04-16 LTS
Java(TM) SE Runtime Environment (build 21.0.3+7-LTS-152)
Java HotSpot(TM) 64-Bit Server VM (build 21.0.3+7-LTS-152, mixed mode, sharing)

Thank you for the test suggestion!
Regards.

GotnOGuts · March 24, 2025, 4:23pm

Do you have a proxy server between your development system and your Zeebe cluster (assuming that the Zeebe cluster is SaaS)?

If so, you need to make sure that the Proxy Server speaks HTTP2,not just HTTP1.2

MrDeeb · March 25, 2025, 12:26am

Hello @GotnOGuts,

I don’t think I have a proxy server since I just cloned the demo project and entered the cluster credentials in the properties file.

Regards.

MrDeeb · March 25, 2025, 5:16am

Solution

Finally, I was able to solve this, but unfortunately, I did not know the reason behind it since it was a trial-and-error approach.

I used an old demo from Camunda 8 - Getting Started with Microservice Orchestration course as a guide to make this one work.

To solve this problem, I needed to do 2 things and only one of them was not enough to solve this error.

1. Loading Credentials From `.env` Instead of `application.properties`

I moved credentials to be loaded from .env file which I created in the project root directory.
I used java-dotenv maven dependency to achieve this.

2. Using `gatewayAddress` instead of `grpcAddress` and `restAddress`

Even though gatewayAddress is marked as deprecated, it’s the one that worked.

This is the demo after the changes (not including .env for sure).

Since I didn’t get why it didn’t work before, this solution may:

Contain unnecessary steps.
Could have been done in a more proper way (e.g. without using the deprecated method).

@nathan.loding, you might want to share this with the Education Team since this was quite hard to figure out.

Thank you @Alex_Voloshyn and @GotnOGuts for taking the time to look into it.