Memory Leak - Thread [grpc-nio-worker-ELG-1-6] has failed to stop it

Camunda 8 Topics Discussion & Questions
1

Hi Team,

When we try to restart the pod which contains the worker deployed in it, we are getting below error. Any clue why it’s happening?

2024-02-16T12:44:20Z
2024-02-16T07:14:20.419Z WARN 1 --- [ main] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [grpc-nio-worker-ELG-1-6] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
2024-02-16T12:44:20Z
java.base@17.0.10/sun.nio.ch.EPoll.wait(Native Method)
2024-02-16T12:44:20Z
java.base@17.0.10/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
2024-02-16T12:44:20Z
java.base@17.0.10/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
2024-02-16T12:44:20Z
java.base@17.0.10/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
2024-02-16T12:44:20Z
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
2024-02-16T12:44:20Z
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883)
2024-02-16T12:44:20Z
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526)
2024-02-16T12:44:20Z
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
2024-02-16T12:44:20Z
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
2024-02-16T12:44:20Z
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2024-02-16T12:44:20Z
java.base@17.0.10/java.lang.Thread.run(Thread.java:840)
2

Hi @aravindhrs - I believe this is safe to ignore. It’s a quite common warning from Tomcat. Their documentation outlines many common causes and fixes, so it might be worthwhile to review those. But I think it’s safe to ignore.

3

What is the settings[CPU,memory] are you using for POD? Did you apply tuning for worker PODs. Is this on-prem POD or public cloud POD?