How to get rid of ThreadPoolExecutor did not accept task exception?


We are getting the following exception when we have a high number of asynchronous jobs(~3000):

ENGINE-03051 There was an exception while invoking the TaskListener.
Message: 'Exception while invoking TaskListener: 
Executor [java.util.concurrent.ThreadPoolExecutor@14fc23bb[Running, pool size = 10, active threads = 10, queued tasks = 6, completed tasks = 13937]] did not accept task

We have tried increasing the queue, scaling out our replicas, increasing the retry time cycle of the async activity but I still don’t understand why we get this error. Isn’t supposed that the job acquisition waits for the queue to have some space before trying to add more jobs?

This is also happening when running a “instance-deletion” batch.

Please!,any help is highly appreciated!