Hello, please tell me about the purpose of the worker-id property.
If I start multiple instances of the service (with the same worker-id and subscribed to the same topic), each of the instances participates in the execution of tasks. The execution of one task is not duplicated.
If I run several services subscribed to the same topic, but with different worker-ids, then they work in exactly the same way, each of them participates in the execution of tasks. The execution of one task is not duplicated.
So why should I make worker-id unique?
When a worker locks a job the name of the worker is the key used when the job is completed.
You should always give your workers different names because if you don’t you could end up with a situation where a workers no longer have the security that jobs will only be completed by the worker that locked it.
Sorry revisiting this old thread as we have a similar setup.
Is there any performance impact of using the same worker Id in terms of the number of jobs that can be processed?
Effectively our workers are clones and it doesn’t matter which one starts or completes a task.
@riopat the security that @Niall talked about was to prevent two workers to get the same job and try to complete it again (undesired).
Even in HA environment, with many copies of the same worker, its important to use different (maybe random) worker ids for each instance so they can lock jobs and stay with these until completion, without having issues from other worker with same id fetching the same job and executing it again many times.
For performance you will have many clones of your worker running for sure, fetching external tasks with the same “topics”, but each worker with a distinct worker id for it to be safe for concurrent execution.
2 Likes