Hello,
I’d like to know if it is possible to put additional filters for a worker, like to put a bpmn name when I define a “type” for the worker.
As it looks strange that is not already possible, I would need to maintain unique names of workers (combination of a bpmn and a service task) if I have several bpmns and a lot of service tasks and do not want to mix them.
How this one is resolved?
Task types have global scope on a broker. They are not namespaced by bpmn process models.
That is clear, but my point is that if I have 10 bpmns which are doing a job with for each bpmn inside like 30 service tasks, lets assume that my bpmns are not an atomic command which should be executed in 1 go (more less), it;s a process which will be completed in 5-10 days.
In the such approach, I will have like 300 diff workers which will execute something specific for a service task block.
By my understanding I will need to have 300 unique types, correct?
Therefore, I asked is there a better way to organize my workers with the client lib @ZeebeWorker, like a process name + a type?
My first intuition is that you are thinking about it in a way that is not how the system actually works.
Do you really need 300 worker types?
I would be surprised if that were the best way to architect a solution to your use-case using Zeebe. But without knowing the domain, I can’t say for sure.
Can you give more concrete details about your use-case?
I’m just researching here, doing small POC for my business clients.
Lets assume that I have a cluster of 3 brokers, which I want to use to orchestrate either a service or a send a command to an external service, then I wait for a callback to write a msg on a bpmn, so the process will not finish fast, it may happen to run from 1min up to few days. For this process I will have 1 bpmn.
If I have to cover more business things and not only to pilot 1 this use case, with zeebe, then I will need more bpmns, a second bpmn which will handle a different thing, does not matter.
So, If I can have multi bpmns in the brokers and as many workers as I need (to make small services and split duties to small chunks of commands), which are related to my business, I want to be able to group them in the client.
Maybe zeebe is not best candidate, but my point is that if have a client @ZeebeWorker then I would like to have more options to put as filters when the worker will be applied.
eg:
@ZeebeWorker(type = “NotifyClient1”, processId=“processDefKey1”)
@ZeebeWorker(type = “NotifyClient1”, processId=“processDefKey2”)
In this example I have NotifyClient1 2x, as the service task has the same type in 2 bpmns.
Actually… I was learning examples and also thought that i need many workers. Actually right now I use I worker for 1 service with many workflows.
And use my own router, which uses bpmn model, element id for routing.
Or there is other approach, I added input variable method in every task, and use routing by method name.
async function router(task, complete) {
const { workflowInstanceKey, bpmnProcessId, elementId } = task;
/* console.log (JSON.stringify(task)); */
switch (bpmnProcessId) {
case 'payment-retrieval':
await paymentretrieval(task, complete);
break;
default:
{
console.log('Unknown ProcessId ' + bpmnProcessId);
await complete.failure('Unknown ProcessId ' + bpmnProcessId, 0);
}
}
};
Hi @MaximMonin, thanks on the proposal for the solution.
You gave a good example how we should manage service tasks by ourselves, but from our point of view, as clients we should not write an additional layer (routers, resolves, etc) in order to handle things as they should be, the client should be extended and handle these cases for us.
Maybe to give us several match or search strategies and we decide what we will use. All these, is not related to Zeebe as zeebe, it;s related to the client which is our bridge to the wf solution.
In this case I would have a single worker and specialise the process model using custom headers on the service tasks, like this: Transactional Email Microservice with Zeebe and NestJS | Zeebe.
I’m not clear whether your Zeebe process takes days, or whether the external task takes days to get back to the Zeebe engine. Both of these are common use-cases. If it is the external task that takes a long time to get back to the engine, then this may help you think about it: How do I structure a system with Zeebe and RabbitMQ?.
Is the Spring Worker synchronous and blocking? I’m not sure about that. You could find out by reading the source code, or probably faster: just do a quick test.
If it is synchronous and blocking (ie: can process only one job at a time), then I would either use another async client, or patch the Spring one to make it async and PR it back in, or use the external task pattern described in the RabbitMQ post.
It is not a good idea to keep a job active in a worker for a long time. If the worker activates the job from the broker, then throws an unrecovered exception before passing work off to your external system, the broker will not retry that task until the maximum expected duration of the external task itself.
So you want your worker to trigger the external system and let the broker know: “This external task is underway” pretty quickly. If the broker doesn’t get this ACK back from the worker in a shorter period of time, it knows to make that job available for activation again. And if the workers cannot activate the external system after retries, it will quickly raise an incident and you can investigate.
Hi jwulf, here we are not talking about a way of development nor we are resolving a specific use case nor something will be resolved in sync or async way.
I started the topic to see is there a way to handle by default without additional development (routers, resolvers) more service task types, not to do a custom resolver and have only 1 task type in zeebe.
Is there a recommendation for the number of workers (types)?
Should we look at one bpmn as 1 worker and have an internal resolver, or keep one type for all service tasks?
The worker boundaries are are a separation of concerns - typically around reusability (whether they can be reused as a task in another process or in another part of the existing process), around functionality, or around resource requirements for scaling.
If you think of your process diagrams as the main file in a program, then the workers are functions.
Any code that gets called from multiple places gets put into its own function (a worker).
If you need to scale a function, even if it is called only in one place in your processes, then you can put it in a dedicated worker and give it more resources or scale up instances of it.
Sometimes you’ll put code into a function even if it is only used in one place, to encapsulate the concern. It may be that a particular team maintains it, or it is written using a different language.