How to trigger multiple springboot instances of a task

[i.m.superman]

Hi to all
My process scenario: a task in the process needs to write data for 300 cities to the database. I started five springboot instances to handle this, which is 60 cities per instance. The task completes when all 5 instances are written.

All five instances have the same annotation (@ZeebeWorker(type = "INSERT_WORKER")) and only one instance was notified when the process started.

Here is my pom and properties:

<dependency>
     <groupId>io.camunda</groupId>
     <artifactId>spring-zeebe-starter</artifactId>
     <version>1.2.7</version>
</dependency>

zeebe.client.broker.contactPoint=xxx.xxx.xxx.xxx:26500
zeebe.client.worker.max-jobs-active=10
zeebe.client.worker.threads=10
zeebe.client.security.plaintext=true

Do I need to add some other configurations ?

Thanks

[hassang]

Hi @i.m.superman

Could you please share your model? If it is a single running instance then it means only a single job is to be activated.

[i.m.superman]

Here is my test model. There is nothing special.
test_multi_instance.bpmn (2.5 KB)

[hassang]

It is a single service task instance, meaning only one job will be generated.

To generate multiple jobs, you can designate your service task as a multi-instance task, supplying it with an array of items to be inserted. This way, each job will handle a single record. Alternatively, you could take a different approach, such as assigning each job to manage a batch of records (for example: 5 records per job).

Multi-instance | Camunda 8 Docs

[i.m.superman]

I started three instances and just one instance (middle in the pic) got three jobs.
Is there something wrong with my model ?

test_multi_instance.bpmn (3.0 KB)

Here is my code:

@Component
public class DemoService {
    @ZeebeWorker(type = "TEST_MULTI_INSTANCE_WORKER")
    public void execute(final JobClient client, final ActivatedJob job) throws Exception {
        System.out.println(job.getVariablesAsMap());
    }
}

[hassang]

Set zeebe.client.worker.max-jobs-active to 1 so each worker picks 1 job.

[i.m.superman]

It works.
Thanks

[hassang]

You still can have the max set to 10 but since you try with only 3 instances, a single job will pick all the 3 as long as 3 is less than the max.

[i.m.superman]

Can input collection be used as a startup variable ？

[i.m.superman]

I simulated a job that took a long time to execute and found that about 5 minutes later, the engine retried the job, so the middle instance in the picture was assigned the job again.

[hassang]

Hi @i.m.superman

This is due to timeout setting

Timeout: The time a job is assigned to the worker. If a job is not completed within this time, it can be reassigned by Zeebe to another worker.

Job workers | Camunda 8 Docs

[hassang]

You can override timeout value via configuration file

camunda:
  client:
    zeebe:
      override:
        foo:
          timeout: PT10S

Notice: foo is the worker type

[i.m.superman]

I don’t know how long this job will take because it depends on the size of the input data. Do I need to set a large timeout value? Or is there another way ? Can input collection be used as a startup variable ？

In another scenario, if I set this timeout to 1 hour, and I restart it during the instance execution, will the instance not receive a notification from the engine until 1 hour later?

[hassang]

Hi @i.m.superman

job timeout can be dynamically extended or shortened using UpdateJobTimeout gRPC command

https://docs.camunda.io/docs/components/concepts/job-workers/#timeout-update