Microservices Orchestration with Camunda - how to use REST APIs from a microservice as a Task in process

Camunda 7 Topics Discussion & Questions
1

I have some microservices, and each provides some functionalities with REST APIS like CRUD APIs or some functionality with REST API.

If I want to use a “Service Task” in the Camunda process with Java Delegate then I can call a REST endpoint within the Java Delegate class. But the problem is that the REST API cannot provide a long time taking job. Suppose I want to perform some task that has some steps and all steps can be done within a microservice then I create a REST endpoint with all the steps.

I usually create a REST API (with URI //start) which actually encapsulates all the steps of a task. Now this can be a time taking job so I immediately return the API and runs the functionality in asynchronous manner.

Now my problem is how to update the final status of that Task?
Problem1: If I use a Service Task and call this start endpoint from a Java Delegate then that will be completed just after calling that Rest endpoint and it will move ahead in the process. Whereas I want to wait till the task (means all steps performed in the task). I am also not seeing the option to call back the status to the same task.

Should I use some polling mechanism? This means starting the task asynchronously and returning a taskRunId to poll the status in the Java Delegate. But I am not seeing this as a good approach especially after using Camunda I do not see it right to do polling to get the Job status.

I know there is an External Task way, but I think that creating a topic and polling continuously will be a performance issue. What is your opinion about the External Task pattern, I know that the microservices which are going to participate in a process are not going to be triggered so frequently, So I feel that a microservice continuously polling for the task on a topic is not good.

I feel creating a REST API and doing the actual task asynchronously is the main issue.
Any good suggestion or example for creating a Java Delegate where the task is performed by other microservice and REST APIs are only provided to call the functionalities.

2

External tasks use long polling, 1 api rest call per long polling interval, for example 60 sec.
Connected workers fetching and commiting results in out within this 1 long polling request

3

Any idea what is the long polling interval in Camunda for External task?

My question is that is this polling not a performance issue for the calling service and Camunda engine service?

How reliable is this polling (or topic) mechanism? What if I create a very long task which is going to be done in 20 mins for example?

5

You can extend lock time inside worker procedure.
And commit, polling interval doesnt break connection. I have queries with hour and more response time.

You can define interval for worker as you wish, I use 60 seconds.
Long polling is not performance issue

6

Ok, so 60 seconds is the time it maintains the connection?

So your suggestion is to use the External Task mechanism for log running tasks.
And I should use Camunda java client in the microservice where I want to poll for an External Task and do the long running steps. I hope it is reliable in both side, I mean no miss for polling the job for a topic and no miss in replying back the status back to process task in Camunda engine.

7

My production system use only external task pattern for all tasks.

1 Like
8

I am using Custom Authorization for CAMUNDA REST endpoints and that needs a JWT token.

I hope the Camunda JAVA Clients can handle the security part in that case.

9

Hi @Himanshu_Singh,

the Camunda External Task Client provides an interceptor in the ExternalTaskClientBuilder:

Here you can add a header with your token to the request.

Hope this helps, Ingo

1 Like
10

Thanks @Ingo_Richtsmeier

I have built the client with an interceptor, which adds the JWT token
I am using the Shared Tenant Model for process deployments, which means the process is deployed for all tenants but it executes with a tenant provided at the time of process execution (through the User’s attached tenant). Now I am not able to poll for all tenants because that is now hardcoded in my client as below:

        ExternalTaskClient client = ExternalTaskClient.create()
                .baseUrl("http://localhost:8080/engine-rest")
                .asyncResponseTimeout(10000)// long polling timeout
                .addInterceptor(new ClientRequestInterceptor() {

                    @Override
                    public void intercept(ClientRequestContext requestContext)
                    {
                        String jwtToken = getJwtToken();
                        requestContext.addHeader("Authorization", "Bearer "+jwtToken);
                        requestContext.addHeader("X-TENANT", "testTenant");
                    }
                })
                .build();

In the above code “X-TENANT” is used as Tenant for my request, but as the execution is attached to a tenant it is able to poll for the mentioned Tenant in the header.

I want this External Task Client to be independent of the tenant so that it can poll any task for the subscribed topic.

1 Like
11

Hi @Himanshu_Singh,

as the process engine restricts the access to member of the tenant: Multi-Tenancy | docs.camunda.org, I see two options here:

  1. The external task client logs in as a member of the tenant (change for each request).
  2. The external task client uses a super user account that has access to all tenants.

Hope this helps, Ingo

12

So if I use a JWT token for the user having access to all Tenants (means Tenant Authorizations for all tenants) then it will be able to poll the tasks for any tenant.

I will try that and let you know the result.

13

I have tried a user having Authorization for all Tenants, but looks like it is still not fetching the jobs for other tenants.
To be clear, my Camunda REST APIs demand for a tenant always and to satisfy that I pass a tenant, but I use a JWT for the user having access to all tenants and hoping that it will access all the jobs.

14

Should I disable the Tenant provider in my Camunda Spring Boot service, so that I can poll for any task from the External Task Client?

Because in case of tenant attached execution I am not able to poll for all tasks, even if I use a token having Authorization for all tenants.

15

I have disabled the TenantId provider in my Camunda Spring Boot service, and then using a User having Authorization to all tenants is able to poll for all tasks.

But I do not want to disable TenantId provider. It is perfectly fine for me to start the process attached with a tenant Id so that users are bound to login as per the tenantId. But the problem I am facing is in External Task Client as it cannot poll for all tasks from all tenants. If I add a tenantIf in the header then it becomes specific to one tenant.

16

Hi @Himanshu_Singh,

the TenantIdProvider as mentioned here: Overview (Camunda Platform Javadocs 7.16.1-ee) does the opposite job than you need in your external task worker.

The TenantIdProvider implemented here: Multi-Tenancy | docs.camunda.org uses the user database to get the tenantId for the user who starts the process instance.

In the external task worker the tenant Id is already set at the process instance and could not be changed. Having a single external task lacks the concept of data separation. That’s what multi tenancy was invented for.

You can either start an external task worker for each tenant or use a round robin login mechanism and check each request for a different tenant.

Hope this helps, Ingo

17

I am using Camunda Platform Version 7.13.

ok, I understand that in the case of Shared Definition, the process instance is already attached with a tenant, and to follow data separation I have to use different clients.

Round robin login mechanism should work, I am only concerned that it should not be like performance issue if I create multiple clients polling to the process engine.

The client is created with a tenant and the call to poll is always polling for the task. How to run it in a round robin fashion? I don’t want to open multiple external clients for different tenants at the same time in one micro service.

18

@Ingo_Richtsmeier
Any idea if the below methods from TopicSubscriptionBuilder class can help in this case to fetch for any tenant from External Task Client:


  /**
   * Filter for external tasks without tenant
   * @return the builder
   */
  TopicSubscriptionBuilder withoutTenantId();

  /**
   * @param tenantIds to filter for external tasks that are supposed to be fetched and locked
   * @return the builder
   */
  TopicSubscriptionBuilder tenantIdIn(String... tenantIds);
19