How can we use camunda for processing json file which contains list of objects, each object should make a some outbound request

How can we use camunda for processing json file which contains list of objects, each object should make a some outbound request.

Am using the Camunda 8 bpmn procedure right now. I have a file with 100K records in it, and I need to make a separate outbound call for each each entry.

I attempted to extract all of the data from the list using a multi-instance subprocess, but it failed and returned an error code of 502.

Also, I have tried to process on each entry level using kafka messaging ideas.

However judging by how each object entry needs to send back an acknowledgement, I guess that’s hard to accomplish using this method.

It is difficult to maintain the record as an environment variable because the file contains a significant number of records.

Could someone kindly offer their thoughts on how to resolve this problem?

I would really appreciate it if you shared a reference link or an example.

My sample code looks like this on the job worker.

@JobWorker(type = “user-creation”)
public Map userCreation(final ActivatedJob activatedJob) {
// Business logic
return Collections.emptyMap();
}

Hi @GRajaMca,

multi instance is the way to go in BPMN. But with a large collection of 100.000 elements, you have to be careful with the configuration and variable handling.

You have to chunk the data into several buckets and use nested multi-instance activities.

You can find an example on this branch of the camunda-8-examples repository: camunda-8-examples/large-multi-instance-example at large-multi-instance-example · camunda-community-hub/camunda-8-examples · GitHub.

Hope this helps, Ingo

2 Likes