Pattern Review: DMN Looping for Array Input

[StephenOTT]

Post osme pictures of your configuration from the modeler

[hetal_rathod]

Hey Stephen,

Thanks for the quick response! Below are the screenshots. Attaching the bpmn as well.

Input/output mapping. It’s a send task.

Thanks.agile_connect_network.bpmn (44.9 KB)

[StephenOTT]

So few things to explain:

1. YOu have ${elementName} as a input variable and as the variable name for “Element Variable”. Can you explain your usage here? “Element Variable” is the name of the variable that camunda will create inside each instance.

2. You have Loop Cardinality and Collection. Usually these are used as one or the other. Can you explain your usage?

3. Your collection: ${name}, this is a array of “names” ?

Where is the specific error line number? There would be a stack trace in the logs with more info about specifically where the error occurred.

[hetal_rathod]

So,

Name is defined as a script task

def names=[“Camunda System 1”, “Camunda System 3”]
execution.setVariable(“name”, name)

1. What I understood is, ${elementName} is a variable which camunda creates in each instance based on the variable names like in the example posted above for collName . Which I want to use as an input for that instance.
   Is it like it is not able to find the variable at the start of the Instance (because the variable is created after the instance starts)?

2. Yes, my collection object can have more values. But looping has to be on the Loop Cardinality. Isnt that possible?

3. Yes.

The complete stack trace :
2018-12-27 21:04:13.734 ERROR 10892 — [ntainer#0-0-C-1] org.camunda.bpm.engine.context : ENGINE-16006 BPMN Stack Trace:
Task_createL3VPNInstance (activity-start, ConcurrentExecution[dd10377e-09ec-11e9-a8a2-507b9dfa65b0], pa=catalogue-processes)
Task_createL3VPNInstance, name=createL3VPNInstance
^
|
Task_createL3VPNInstance#multiInstanceBody
^
|
Task_0722qey, name=dummy_1
^
|
Task_1nxjlhw, name=groovy-task

2018-12-27 21:04:13.743 ERROR 10892 — [ntainer#0-0-C-1] org.camunda.bpm.engine.context : ENGINE-16004 Exception while closing command context: Unknown property used in expression: ${nameElement}. Cause: Cannot resolve identifier ‘nameElement’

org.camunda.bpm.engine.ProcessEngineException: Unknown property used in expression: ${nameElement}. Cause: Cannot resolve identifier ‘nameElement’
at org.camunda.bpm.engine.impl.el.JuelExpression.getValue(JuelExpression.java:62) ~[camunda-engine-7.10.0.jar:7.10.0]
at org.camunda.bpm.engine.impl.el.JuelExpression.getValue(JuelExpression.java:50) ~[camunda-engine-7.10.0.jar:7.10.0]
at org.camunda.bpm.engine.impl.el.ElValueProvider.getValue(ElValueProvider.java:39) ~[camunda-engine-7.10.0.jar:7.10.0]

Thanks.

[steftriant]

@StephenOTT

I then used a listener to grab the DMN Output at the end of each execution of the DMN instances.

dmn listener.png664x562 36.9 KB

var combinedResult = execution.getVariable("combinedResult");
var dmn_output = execution.getVariable("dmn_output")["user"];

combinedResult.prop("values").append(dmn_output)

execution.setVariable("combinedResult", combinedResult)

Hi @StephenOTT.

I 've been reading your example here and I would like to ask you the following:

In my case where I don’t have a Business Rule Task, what must I define as the output of each instance in order to be grabbed by the End Execution Listener ?

My issue here is that my result variable is unique (my json variable) and every output of my Multi-Instance subprocess is written there.
So, this json value is overwritten after the completion of each instance

Thanks,
Steve

[steftriant]

@StephenOTT, I think that I found the way but I have to test it in a new deployment to be sure.
As an output of each instance, I’ve defined my json variable (“selectedProduct” in my case) and instead of the “combinedResult” variable of your example, I’ve defined the array of assignees for my Multi-Instance subprocess) which is created in the End Execution Listener of the 1st task (like as your Script task).

I will inform you in any case,
Steve

[steftriant]

Hi @StephenOTT and to anyone here.

I ran again my process with my changes (mentioned above) but when I try to complete the first instance of User Task in Subprocess (“Select Products from the List”), the server displays the following error :
SPIN/JACKSON-JSON-01005 Unable to create node for object of type 'Undefined’

This is the relative server log file for anyone interested :
catalina.2019-01-19.log (50.0 KB)

What I did here, was to use an End Execution Listener in the Multi-Instance Task of my Subprocess (“Select Products from the List”) in order to grab the values of my created json variable (selectedProduct) on the completion of each instance. I must mention here, that this json variable is created (via camForm function) on the completion of each instance. I don’t know if this fact plays a role for my server error.

A screenshot with the script from my Multi-Instance Task in Subprocess :

And from my process :

What I’ve been trying to achieve here, is to append the values to this json variable (selectedProduct) on the completion of each instance.

Can anyone help me please ?

Thanks in advance,
Steve

[StephenOTT]

For anyones future reference: here is another way to do the looping but with parallel execution: https://github.com/StephenOTT/camunda-concurrency-helpers-process-engine-plugin

You can use this helper to have the concurrent map update from each of your executions and then update your process variables at the end with the final result.

[steftriant]

Hi @StephenOTT.

I’ve been using Maven. Could you please tell me where (into the pom.xml file) must I write your suggested snippet (for installing the plugin as a dependency) and as a consequence the relative dependency ?

My pom.xml file :
pom.xml (1.7 KB)

As far as the version is concerned, which number must I write and where can I find it ?

Sorry for my many questions here but I’m not so experienced in some aspects.

Thank you very much,
Steve

[StephenOTT]

@steftriant the readme outlines what to do in the pom and in the processes.xml file. Which part of the read me are you unsure about?

[steftriant]

@StephenOTT, I’m not sure at which line of my pom xml file, I must include the repository source (JitPack) because I haven’t included until now any similar source as you can see in my attached file here.
Could you please edit my attached pom xml file here as far as the plugin installation as a dependency is concerned ?

Thanks a lot,
Steve

[steftriant]

@StephenOTT

Here is a updated version that uses JSON objects and javascript throughout:

array_test.dmn (1.4 KB)
array-input-dmn.bpmn (7.8 KB)

Few major changes were needed to accommodate JSON objects:

1. adding .value() to the input expression of the DMN. So now the expression for the input is: collElement.value() .
2. The Collection field of the Business Rule Task, in the Multi-Instance section was set to: ${collection.prop("collection").elements()} . An interesting aspect was that when the elements are passed to the DMN they are still passing as JSON rather than a String. And this is occurring even though the JSON is not valid json. @camunda this is a possible bug? Basically each element that was passed in the DMN was Typed as JSON but when you inspected the data in the cockpit it would show the value of the JSON object as something like System 1 . No array, no JSON brackets, etc. So it would appear as a String.
3. Array Generation was changed to:

var collection = S('{ "collection" : ["System 1", "System 3"] }');
execution.setVariable("collection", collection)

var jsonValue = S('{ "values":[] }')

execution.setVariable("combinedResult", jsonValue)

Hi @StephenOTT.

I’ve been trying to apply your steps in my BPMN model and I would like to ask you some points:

1. What does the “jsonValue” represent ? The final json variable which includes the values from all completed instances ?
2. What does the “values” represent ?
3. The “combinedResult” variable represents some collection (e.x. of users) or the final json variable which includes the values from all completed instances ?
4. What does the “user” represent here ?

var combinedResult = execution.getVariable("combinedResult");
var dmn_output = execution.getVariable("dmn_output")["user"];

combinedResult.prop("values").append(dmn_output)

execution.setVariable("combinedResult", combinedResult)

I’m asking all these as I’ve been trying to make clear (in my example) the meaning of each variable.

Thank you in advance for your time,
Steve

[StephenOTT]

1. Json value is the variable that is updated in each iteration of the multi-instance. Look at the BPMN files and see the listeners on the multi-instance.

2. Values is just a generic property name i used within the json. Each object result of the mulit-instance integration is added into the json in the “values” property’s array.

3. combined result is a json variable that is updated on each iteration. See the listeners as discussed above.

4. See the DMN file and the dmn_output variable. It is a map that is result and navigated. Remember that this is navigated as Java with the Jackson helpers on top. So you need to look at the cockpit and understand the types that are being saved in the DB and being returned to

The code snippet of

var combinedResult = execution.getVariable("combinedResult");
var dmn_output = execution.getVariable("dmn_output")["user"];

combinedResult.prop("values").append(dmn_output)

execution.setVariable("combinedResult", combinedResult)

is doing:

1. get the combined result variable which holds the current aggregated result of all executed instances
2. get the user value from the DMN result (the output of the DMN was saved into the dmn_output variable).
3. inside of the combinedResult variable in the “values” property, we apparent the dmn_output user value into the array.
4. We re-save the new variable which overwrites the previous version of the combined result variable.

[steftriant]

Hi again @StephenOTT and thanks for your immediate reply.

Based on the above points, I tried again to run my BPMN model (which is slightly different from your example) but without any success again.
When I try to complete the 1st instance of my Subprocess, I’m always taking the following server error:
SPIN/JACKSON-JSON-01005 Unable to create node for object of type 'Undefined’

test.bpmn (7.8 KB)

If you take a look at my BPMN file, you can see that:

1. I’ve used an Execution Listener instead of a Script Task for the Array generation just before the Multi-Instance Subprocess starts.
2. I’ve used the “aggregatedList” as the json variable which is updated on each iteration and holds the current aggregated result of all executed instances.
3. My collection array variable consists of multiple users (instead of multiple systems as in your example)
4. Inside of the “aggregatedList” variable in the “values” property, I append the “selectedProduct” user’s value (which is created on the completion of each instance into the Subprocess via camForm function).

Could you please tell me what I’m doing wrong?

Thanks a lot,
Steve

[ii4q7j]

Hello!
Is this relevant for 2021 and camunda 7.14? Can’t run the example, got an error:

The process could not be started. :

FEEL/SCALA-01008 Error while evaluating expression: failed to evaluate expression ‘collElement.value()’: expect Context but found ‘ValString(System 1)’

[IdemenB]

Dear all, thank you very much for your contribution in this topic, it is extremely valuable for the Camunda ecosystem.

My need is also to deal with an array, but without a multi instance. In our project, all tasks are implemented with “external task pattern” including the decision tasks. So, we call the REST API for each decision we need to make. If there is a need of “multi-instance” anywhere, we instead put the load on the worker, not to the process. So, the MI steps are designed as Non-MI, instead we load all the rows from the DB in that single task, process them all through iteration, and put the result back to the DB and just move the process into the next step. So, the Camunda engine stays as light as possible.

For most of the DB operations it is possible to do bulk operations (such as bulk update etc.) so we are fine. But with the DMN, right now we have to make a HTTP call, send the variables, receive the result and continue the loop. This is a big bottleneck. My question is, how can we send the whole set of variables in a single JSON array, let the Camunda DMN engine process each, come up with the decision for each and put all calculated decisions in a single array output and give that single array back as the HTTP response? Is that something possible?

[CamilaDemier]

I had the same problem. Has anyone found a solution?

[Webcyberrob]

Hi,

If you are using an external worker, you could use an embedded dmn engine in your worker and thus the worker can iterate over a collection, call the dmn and accumulate the result…the down side is you dont get a decision audit trail out of the box…

Regards

Rob

[Raybiwo]

i had the same proble, i change collElement.value() to collElement instead.