Avoid Persisting Process Variables

We are attempting to use Camunda in a novel way (for us, at least), as a mapping process. The BPMN is the process which delegates to DMN that contain the rules, and groovy scripts that apply the rules to the variables.

This is working great, but the problem with our current approach is that we are mapping java objects that could contain thousands of instances. For example, an Invoice with 10,000 line items (this is real data and could even be much larger).

After about 3k line items, the process begins to slow to a crawl. We have turned off history mode, but I believe that the process engine must be persisting the mapped objects to the database, and this is somehow having the dramatic impact on performance as more and more objects are mapped and modified.

To experiment, it would be great to be able to turn off any database persistence for the process variables. The process does not need to track state of any kind. I have History Mode set to none, but other than that is there a way to disable persisting process variables to the database, or is that an unrealistic request?

Hey, if your process runs synchronously then transient variables can be used as these are not persisted.

https://docs.camunda.org/manual/7.13/user-guide/process-engine/variables/#transient-variables

As mentioned by @vesiir you can use transient variables . Sample code below

    String itemsList = "{\n" +
            "  \"firstName\": \"John\",\n" +
            "  \"lastName\" : \"doe\",\n" +
            "  \"age\"      : 26,\n" +
            "  \"items\": [\n" +
            "    {\n" +
            "      \"value\"  : \"iPhone\",\n" +
            "      \"category\": \"mobile\"\n" +
            "    },\n" +
            "    {\n" +
            "      \"value\"  : \"Pixel\",\n" +
            "      \"number\": \"mobile\"\n" +
            "    }\n" +
            "  ]\n" +
            "}";
    Spin itemDataJSON = JSON(itemsList);
    TypedValue typedTransientObjectValue = Variables.objectValue(itemDataJSON, true).serializationDataFormat(Variables.SerializationDataFormats.JSON).create();
    execution.setVariable("itemsData", typedTransientObjectValue);

Hi,

You can also write a custom history event handler as an engine plugin where you have complete control over what gets persisted…

Regards

Rob

That is good advice, but be are already using that for the variables passed to the process. I am wondering about the variables created by the process with inline Groovy scripts, though. These are just “newed” up in Groovy script tasks, and I haven’t been able to find in the docs if these variables are persisted to the Camunda database or not. Do you know?

Also good advice. One big difference, however, is that we are using the Java object API.

Hi @Jeff_Bradley,

there is a section about this in the docs: Script Task | docs.camunda.org.

Hope this helps, Ingo

Thank you, that document does clarify my question about script variables. It looks like any script task result is stored as a process variable in the execution, which I expected. Am I correct that those variables will not be persisted to the database unless the execution reaches a wait state?

Hi @Jeff_Bradley,

yes.