Missing process versions in filter list of processes

Vanguard · August 16, 2023, 9:23am

I deployed several times the same process to our self managed zeebe cluster. Every time i started a process instance, which runs successfully. As a checked the operate website i see a strange behavoir: In the filter list of versions of my process one version was missing

When i filter “Name=All, Version=All” i found a runned process instance with version 2

but if i tried to open it i got an 404 on the xml request and a message with “Diagram could not be fetched” on the website.
If i filter “Name: ConditionTest Versions: All” the process instance with version 2 is missing.
Is is a known bug or can someone explain what happend here?
I am running version 8.2.7 of camunda platform.

aleksander-dytko · August 17, 2023, 10:23am

Hi @Vanguard,

Thanks for mentioning it here.
@pedesen_1 could you take a look at this?

Thanks,
Aleksander

Vanguard · August 31, 2023, 12:36pm

Hi @aleksander-dytko @pedesen_1
What is the status of this issue? Is it a bug, you now already know and can you send me then the link to the bug ticket?

Thank you

ralfpuchert · September 1, 2023, 9:31am

Hi @Vanguard,

Could you please use the developer tools to get the response from api/processes/grouped?

There should be all versions you expect.

Thanks.

Best regards,
Ralf

Vanguard · September 1, 2023, 12:21pm

Hi @ralfpuchert
I used the developer tool, but the version i expected is not part of this result.
I inserted you here my result and you can see, that the process id ConditionTest has not the version 2. Also the process id p_18d1b105-ad91-467c-9fc1-c752aab6ba00 has the version 25, but not 26, 27 and 28. 29 is the next version, this result gave me back.

[ {
  "bpmnProcessId" : "WaitTask_3_timer",
  "name" : "WaitTask_3_timer",
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799813710704",
    "name" : "WaitTask_3_timer",
    "version" : 1,
    "bpmnProcessId" : "WaitTask_3_timer"
  } ]
}, {
  "bpmnProcessId" : "ConditionTest",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814233717",
    "name" : null,
    "version" : 10,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814233714",
    "name" : null,
    "version" : 9,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814233710",
    "name" : null,
    "version" : 8,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814233697",
    "name" : null,
    "version" : 7,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814233688",
    "name" : null,
    "version" : 6,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814153601",
    "name" : null,
    "version" : 5,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814153575",
    "name" : null,
    "version" : 4,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814153543",
    "name" : null,
    "version" : 3,
    "bpmnProcessId" : "ConditionTest"
  }, {
    "id" : "2251799814153518",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "ConditionTest"
  } ]
}, {
  "bpmnProcessId" : "ConditionTestSolo",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814153699",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "ConditionTestSolo"
  } ]
}, {
  "bpmnProcessId" : "ConditionTestWithoutDefault",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814153707",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "ConditionTestWithoutDefault"
  } ]
}, {
  "bpmnProcessId" : "p_005c088c_6c54_4794_b916_54c4589976ca",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814221015",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_005c088c_6c54_4794_b916_54c4589976ca"
  } ]
}, {
  "bpmnProcessId" : "p_030b679e-89bb-40b6-a7bb-315bb2072641",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814115225",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_030b679e-89bb-40b6-a7bb-315bb2072641"
  } ]
}, {
  "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814233595",
    "name" : null,
    "version" : 32,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814232970",
    "name" : null,
    "version" : 31,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814232892",
    "name" : null,
    "version" : 30,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814232508",
    "name" : null,
    "version" : 29,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814204945",
    "name" : null,
    "version" : 25,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814200343",
    "name" : null,
    "version" : 23,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814192948",
    "name" : null,
    "version" : 22,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814159831",
    "name" : null,
    "version" : 21,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814159827",
    "name" : null,
    "version" : 20,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814159822",
    "name" : null,
    "version" : 19,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814159818",
    "name" : null,
    "version" : 18,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814159814",
    "name" : null,
    "version" : 17,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814118880",
    "name" : null,
    "version" : 16,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814118875",
    "name" : null,
    "version" : 15,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117882",
    "name" : null,
    "version" : 14,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117878",
    "name" : null,
    "version" : 13,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117867",
    "name" : null,
    "version" : 12,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117855",
    "name" : null,
    "version" : 10,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117792",
    "name" : null,
    "version" : 9,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117785",
    "name" : null,
    "version" : 8,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117780",
    "name" : null,
    "version" : 7,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117775",
    "name" : null,
    "version" : 6,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117757",
    "name" : null,
    "version" : 5,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117753",
    "name" : null,
    "version" : 4,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117719",
    "name" : null,
    "version" : 3,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117706",
    "name" : null,
    "version" : 2,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  }, {
    "id" : "2251799814117457",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_18d1b105-ad91-467c-9fc1-c752aab6ba00"
  } ]
}, {
  "bpmnProcessId" : "p_4bfd12ad-615b-4a42-9c52-20dd62586286",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799813800456",
    "name" : null,
    "version" : 4,
    "bpmnProcessId" : "p_4bfd12ad-615b-4a42-9c52-20dd62586286"
  }, {
    "id" : "2251799813798602",
    "name" : null,
    "version" : 2,
    "bpmnProcessId" : "p_4bfd12ad-615b-4a42-9c52-20dd62586286"
  }, {
    "id" : "2251799813779324",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_4bfd12ad-615b-4a42-9c52-20dd62586286"
  } ]
}, {
  "bpmnProcessId" : "p_85094497-259e-40e7-aafe-9a17f08d95af",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799813685373",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_85094497-259e-40e7-aafe-9a17f08d95af"
  } ]
}, {
  "bpmnProcessId" : "p_cafefa62_4cb1_4346_aece_860d2e8d7705",
  "name" : null,
  "permissions" : [ ],
  "processes" : [ {
    "id" : "2251799814202309",
    "name" : null,
    "version" : 1,
    "bpmnProcessId" : "p_cafefa62_4cb1_4346_aece_860d2e8d7705"
  } ]
} ]

In difference to this result the Endpoint api/process-instances gave me a runned process for version 2.

 {
    "id" : "13510798882579166",
    "processId" : "2251799814153534",
    "processName" : "ConditionTest",
    "processVersion" : 2,
    "startDate" : "2023-08-16T07:06:46.116+0000",
    "endDate" : "2023-08-16T07:06:46.116+0000",
    "state" : "COMPLETED",
    "bpmnProcessId" : "ConditionTest",
    "hasActiveOperation" : false,
    "operations" : [ ],
    "parentInstanceId" : null,
    "rootInstanceId" : null,
    "callHierarchy" : null,
    "sortValues" : [ {
      "value" : "1692169606116",
      "valueType" : "java.lang.Long"
    }, {
      "value" : "13510798882579166",
      "valueType" : "java.lang.Long"
    } ],
    "permissions" : [ ]
  }

ralfpuchert · September 5, 2023, 9:17am

Hello @Vanguard ,

Thank you for the results. To be sure that there is maybe a bug, could you please ask for process definitions via public API?
You can use for this the swagger.ui e.g. http://<your-operate-url>/swagger-ui.html

In the result there should be your expected process definition with version like this:

{
  "items": [
    {
      "key": 2251799813685249,
      "name": "Always completing process",
      "version": 1,
      "bpmnProcessId": "always-completing-process"
    },
    {
      "key": 2251799813685251,
      "name": "Input Output Mapping Test",
      "version": 1,
      "bpmnProcessId": "Process_b1711b2e-ec8e-4dad-908c-8c12e028f32f"
    },
    {
...

Regards,
Ralf

Vanguard · September 15, 2023, 5:36am

Hello @ralfpuchert

I now tested your endpoint, and i am getting also a result, with the missing version numbers in it. So the problem must be deeper.

ralfpuchert · September 15, 2023, 11:31am

Hi @Vanguard
This looks like a unknown bug.
Do you have access to Elasticsearch instance Operate is using?
If yes we can check there for data by executing queries on Elasticsearch:
Please replace host, port if needed.

Get all process definitions:
curl --location 'http://localhost:9200/operate-process-1.0.0_*/_search' \ --data ''
Get all process definitions with version 2:

curl --location --request GET 'http://localhost:9200/operate-process-1.0.0_*/_search' \
--header 'Content-Type: application/json' \
--data '{
    "query": {
        "term": {
            "version": 2
        }
    }
}'

Thank you,

Ralf

Vanguard · September 15, 2023, 2:34pm

Hi @ralfpuchert
Yes, i have access to the elasticsearch.
But i thing, anything is not correct with your command or with my elasticsearch.
I got the following message on your first command. I expected, that there must come a big list back

curl --location --request GET 'http://localhost:9200/operate-process-1.0.0_*/_search' --header 'Content-Type: application/json' --data ''
{"took":0,"timed_out":false,"_shards":{"total":0,"successful":0,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":0.0,"hits":[]}}

Vanguard · September 15, 2023, 2:40pm

As another information: We are running elasticsearch with this image
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION:-7.17.0}
and the camunda platform in version
image: camunda/zeebe:${CAMUNDA_PLATFORM_VERSION:-8.2.12}
image: camunda/operate:${CAMUNDA_PLATFORM_VERSION:-8.2.12}

Maybe this info can help.

I am also able to change this versions immediately, because it is only a development system.

ralfpuchert · September 19, 2023, 2:08pm

ralfpuchert:

curl --location --request GET 'http://localhost:9200/operate-process-1.0.0_*/_search' \
--header 'Content-Type: application/json' \
--data '{
    "query": {
        "term": {
            "version": 2
        }
    }
}'

Hi @Vanguard, you’re right. Depending on your camunda platform version the version of the index can be different. For that we use a wildcard to get processes from that index:

curl --location --request GET 'http://localhost:9200/operate-process-*/_search' \                                             ✔ 
--header 'Content-Type: application/json' \
--data '{
    "query": {
        "term": {
            "version": 2
        }
    }
}'

Here you should see all the processes you have deployed in Zeebe. And this should also align with the processes in the Operate user interface. If not this is an unknown bug.

Best regards,
Ralf

Vanguard · September 20, 2023, 5:14am

Hi @ralfpuchert
Ok, now i got an response with content. But i can NOT find a version 2 entry with the bpmnProcessId “ConditionTest”. This is the process id, i see in the processes of the operate user interface.
Is this now this unknown bug?

ralfpuchert · September 21, 2023, 7:18am

Hi @Vanguard ,
Thank you for checking this. My understanding is: Operate (8.2.7) doesn’t have a process definition for “ConditionTest” version 2 but it shows a process instance for such a process. We checked the index where Operate stores information about process definitions. No process was found. That’s the reason that Operate can’t show the diagram and also can’t show it in “version” selection.
This is not intended.
Is this the only process definition where you have seen such behaviour?
We can try to investigate the process instances for “ConditionTest” version 2 and track the references.
For that we can look at the indices which stores data about instances for process “ConditionTest” version 2.
I will also check how such effect could happen in 8.2.7 and create a bug issue.

Best,

Ralf

Vanguard · September 21, 2023, 8:24am

Hi @ralfpuchert
No, it is not the only process definition, where i see this behaviour. As i wrote in some replys above, it also affects another one

If i call the console query, you gave me and set the version to f.e. 26, i also got no result for this process id.

Vanguard · September 27, 2023, 6:04am

Hello @ralfpuchert
can you tell me the link to the bug ticket? I will be able to watch it.

ralfpuchert · September 28, 2023, 2:49pm

Hello @Vanguard ,

Sorry for the late response. I added a bug issue for this here:

github.com/camunda/issues

Process definitions are not shown in Operate

opened 02:48PM - 28 Sep 23 UTC

engineering-issue-sync-app[bot]

component:operate public kind/bug

### Describe the bug After deploying processes with several versions and creati…ng process instances from it some versions are not shown. See details from report in forum here: https://forum.camunda.io/t/missing-process-versions-in-filter-list-of-processes/46936 ### Expected behavior All deployed processes with several version can be seen in Operate. ### Environment Seen in Camunda 8.2.12 ### Additional context --------------------------------------------------------------------------------------------- ### Acceptance Criteria ### Definition of Ready - Checklist - [ ] The bug has been reproduced by the assignee in the environment compatible with the provided one; otherwise, the issue is closed with a comment - [ ] The issue has a meaningful title, description, and testable acceptance criteria - [ ] The issue has been labeled with an appropriate `Bug-area` label - [ ] Necessary screenshots, screen recordings, or files are attached to the bug report For UI changes required to solve the bug: - [ ] Design input has been collected by the assignee ### :point_right: Handover Dev to QA - Changed components: - Side effects on other components: - Handy resources: BPMN/DMN models, plugins, scripts, REST API endpoints + example payload, etc : - Example projects: - Commands needed to test; Versions to validate: - Docker file : in case that it needed to be tested via docker share the version contain the fixed along with version of other services .  - Release version ( in which version this fixed/feature will be released): ### :green_book: Link to the test case <sup>:robot: This issue is automatically synced from: [source](https://github.com/camunda/operate/issues/5494)</sup>

Best regards,

Ralf