Problem with active processes when migration from 8.7 to 8.8

Camunda 8 Topics
, , ,
1

Hello. I’m using self-managed version for local development and try to update to new version. I have finished and active processess on old version. And what i did

  1. update to latest patch zeebe -8.7.12 operate - 8.7.11 tasklist - 8.7.7

  2. In application.yaml

    1. add spring.profiles.active: “process-migration,task-migration,operate,tasklist,broker,consolidated-auth” for new orchestration
    2. remove camundaExporter
    3. Set camunda.operate.importerEnabled = true camunda.tasklist.importerEnabled = true

  3. start docker compose with new version orchestatration

  4. Open kibana and see new indexes created and data for example from tasklist-task-8.5.0_ moved to tasklist-task-8.8.0_

  5. Open Operate and try to edit active process(complete task\add variable) started from previous version, but API for modification always return NOT_FOUND.

  6. But at the same time, the API for searching for tasks and processes finds active processes.

Maybe someone can tell me how to implement the migration step by step, since there are points that are not obvious from the documentation.

In docker container exception when trying to do sth with active process:

[2025-10-21 16:29:44.923] [operation_executor_1] ERROR

	io.camunda.operate.webapp.zeebe.operation.AbstractOperationHandler - Unable to process operation with id 3479645359974468945. Reason: io.camunda.service.exception.ServiceException: Command 'MODIFY' rejected with code 'NOT_FOUND': Expected to modify process instance but no process instance found with key '2251799813686594'. Will NOT be retried.

java.util.concurrent.CompletionException: io.camunda.service.exception.ServiceException: Command 'MODIFY' rejected with code 'NOT_FOUND': Expected to modify process instance but no process instance found with key '2251799813686594'

	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)

	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)

	at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source)

	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source)

	at java.base/java.util.concurrent.CompletableFuture$Completion.run(Unknown Source)

	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

	at java.base/java.lang.Thread.run(Unknown Source)

Caused by: io.camunda.service.exception.ServiceException: Command 'MODIFY' rejected with code 'NOT_FOUND': Expected to modify process instance but no process instance found with key '2251799813686594'

	at io.camunda.service.exception.ErrorMapper.mapError(ErrorMapper.java:50)

	at io.camunda.service.ApiServices.lambda$sendBrokerRequestWithFullResponse$0(ApiServices.java:71)
2

Hello @Bantexx,

This is a known issue that can occur during migration from Camunda 8.7 to 8.8. The NOT_FOUND errors you’re experiencing when trying to modify active processes have several potential causes:

Root Causes

1. API Deprecation

The Operate API is deprecated in Camunda 8.8 and will be removed entirely in 8.10. This deprecation can cause issues with existing API calls after migration.

2. State Desynchronization

The most likely cause is a desynchronization between Operate’s Elasticsearch index and the actual Zeebe state. While Operate shows the process instances as active, Zeebe (the workflow engine) may no longer recognize them, resulting in NOT_FOUND errors when attempting modifications.

3. Task Type Mismatch

If your user tasks are job worker-based rather than Camunda user tasks, the new APIs won’t be able to interact with them properly.

Recommendations

Immediate Solutions:

  1. Switch to the new Camunda 8 REST API (v2) for all user task and process instance operations instead of the deprecated Operate API

  2. Verify your task implementation type in your BPMN models - ensure you’re using Camunda user tasks, not job worker-based tasks

  3. Check the actual state in Zeebe - the process instances may no longer exist in the engine even though they appear in Operate

If Issues Persist:

Unfortunately, there’s currently no official tool or documented procedure to force resynchronization between Operate and Zeebe. If the desynchronization issue affects critical processes, you may need to:

  • Cancel the affected process instances
  • Start new instances with the updated process definition

Migration Considerations

Also ensure that your migration plan is valid and all active elements have proper mappings, as not all process instances can be successfully migrated.

References

Could you share more details about:

  1. What specific API endpoints you’re using?
  2. The task implementation type in your BPMN models?
  3. Whether you can see these process instances in Zeebe directly?

This will help determine the exact cause and best solution for your situation.

3

Hello, before upgrade on 8.8 version i used v1 version for tasklist and operate and v2 for camunda API.

  1. Since 8.7 it was camunda user task

  2. What does it mean? How can i see in Zeebe? I see in operate

Maybe it’s problem with config when i start process migration.

I upload my docker compose file for orchestration and application.yaml.

docker-compose-test.yaml (2.9 KB)

application.yaml (3.2 KB)

4

Thanks for providing the configuration files! Based on your setup and the additional information, let me help you troubleshoot this issue.

How to Check Zeebe State Directly

To verify if your process instances actually exist in Zeebe (not just in Operate), you can use the Zeebe client API to query directly:

// Using Zeebe Java client
ProcessInstanceFilter filter = new ProcessInstanceFilterImpl();
filter.active(true);
filter.bpmnProcessId("your-process-id"); // replace with your process ID

SearchQueryResponse<ProcessInstance> response = 
    zeebeClient.newProcessInstanceQuery()
        .filter(filter)
        .send()
        .join();

List<ProcessInstance> instances = response.items();

This will show you the real-time state from Zeebe engine, independent of what Operate displays.

Configuration Analysis

Looking at your migration approach, I notice a few potential issues:

1. Profile Configuration

Your profile combination includes process-migration,task-migration,operate,tasklist,broker,consolidated-auth. However, there’s limited documentation about these specific migration profiles in 8.8, and some of these might be causing conflicts.

2. API Version Mismatch

Since you mentioned using:

  • v1 APIs for Tasklist/Operate (which are deprecated in 8.8)
  • v2 APIs for Camunda

The issue might be that your v1 API calls are failing because of the deprecation, while the data migration itself worked correctly.

Recommended Troubleshooting Steps

Step 1: Verify Zeebe State

Create a simple test to query Zeebe directly and compare with what you see in Operate.

Step 2: Switch to v2 APIs

Since you’re already using Camunda user tasks, try switching your task completion calls to the new Camunda 8 REST API (v2):

POST /v2/user-tasks/{userTaskKey}/completion

Instead of the deprecated Operate/Tasklist v1 APIs.

Step 3: Check Migration Profiles

The combination of migration profiles you’re using might be causing issues. Consider:

  • Testing with a simpler profile configuration first
  • Checking if consolidated-auth is conflicting with your authentication setup

Step 4: Verify Authentication

Since you’re using consolidated-auth, ensure your authentication configuration is correct, as 401 errors during migration are common with auth misconfigurations.

Next Steps

Could you try:

  1. Query Zeebe directly using the client API to see if the process instances exist there
  2. Test with v2 APIs instead of v1 for task operations
  3. Share any error logs from your application during startup or when making API calls

This will help us determine if it’s a:

  • Data migration issue (instances don’t exist in Zeebe)
  • API compatibility issue (v1 APIs not working after migration)
  • Configuration issue (auth or profile conflicts)

Would you be able to test these approaches and share the results?

5

The main problem was in volume. Config in github have errors. If u update from 8.7 to 8.8 use
volumes: PathToFolder/zeebe/data:/usr/local/camunda/data

6

Problem Resolved! :white_check_mark:

@Bantexx has identified and solved the issue. The problem was with the Docker volume configuration during the migration from Camunda 8.7 to 8.8.

Solution

When updating from Camunda 8.7 to 8.8, the correct volume mapping should be:

volumes:
  - PathToFolder/zeebe/data:/usr/local/camunda/data

The issue was that the GitHub configuration examples contained errors in the volume paths, which prevented proper data access after the migration.

Key Takeaway

For anyone migrating from 8.7 to 8.8 in self-managed environments:

  • :white_check_mark: Double-check your Docker volume mappings
  • :white_check_mark: Ensure data paths are correctly mapped to the new 8.8 structure
  • :white_check_mark: Verify that existing process data is accessible after the upgrade

This explains why the process instances were visible in Operate (data was migrated in Elasticsearch) but API operations returned NOT_FOUND errors (Zeebe couldn’t access the actual process data due to incorrect volume mapping).

Thanks for sharing the solution - this will help other users facing similar migration issues!