Custom Tasklist Implementation: Complex Sorting by Process Variables, Search, and Performance for 3000+ Tasks - Recommended Architecture?

Camunda 8 Topics Discussion & Questions
1

Hi Camunda Team and Community,
I’m building a custom tasklist frontend for a headless Camunda 8 Self-Managed deployment (version 8.8) using Node.js with the camunda-8-js-sdk. Before implementing, I’d like to get official recommendations on the best architecture approach for my requirements.

I’ve researched extensively on the forums and found valuable threads like:

These threads helped me understand Camunda 8’s eventually consistent architecture, but I still have specific questions about the recommended approach for my use case.

Requirements:

  1. Complex Sorting by Process Variables:
    We need to display tasks sorted by business process instances variables (business priority), NOT just by creation time.

  2. Variables in Task List Response:
    For each task, we need to display some process instances variables and we need these variables returned with the task search results (ideally in a single API call).

  3. Universal Search:
    A single search input field that searches across multiple process variables.

4. Multiple Task Screens:
Each user task needs its own screen:

  • Created tasks - Unassigned tasks for claiming
  • Created + Assigned - Tasks assigned to the current user
    For 5 user tasks = 10 different task list views.

5. Scale and Performance:

  • 3000+ active tasks at any time
  • 30+ concurrent users
  • Lightning-fast response times (sub-second)
  • Proper pagination support

From the forum threads and documentation, we understand:

  1. Eventually consistent architecture: Zeebe exports to Elasticsearch, and Tasklist/APIs query Elasticsearch. There’s
    inherent delay (2-10 seconds).

  2. Tasklist API v1 limitations:

    • taskVariables filter only supports eq operator (no LIKE/pattern matching)
    • includeVariables works but only for truncated/small variables
    • Cannot sort by process variables
    • Sorting only supported for built-in fields (creationTime, dueDate, etc.)

  3. Orchestration Cluster API v2: Does not yet support includeVariables in search response.

  4. Community solutions: Several users have implemented their own database layer (per threads linked above).

Is there an official recommended architecture for custom tasklists that need:

  • Sorting by process variables
  • Full-text search across variables
  • Sub-second response times for 3000+ user tasks?

What is the Camunda-recommended architecture for this use case?

Thank you for your time and guidance!

2

For your requirements with 3000+ tasks, complex sorting by process variables, and sub-second response times, the recommended architecture is the event-driven local database pattern rather than relying solely on Tasklist APIs. I found the following relevant resources:

Does this help? If not, can anyone from the community jump in? :waving_hand:

:light_bulb: Hints: Use the Ask AI feature in Camunda’s documentation to chat with AI and get fast help. Report bugs and features in Camuda’s GitHub issue tracker. Trust the process. :robot:

3

Hello,

If I understand correctly, your challenge can be broken down into three main issues. I’ll list them below along with possible solutions or workarounds for each:

1. Retrieving process instance variables together with the task list
This feature is not yet available in the Camunda 8.8 API v2 (only in Tasklist API v1).
As a workaround, you can continue using API v2 to retrieve user tasks filtered by process instance variables as described in the documentation here..
For retrieving variables, you can do it in two requests:

  • First, get the paginated/filtered/sorted list of user tasks.

  • Then, from that list, extract the instance keys and send them to API v2 to retrieve the process instance variables (by the names of your business variables), passing the list of instance keys as input to this API (documentation here).

2. Sorting by process instance variables while retrieving the task list
This is not possible with either API v1 or v2.
Camunda does offer the possibility to set a priority on a user task (a number between 0 and 100), which you can then sort by. This could serve as a workaround for your business priority sorting needs. (documentation here)

3. Performance and near real-time retrieval of newly created user tasks
There will always be an export delay before a user task appears in the API.
If you paginate properly when retrieving user tasks via API v2, response times should be fine. However, if your concern is the time from task creation to its availability in the API, you’ll need to create your own storage system through an event-driven architecture.
Specifically, use User Task Listeners to listen for events on user tasks, then store them in your own database where you can filter, sort, and paginate as needed. This approach is highly performant because you avoid waiting for the export delay, you simply retrieve tasks directly from your database (documentation here).

Recommendation:
If the above workarounds don’t fully solve your problem, I strongly recommend adopting an event-driven architecture with User Task Listeners and a custom database layer.

Let me know if this helps or if you still need clarification!

1 Like