Camunda transaction flushing / querying from cache

Camunda 7 Topics Discussion & Questions
1

Hi all,

I’m having issues with the DbEntityManager from Camunda. It does not get flushed or queries do not fetch the data from the cache so it seems. I’ve tried multiple things (moving create code to a new EJB with REQUIRES_NEW transaction management) but I cannot get things to work.

I have 2 user tasks.
On the first task (NewNotification), I have a complete task listener with the following code:

User camundaUser = identityService.newUser(identifierGenerationService.getNewUserId());
        camundaUser.setFirstName(selectedUser.getFirstName());
        camundaUser.setLastName(selectedUser.getLastName());
        camundaUser.setEmail(selectedUser.getEmail());
        camundaUser.setPassword(newPassword);

identityService.saveUser(camundaUser);
identityService.createMembership(camundaUser.getId(), EXTERNAL_GROUP_ID);

LOGGER.info("created a new Camunda user: {}", camundaUser.getId());

On the second task, I have a create task listener with the following code

String camUserId = variableMap.getValue(VariableConstants.CAMUNDA_USER_ID, String.class);
User camundaUser = identityService.createUserQuery()
                .userId(camUserId)
                .singleResult();

LOGGER.info("Selecting a user with camUserId {}: {}", camUserId, camundaUser);

Server Logging;

…pop.listener.NewNotification] (default task-33) created a new Camunda user: 1177@poprocess
…pop.listener.ConfirmNotification] (default task-33) Selecting a user with camUserId 1177@poprocess: null

Is this a bug in Camunda 7.4?

2

Hi @nvanbelle,

That is a limitation in how the entity cache works and when the engine actually performs database operations.

The entity cache is only flushed at the very end of a command and only at that time are UPDATEs and INSERTs executed. The idea behind this is to reduce the time that the process engine keeps write locks to a minimum. In addition, the entity cache is not able to resolve range queries, i.e. any XXService.createYYQuery directly hits the database. The entity cache only filters the response, i.e. it replaces any value returned by the query if there is a copy of it already in the cache.

In consequence, your user query does not fetch the user you created before if both listeners are executed within the same command. Now for this specific query where you fetch by id and expect a single result, it would be easily possible to use the cache. In fact, the cache is used internally whenever an entity is selected by its ID.

However, I think this problem can hardly be generally resolved without dropping the cache entirely. With the cache in place, solving it would require to evaluate a query on the cache and integrate it with the SQL query. The big challenge here would be making this transparent to the user. For example, I can imagine this causing problems with pagination and sorting. Any query that requires a join would be fun as well, etc.

TLDR: Depending on the viewpoint you can call this a bug. I rather call it a limitation due to the engine’s architecture and that we can hardly fix it.

Cheers,
Thorben

1 Like
3

You could try the following code to create the user and trigger a flush within the first listener:

ProcessEngineConfigurationImpl processEngineConfigurationImpl = Context.getProcessEngineConfiguration();
processEngineConfigurationImpl.getCommandExecutorTxRequiresNew().execute(new Command<Void>() {

  @Override
  public Void execute(CommandContext commandContext) {
    // create user and membership here

    return null;
  }

});

The CommandContext manages the entity cache. By default, calling an API command is going to use a command executor that participates in the current context if there is already one. The command executor accessible via getCommandExecutorTxRequiresNew always creates a new context. When that command returns, it is going to close the command context and in turn flush the entity cache. Be aware that the current transaction holds write locks from then on, which, in the general case, may lead to deadlocks with parallel transactions (probably not here, though).

Of course that is not public API, but you could try that to validate the idea.

Cheers,
Thorben

2 Likes
4

Hi Thorben,

Thank you for elaborating and posting some example code to bypass this issue. The solution as you proposed is doing an excellent job. These sort of ‘hacks’ are giving me a better view on the overall workings of Camunda and it’s possibilities, even when they should be used with great care.

Kind regards,
Nico

5