Storing history over a rabbitMq

Camunda 7 Topics Discussion & Questions
1

Hi,

We are looking at offloading the history to another database. One way we thought of doing so was to use a custom HistoryEventHandler to push all the events onto a rabbitMq and then pushed into another camunda instance to store in the standard camunda history tables.

We have successfully pushed the history across a rabbitMq but we are having issues the other side putting the HistoryEvent into the database.

We were hoping to do something like this

  @Override
  public boolean handleMessage(Message message) {
    try {
      Context.getProcessEngineConfiguration()
          .getHistoryEventHandler()
          .handleEvent(
              objectMapper.readValue(message.getPayload().getBytes(), HistoryEvent.class));
      return true;
    } catch (Exception ex) {
      return false;
    }
  }

  @Override
  public boolean handleMessage(Message message) {
    try {
      ProcessEngine processEngine = ProcessEngines.getDefaultProcessEngine();
      ProcessEngineConfigurationImpl processEngineConfiguration = (ProcessEngineConfigurationImpl) processEngine.getProcessEngineConfiguration();
      processEngineConfiguration
          .getHistoryEventHandler()
          .handleEvent(
              objectMapper.readValue(message.getPayload().getBytes(), HistoryEvent.class));
      return true;
    } catch (Exception ex) {
      return false;
    }
  }

But in both cases it seems dependent on the Context which causes issues.

Is there a way of doing this? We thought of this way as it would keep the history in the correct way and keep the ability to use the camunda history api still but separate the history to a completely separate location if needed.

Thanks,

Matt

2

Hi,

So I have gotten further and I seem to be getting the DbEntityManager which wasn’t happening before and no errors are thrown, but the history doesn’t get persisted to the database and I do not no why.

This is what I have done:

Created a custom history event handler to extend the DbHistoryEventHandler

public class MqHistoryStoreEventHandler extends DbHistoryEventHandler {

  private CommandExecutor commandExecutor;

  public MqHistoryStoreEventHandler(CommandExecutor commandExecutor) {
    this.commandExecutor = commandExecutor;
  }

  @Override
  protected DbEntityManager getDbEntityManager() {
    return commandExecutor.execute(CommandContext::getDbEntityManager);
  }
}

Extended the HistoryServiceImpl

public class MqHistoryServiceImpl extends HistoryServiceImpl {

  public MqHistoryStoreEventHandler createMqHistoryEventHandler() {
    return new MqHistoryStoreEventHandler(commandExecutor);
  }
}

Set the custom history service on the config

@Component
@Order(Ordering.DEFAULT_ORDER + 1)
public class HistoryStoreHandlerConfig extends AbstractCamundaConfiguration {

  @Override
  public void postInit(SpringProcessEngineConfiguration config) {
    MqHistoryServiceImpl historyService = new MqHistoryServiceImpl();
    historyService.setCommandExecutor(config.getCommandExecutorTxRequiresNew());
    config.setHistoryService(historyService);
  }
}

and this is how I call into it when I recieve the message from the rabbitMq

  @Autowired
  private HistoryService historyService;

  @Override
  public boolean handleMessage(Message message) {
    try {
      HistoryEvent historyEvent = SerializationUtils.deserialize(message.getRawPayload());

      ((MqHistoryServiceImpl) historyService)
          .createMqHistoryEventHandler()
          .handleEvent(historyEvent);
      return true;
    } catch (Exception ex) {
      return false;
    }
  }

Would anyone know if I need to do anything different, or why it would seem to have no issues, but not actually save it to the database?

Cheers,

Matt

3

So We have finally got this working and thought I’d share the final working solution. When we recieve the message from the rabbitMq we just need to do the following and the history seems to get populated into the history tables correctly.

  @Override
  public boolean handleMessage(Message message) {
    CommandExecutor commandExecutor = processEngineConfiguration.getCommandExecutorTxRequired();

    Command<Object> command = commandContext -> {
      HistoryEvent historyEvent = SerializationUtils.deserialize(message.getRawPayload());
      DbHistoryEventHandler dbHistoryEventHandler = new DbHistoryEventHandler();
      dbHistoryEventHandler.handleEvent(historyEvent);
      return null;
    };

    commandExecutor.execute(command);

    return true;
  }
1 Like
4

Hi Matt,

Thanks a lot for sharing the solution!
Feel free to let us know how this solution works for for you as you continue to use it.

-Niall

5

hi! im trying to save HistoryEvent in pretty same way.But I got a error during operation:
Caused by: org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Unexpected exception while executing database operations with message ’

Error updating database. Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for insertHistor

Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for insertHistor’. Flush summary:

[
INSERT HistoryEvent[5ebe7118-2e94-11ed-8a13-4a34d0042cd5]
]
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationUnexpectedException(EnginePersistenceLogger.java:666) ~[camunda-engine-7.16.0.jar:7.16.0]
… 46 common frames omitted
Caused by: org.apache.ibatis.exceptions.PersistenceException:

Error updating database. Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for insertHistor

Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for insertHistor

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) ~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:199) ~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:184) ~[mybatis-3.5.6.jar:3.5.6]
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeInsertEntity(DbSqlSession.java:359) ~[camunda-engine-7.16.0.jar:7.16.0]
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.insertEntity(DbSqlSession.java:353) ~[camunda-engine-7.16.0.jar:7.16.0]
at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:44) ~[camunda-engine-7.16.0.jar:7.16.0]
at org.camunda.bpm.engine.impl.db.sql.BatchDbSqlSession.executeDbOperations(BatchDbSqlSession.java:62) ~[camunda-engine-7.16.0.jar:7.16.0]
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:341) ~[camunda-engine-7.16.0.jar:7.16.0]
... 45 common frames omitted

Caused by: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for insertHistor
at org.apache.ibatis.session.Configuration$StrictMap.get(Configuration.java:1031) ~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.session.Configuration.getMappedStatement(Configuration.java:821) ~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.session.Configuration.getMappedStatement(Configuration.java:814) ~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:196) ~[mybatis-3.5.6.jar:3.5.6]
… 51 common frames omitted