We noticed that writing to the (large) history db takes up to 50% of execution time to our Camunda application.
So I came up with the idea of using springs TransactionalEventListener to asynchronously postpone writing to the history schema after the runtime transaction commits.
So I replace the default DbHistoryEventHandler with a handler that publishes the (wrapped) HistoryEvent to the spring eventBus and calls the DBHistoryEventHandler “manually” after the transaction commits.
Debugging shows that this is actually happening … but the entries to the act:_hi_actinst for example are not written, neither do I get any error messages …
I guess that this is somehow caused by the internal, thread-local, Context/Command handling. I tried to bridge this by initializing the commandContext manually via
@Bean
public DbHistoryEventHandler dbHistoryEventHandler(@Lazy ProcessEngineConfigurationImpl configuration) {
var setContext = new Command<Void>() {
@Override
public Void execute(CommandContext commandContext) {
if (Context.getCommandContext() == null) {
Context.setCommandContext(commandContext);
}
return null;
}
};
return new DbHistoryEventHandler() {
@Override
protected DbEntityManager getDbEntityManager() {
configuration.getCommandExecutorTxRequired().execute(setContext);
Assert.notNull(Context.getCommandContext(), "context is null");
return super.getDbEntityManager();
}
};
}
but it is not working as expected.
I set up a minimal working example, so if anyone feels like helping out, please head over to my demo repo, launch the spring boot application and start the dummy process. After checking the hi tables in the /h2-console, I would expect the events to show up … but they don’t. Thank you!
What’s your other 50% of your execution time made up of? The history writes as essentially double multiple writes that duplicate the runtime tables. So makes sense that it would be ~50%. Would also check when the db ca
When you say not working as expected, what is the unexpected behaviour? The history handler expects to have a command context with a dB connection. Is your async still transactional aware? If the history even fails, how are you dealing with rollback of transaction?
Are you sending each event as a different transaction? Or are you keeping the batchings of events grouped by transaction?
Assert.notNull(Context.getCommandContext(), "context is null"); is failing but does not bubble, so the command context is not being passed into your next context.
If you register a commandContextListener, I can see that the command context (and therefore the DB session?) has closed before reaching the assert statement.
The dbHistoryEventHandler(@Lazy ProcessEngineConfigurationImpl configuration) bean seems to be a empty configuration without any services or configs.
During the execution of the async handler you can see the db entity cache store the entities, but they are likely not being flushed/committed into DB at end of Session. If DbSqlSession.class Session is not created and closed during the context then unsure what would push the data into the db (hence why the db is empty?)
Is your intention to reuse the existing command context in your async handler? Or are you expecting to create a new thread? If you are expecting to create a new thread, then I would assume you cannot reuse the same CC as you do here:
/
If you create a new thread, then i would assume you are part of a new Command Context(CC) and the previous CC was closed and the DB cache would have been committed and then flushed. If you implement this async behaviour, how would you expect rollbacks to occur if your historic data failed to commit?’
Few other considerations:
What does your typical BPMN look like? How many activities are being executed in a single sync execution? If you have many individual actions occurring this will generate a lot of ACTINST records and would add up in execution time. Example from your BPMN in your project above:
