Hello,
Normally camunda provides its default feature to store an activity in DB by selecting aync after/before check but this will increase a high DB write during production heavy load. In case of any failure happens on the node, if we do not put proper save point, camunda starts the flow again from the start activity.
To avoid the DB activity, we thought of using process variable to store the last activity id and during the node restart while teh flow is coming up we are checking the same process variable value, assuming this process variable will be persistent in DB. camunda history level is FULL in my set up, but we have seen that this value is not getting persistent and i am not able to get the last stored activity detail after restart and it is starting from the start activity again.
Is there any other way to store the last acitivity id which will be remain persistent even after node restart ?attached the BPMN test flow for reference.
save_point_test.bpmn (8.6 KB)
Logs after restart of the node
Value of the process variable :: null
First Script with order id :: 90915
First Script with order id :: 90915
First Script with order id :: 90915
First Script with order id :: 90915
First Script with order id :: 90915
Script 2 activity id :: Activity_13vbm54
Second Script Current time: 2024-04-04 19:02:00
Second Script with order id :: 90915
Second Script Current time: 2024-04-04 19:02:05
Second Script with order id :: 90915
Third Script Current time: 2024-04-04 19:02:10
Third Script :: 90915
Third Script Current time: 2024-04-04 19:02:15
Third Script :: 90915
Third Script Current time: 2024-04-04 19:02:20
Third Script :: 90915
Hi @bipra.banerjee and welcome back.
If you do not provide a save point via asyncBefore/asyncAfter and the process reaches no other “natural” wait state (e.g. user task or timer), then no data will be written to the database. This also includes the process variables.
In the background there is a technical transaction, that is kept running as long as no wait state (or save point) is reached. No data will be committed to the database prior to the commit of the transaction.
Actually I’m also wondering about your high database load. I have seen setups with a lot of process instances using FULL history level and when the database is correctly scaled, it still performs pretty good.
Are you using a lot of process variables or do you have a lot of activities? That would of course increase the number of SQL statements needed.
1 Like
Thanks @rohwerj for the response.
The scope of the flow is not simple since the flow interacts with 12 different southbound applications for one order under different products with specific logics. So, we have to put save point on specific calls such as interface calls and order DB state updates. we did not use many variables. The ask of TPS is high.
so for each southbound application calls we have some set of activities which must be done hence, number of activities (mostly service task) is on the higher side. Issue is, during failover to different site, processes getting started again from start if not proper save point is being set. Not sure why only external service task is considered as waiting state, which I think camunda can update to consider any service task as wait state.
So we were looking for an option on one process variable which will hold the last activity id over the process lifetime and can be used till completion.
Regards,
Bipra