How would you match message pairs in BPMN and in Camunda 8?
-
Msgs arrive independently and are different except for 2-3 key attributes.
-
Large size of each msg : 50 to 1000+ json properties
-
High volume : ~ 100K + per day
-
Match happy path : most matches will come within a few minutes (e.g. first of pair arrives and waits a min or two before second of pair arrives), once matched - one of the msgs continues, the other has status updated to “Matched”
-
Match timeouts : when exceeded for some time (e.g. 2 hours) must throw an error - not currently considering if second comes after first times out waiting
In searching have come across these sources:
Given the biggish data scenario, the variable size limit suggests we should be checking / registering msgs via a job integrating with an external db , rather than pooling the msgs within a flow.
Help appreciated, thanks