We are evaluating the usage of a free self-managed Camunda8 installation.
In order to provide visibility on running processes, and since we cannot use operate and tasklist without a license we came across the Zeebe Simple Monitor project, which fits our needs.
However we were thinking of the following approach.
Instead of using the custom hazelcast exporter, we were planning to use the out of the box elastic search exporter, and modify simple monitor web app so that the services querying the postgres database to query elasticseach.
Would such an approach be feasible?
In general creating a custom web app to query zeebe-record-variable indices is a viable solution for production usage?
Tagging @Philipp_Ossler , as one of the creators of the simple monitor app
It depends. The ES indices from Zeebe’s ES exporter contain the raw data. For access from an application like Zeebe Simple Monitor, it could be more efficient to store the data in a custom database schema, like the database of Zeebe Simple Monitor. This follows the CQRS pattern with materialized views.
Please note that Operate and Tasklist are free for non-production usage.
Thanks a lot for your feedback. Some additional questions regarding Hazelcast Exporter:
Suppose we have a Hazelcast cluster with a single node to which zeebe records are transmitted by the broker. However due to some issue, the apps consuming the records from the ring buffer (e.g. Simple Monitor) are not up and running. If a restart is made to the single hazelcast node, will these events be lost or are they persisted somehow until read by the consumer?
In case of high load is it feasible to activate many consumers consuming messages from the hazelcast instance (and have them read different records)? Is this approach feasible using a ring buffer? If not we are afraid that in case producers are much quicker than consumers, old entries might be overwritten
By default, the Hazelcast ring-buffer is not persisted.
To increase reliability, some users use an external Hazelcast cluster with more than one node.
The Hazelcast ring-buffer is a structure with a fixed capacity. By default, old entries will be overridden if the capacity is reached.
You can increase the capacity of the ring-buffer to reduce the likelihood.
Please keep in mind that the Hazelcast exporter was not implemented with production usage requirements (i.e. high load, reliability). You may be able to use and configure it for your use case. And this is great. But you may run into some limitations.
If you have ideas for improvements, feel free to contribute to the project.
Can you please let me know why we need to keep separate DB schema after extracting Zeebe data from Elastic. Instead, we can create separate indices related to monitor in Elastic itself
