Extracting data from Zeebe-brokers for custom Tasklist

As requested per Using GraphQL-API for custom tasklist - #5 by jonathan.lukas moved to a separate topic:

@jonathan.lukas Okay, got it. Then just one follow-up question for querying the data:

  1. The Elasticsearch-exporter puts the data to ElasticSearch per default. Should we then go ahead and kind of “intercept” the data while its transported to Elasctic and read it from the exporter or should we in this case query the data directly from Elastic after the exporter has put it there?

  2. And is it then better/easier/more practical to either query the data from Elastic after the Elastic-exporter has put it there, or to use the Hazelcast-exporter and query the data directly from the exporter? Is there a preference?

Hi @the-villain

You could also have a look at this:

This is a custom cockpit (Operate) application that is based on data from the Zeebe brokers being exported to Hazelcast. Maybe you could use the same approach for a task list application.

BR
Michael

3 Likes

Hello @the-villain ,

thank you for opening a new topic on this.

The mechanism is meant like this:

The Elasticsearch holds all data exported by Zeebe, first of all „pure“. This means that the exporter does the exporting regardless of the target system, creating a „history“.

Then, other systems can use this export, for example to create own indexes or writing data to another system.

I would suggest that also the Hazelcast way would be the same.

Jonathan

Hi @jonathan.lukas @mimaom, thanks very much for the information so far, that all clarified a few things for us :slightly_smiling_face:

1 Like

Should we then go ahead and kind of “intercept” the data while its transported to Elasctic and read it from the exporter or should we in this case query the data directly from Elastic after the exporter has put it there?

And is it then better/easier/more practical to either query the data from Elastic after the Elastic-exporter has put it there, or to use the Hazelcast-exporter and query the data directly from the exporter.

custom bottled water