What causes the log warning “As a consequence this variable is not index by elasticsearch”?

Lars Madsen: Quick question: We get this in the logs from time to time - does it have any particular impact on anything? Any actions we should do?

[Broker-0-Exporter-3] [Broker-0-zb-fs-workers-1] WARN  io.zeebe.broker.exporter.elasticsearch - Variable {key: 6755399443318658, name: "foo", variableScope: 6755399443318638, workflowInstanceKey: 6755399443318638} exceeded max size of 8191 bytes with a size of 16457 bytes. As a consequence this variable is not index by elasticsearch.

Josh Wulf: It means the value of the variable will not be available in Operate or ElasticSearch

Josh Wulf: The variable is too large to be exported

Josh Wulf: You have a single variable with a 16K sized value, and there is a limit of 8K on variable size

Josh Wulf: https://docs.zeebe.io/appendix/broker-config-template.html#standalone-broker-with-embedded-gateway

Josh Wulf: Maybe this setting in the exporter config:

        #     ignoreVariablesAbove: 32677

Josh Wulf: Confirmed:

Josh Wulf: https://github.com/zeebe-io/zeebe/blob/0cd611c9f97acb27e13b956859ad3333c58d5458/exporters/elasticsearch-exporter/src/main/java/io/zeebe/exporter/ElasticsearchClient.java#L103|https://github.com/zeebe-io/zeebe/blob/0cd611c9f97acb27e13b956859ad3333c58d5458/ex[…]porter/src/main/java/io/zeebe/exporter/ElasticsearchClient.java

Josh Wulf: You should do one of:

• Reduce the size of the payload variable - “you need to see it in Operate, but it’s too large for ElasticSearch”
• Increase the exporter variable ignore limit - “you need it persisted and inspectable, and you’ve got room to spare”
• Nothing and ignore the message - “you don’t need that variable value in ElasticSearch or Operate”

Lars Madsen: Excellent -thanks!

Note: This post was generated by Slack Archivist from a conversation in the Zeebe Slack, a source of valuable discussions on Zeebe (get an invite). Someone in the Slack thought this was worth sharing!

If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.