Hi
We are using camunda-platform-8.4.9, using externally hosted elasticsearch to push zeebe records, intially recoreds were updated to elasticsearch but offlate observing below with bulk push, any pointers for this issue would be appreciated
│ 2024-08-09 09:39:30.756 [Broker-2] [zb-fs-workers-0] [Exporter-3] WARN ││ io.camunda.zeebe.broker.exporter.elasticsearch - Error on exporting record with key 6755399441714452 ││ io.camunda.zeebe.exporter.ElasticsearchExporterException: Failed to flush bulk ││ at io.camunda.zeebe.exporter.ElasticsearchClient.exportBulk(ElasticsearchClient.java:166) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.exporter.ElasticsearchClient.flush(ElasticsearchClient.java:111) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.exporter.ElasticsearchExporter.flush(ElasticsearchExporter.java:189) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.exporter.ElasticsearchExporter.export(ElasticsearchExporter.java:118) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.broker.exporter.stream.ExporterContainer.lambda$export$5(ExporterContainer.java:202) ~[zeebe-broker-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.util.jar.ThreadContextUtil.runCheckedWithClassLoader(ThreadContextUtil.java:58) ~[zeebe-util-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.util.jar.ThreadContextUtil.runWithClassLoader(ThreadContextUtil.java:34) ~[zeebe-util-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.broker.exporter.stream.ExporterContainer.export(ExporterContainer.java:201) ~[zeebe-broker-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.broker.exporter.stream.ExporterContainer.exportRecord(ExporterContainer.java:179) ~[zeebe-broker-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.broker.exporter.stream.ExporterDirector$RecordExporter.export(ExporterDirector.java:570) ~[zeebe-broker-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.retry.BackOffRetryStrategy.run(BackOffRetryStrategy.java:51) ~[zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:96) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:47) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:122) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:130) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:108) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:227) [zeebe-scheduler-8.4.9.jar:8.4.9] ││ Caused by: org.elasticsearch.client.ResponseException: method [POST], host [https://hostname], URI [/_bulk], status line [HTTP/1.1 403 Forbidden] ││ ││ 403 Forbidden ││ ││
403 Forbidden
││nginx ││ ││ ││ ││ at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:347) ~[elasticsearch-rest-client-8.9.2.jar:8.9.2] ││ at org.elasticsearch.client.RestClient.performRequest(RestClient.java:313) ~[elasticsearch-rest-client-8.9.2.jar:8.9.2] ││ at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288) ~[elasticsearch-rest-client-8.9.2.jar:8.9.2] ││ at io.camunda.zeebe.exporter.ElasticsearchClient.sendRequest(ElasticsearchClient.java:250) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ at io.camunda.zeebe.exporter.ElasticsearchClient.exportBulk(ElasticsearchClient.java:164) ~[zeebe-elasticsearch-exporter-8.4.9.jar:8.4.9] ││ … 16 more