Cannot register custom exporter

Hey guys, I was just playing around with zeebe exporters, I’m trying to register it in zeebe and somehow I’m getting this weirds error. Here is what I done:

1. Implemented a simple zeebe exporter
Right now for testing purposes it’s very simple

public class DemoExporter implements Exporter {
    Controller controller;

    public void open(Controller controller) {
        this.controller = controller;
    }

    public void export(Record record) {
        System.out.println(record.toJson());
        this.controller.updateLastExportedRecordPosition(record.getPosition());
    }
}

2. Started C8 locally
Run docker-compose up -d in this repo → GitHub - camunda/camunda-platform: Links to Camunda Platform 8 resources, releases, and local development config

3. Register exporter in zeebe

a) Moving my target jar zeebe-exporter-demo-1.0-SNAPSHOT.jar from host to zeebe docker conainer in /usr/local/zeebe/lib
b) Editing application yaml (adding exporter)

zeebe:
  broker:
    exporters:
      demoexporter:
        className: io.zeebe.DemoExporter
        jarPath: lib/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar

4. Restarting zeebe container to load new configuration

And on startup this is the error that I’m getting:

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2023-04-23 15:40:58.134 [] [main] ERROR
      org.springframework.boot.SpringApplication - Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:772) [spring-boot-3.0.5.jar:3.0.5]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:753) [spring-boot-3.0.5.jar:3.0.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:317) [spring-boot-3.0.5.jar:3.0.5]
        at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:82) [camunda-zeebe-8.2.3.jar:8.2.3]
Caused by: java.lang.IllegalStateException: Failed to load exporter with configuration: ExporterCfg{, jarPath='/usr/local/zeebe/lib/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar', className='io.zeebe.DemoExporter', args=null}
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:149) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:70) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:49) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:91) ~[camunda-zeebe-8.2.3.jar:8.2.3]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:769) ~[spring-boot-3.0.5.jar:3.0.5]
        ... 3 more
Caused by: io.camunda.zeebe.broker.exporter.repo.ExporterLoadException: Cannot load exporter [demoexporter]: cannot load specified class
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:82) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:147) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:70) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:49) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:91) ~[camunda-zeebe-8.2.3.jar:8.2.3]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:769) ~[spring-boot-3.0.5.jar:3.0.5]
        ... 3 more
Caused by: java.lang.ClassNotFoundException: io.zeebe.DemoExporter
        at java.net.URLClassLoader.findClass(Unknown Source) ~[?:?]
        at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?]
        at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?]
        at io.camunda.zeebe.util.jar.ExternalJarClassLoader.loadClass(ExternalJarClassLoader.java:57) ~[zeebe-util-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:79) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:147) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:70) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.Broker.<init>(Broker.java:49) ~[zeebe-broker-8.2.3.jar:8.2.3]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:91) ~[camunda-zeebe-8.2.3.jar:8.2.3]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:769) ~[spring-boot-3.0.5.jar:3.0.5]
        ... 3 more
Apr 23, 2023 3:40:58 PM org.apache.coyote.AbstractProtocol pause

What am I missing?

1 Like

Also, I think the documentation is slightly out of date. It says that in order to register a custom zeebe exporter one must add some configuration to zeebe.cfg.toml which I think is not true, it looks like registering exporters in zeebe docker container should happen in application.yaml file.

Which documentation are you working from?

Is it this Blog post? That’s almost 4 years old, and written for Zeebe 0.17.0 … Many many versions ago.

You might want to compare your code to this community exporter, as it will likely contain the hints needed…

Hello @GotnOGuts ,

that’s true, I already noticed that, hence I’m pointing out that the documentation might need a little brush up. It looks like I’m following in my code exactly what is done in kafka exporter or elasticsearch exporter, though I’m still facing this errors when trying to register it in application.yaml.

Still did not resolve this issue. Thoughts?

Blog posts aren’t really documentation, and aren’t expected to be kept up to date…

What does your package line in your .java source file say?

@GotnOGuts

My package:

package io.zeebe.exporters;

import io.zeebe.exporter.api.Exporter;
import io.zeebe.exporter.api.context.Context;
import io.zeebe.exporter.api.context.Controller;
import io.zeebe.protocol.record.Record;

public class DemoExporter implements Exporter {
    Controller controller;
...

my application.yaml file config:

zeebe:
  broker:
    exporters:
      demoexporter:
        className: io.zeebe.exporters.DemoExporter
        jarPath: lib/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar

yet I still get the error shared in the original post

How are you accomplishing this?

In the Git Repos that I linked to, they are just mounting it as a volume, based on the Zeebe image.

And can you try it with

jarPath: exporter/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar

and mounting the jar into the image with

    volumes:
      - ./zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar:/usr/local/zeebe/exporter/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar

Yeah, mounting it as a volume is one option, I got two projects that I’m currently debugging in my local env and this is what I do in one of them.

In the other one I move the jar manually by running

docker cp zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar zeebe_container_id:/usr/local/zeebe/lib/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar

I think mounting the jar is not an issue. I even did mount it exactly the same way as it is mounted in kafka-exporter project. I still get the same error.

I see my exporter being registered at the very beginning

"exporters" : {
    "kafka" : {
      "jarPath" : "/usr/local/zeebe/lib/zeebe-kafka-exporter.jar",
      "className" : "io.zeebe.exporters.kafka.KafkaExporter",
      "args" : {
        "producer" : {
          "servers" : "kafka:9092",
          "requestTimeoutMs" : 5000,
          "closeTimeoutMs" : 5000,
          "clientId" : "zeebe",
          "config" : "linger.ms=5\nbuffer.memory=8388608\nbatch.size=32768\nmax.block.ms=5000\n"
        },
        "maxBatchSize" : 100,
        "maxBlockingTimeoutMs" : 1000,
        "flushIntervalMs" : 1000,
        "records" : {
          "defaults" : {
            "type" : "event",
            "topic" : "zeebe"
          },
          "deployment" : {
            "topic" : "zeebe-deployment"
          },
          "deploymentDistribution" : {
            "topic" : "zeebe-deployment-distribution"
          },
          "error" : {
            "topic" : "zeebe-error"
          },
          "incident" : {
            "topic" : "zeebe-incident"
          },
          "jobBatch" : {
            "topic" : "zeebe-job-batch"
          },
          "job" : {
            "topic" : "zeebe-job"
          },
          "message" : {
            "topic" : "zeebe-message"
          },
          "messageSubscription" : {
            "topic" : "zeebe-message-subscription"
          },
          "messageStartEventSubscription" : {
            "topic" : "zeebe-message-subscription-start-event"
          },
          "process" : {
            "topic" : "zeebe-process"
          },
          "processEvent" : {
            "topic" : "zeebe-process-event"
          },
          "processInstance" : {
            "topic" : "zeebe-process-instance"
          },
          "processInstanceResult" : {
            "topic" : "zeebe-process-instance-result"
          },
          "processMessageSubscription" : {
            "topic" : "zeebe-process-message-subscription"
          },
          "timer" : {
            "topic" : "zeebe-timer"
          },
          "variable" : {
            "topic" : "zeebe-variable"
          }
        }
      },
      "external" : true
    },
    "demoexporter" : {
      "jarPath" : "/usr/local/zeebe/lib/demo-exporter.jar",
      "className" : "io.zeebe.exporters.DemoExporter",
      "args" : null,
      "external" : true
    }
  },

Yet somehow once zeebe starts up the issue persists

io.camunda.zeebe.broker.system - Bootstrap Broker-0 partitions [1/1]: partition 1 failed with unexpected exception.
java.lang.IllegalStateException: Failed to load exporter with configuration: ExporterCfg{, jarPath='/usr/local/zeebe/lib/demo-exporter.jar', className='io.zeebe.exporters.DemoExporter', args=null}
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:437) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.lambda$partitionsStep$21(Broker.java:408) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:63) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:61) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:46) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.partitionsStep(Broker.java:423) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.lambda$initStart$10(Broker.java:234) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:63) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:61) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:46) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:180) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:21) [zeebe-util-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.start(Broker.java:160) [zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:60) [camunda-cloud-zeebe-1.0.0.jar:1.0.0]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:819) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:803) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:346) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1340) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329) [spring-boot-2.4.5.jar:2.4.5]
        at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:47) [camunda-cloud-zeebe-1.0.0.jar:1.0.0]
Caused by: io.camunda.zeebe.broker.exporter.repo.ExporterLoadException: Cannot load exporter [demoexporter]: cannot load specified class
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:81) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:435) ~[zeebe-broker-1.0.0.jar:1.0.0]
        ... 23 more
Caused by: java.lang.ClassCastException: class io.zeebe.exporters.DemoExporter
        at java.lang.Class.asSubclass(Unknown Source) ~[?:?]
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:79) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:435) ~[zeebe-broker-1.0.0.jar:1.0.0]
        ... 23 more
2023-04-24 23:09:10.019 [] [main] INFO 
      io.camunda.zeebe.broker.system - Closing Broker-0 partitions succeeded. Closed 0 steps in 0 ms.
2023-04-24 23:09:10.019 [] [main] ERROR
      io.camunda.zeebe.broker.system - Bootstrap Broker-0 [12/14]: zeebe partitions failed with unexpected exception.
java.lang.IllegalStateException: Failed to load exporter with configuration: ExporterCfg{, jarPath='/usr/local/zeebe/lib/demo-exporter.jar', className='io.zeebe.exporters.DemoExporter', args=null}
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:437) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.lambda$partitionsStep$21(Broker.java:408) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:63) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:61) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:46) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.partitionsStep(Broker.java:423) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.lambda$initStart$10(Broker.java:234) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.lambda$startStepByStep$2(StartProcess.java:63) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.startStepByStep(StartProcess.java:61) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.takeDuration(StartProcess.java:92) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.bootstrap.StartProcess.start(StartProcess.java:46) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:180) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:21) [zeebe-util-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.start(Broker.java:160) [zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:60) [camunda-cloud-zeebe-1.0.0.jar:1.0.0]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:819) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:803) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:346) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1340) [spring-boot-2.4.5.jar:2.4.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329) [spring-boot-2.4.5.jar:2.4.5]
        at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:47) [camunda-cloud-zeebe-1.0.0.jar:1.0.0]
Caused by: io.camunda.zeebe.broker.exporter.repo.ExporterLoadException: Cannot load exporter [demoexporter]: cannot load specified class
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:81) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:435) ~[zeebe-broker-1.0.0.jar:1.0.0]
        ... 23 more
Caused by: java.lang.ClassCastException: class io.zeebe.exporters.DemoExporter
        at java.lang.Class.asSubclass(Unknown Source) ~[?:?]
        at io.camunda.zeebe.broker.exporter.repo.ExporterRepository.load(ExporterRepository.java:79) ~[zeebe-broker-1.0.0.jar:1.0.0]
        at io.camunda.zeebe.broker.Broker.buildExporterRepository(Broker.java:435) ~[zeebe-broker-1.0.0.jar:1.0.0]
        ... 23 more

It’s driving me crazy at this point :smiley:

But there’s one significant difference that I only just noticed since the Kafka one loaded in your last test, but DemoExporter didn’t.
Could you try:

  1. Update your package name
package io.zeebe.exporters.Demo;
  1. Recompile your jar
  2. Update your application.yaml
      demoexporter:
        className: io.zeebe.exporters.Demo.DemoExporter

Still the same error. @GotnOGuts

Caused by: java.lang.IllegalStateException: Failed to load exporter with configuration: ExporterCfg{, jarPath='/usr/local/zeebe/lib/zeebe-exporter-demo-1.0-SNAPSHOT-jar-with-dependencies.jar', className='io.zeebe.exporters.Demo.DemoExporter', args={logLevel=debug, prettyPrint=false}}

I found a solution but I’m not going to lie, I’ve wasted a few days, it really might be a good idea to get some good and reliable docs around it …

So the entire time I was using the library that was reffered in the blog post:

       <dependency>
                    <groupId>io.zeebe</groupId>
                    <artifactId>zeebe-exporter-api</artifactId>
                    <version>0.26.6</version>
       </dependency>

while I was supposed to be using

        <dependency>
            <groupId>io.camunda</groupId>
            <artifactId>zeebe-exporter-api</artifactId>
            <version>8.2.3</version>
            <scope>provided</scope>
        </dependency>
1 Like

Not disagreeing, but you’re talking one community member to another.
Since I still don’t know which documentation you’re referring to, I can’t really point you in any direction.

However, most of the documentation is Markdown on Git… So if you think the documentation can be improved, file a pull request with the updates.

1 Like

@GotnOGuts That’s true, you’re right.

I guess mostly these blog posts has led me to this error:

After a while it was difficult to spot such a delicate differeence between <groupId>io.camunda</groupId> and <groupId>io.zeebe</groupId>.

Hi Michael I am trying to export data using custom exporters to postgres need clarity where to add the jar file configuration of zeebee exporter as I am trying to implement using springboot can you please help me regarding this

Hello @Manish_peddurwarr ,

the jar file of your zeebe exporter has to be deployed to zeebe. If you are running zeebe in a docker container, then you can simply copy the jar to zeebe using docker cp command.

The JAR artifact has to be deployed to /usr/local/zeebe/lib/.

After that you need to register your zeebe exporter by modifying application.yaml in /usr/local/zeebe/config/application.yaml.

For example add this:

zeebe:
  broker:
    exporters:
      debuglog:
        className: io.camunda.zeebe.broker.exporter.debug.DebugLogExporter
        args:
          logLevel: debug
          prettyPrint: false
      your_exporter:
        className: io.camunda.zeebe.exporters.your_exporter
        jarPath: /usr/local/zeebe/lib/your_exporter.jar
        args:
          logLevel: debug
          prettyPrint: false

after that just restart zeebe container. It should already contain your jar and approriate application.yaml configuration should force zeebe to detect the jar (observer application logs on zeebe startup to see that your exporter was identified by zeebe).

1 Like

Hi Michal, I want to do it using zeebee broker I am running zeebee broker,then I have added exporters code in springboot application ,now do I need to add the jar artifact of springboot application in yaml file I am not using docker container