I’ve browsed around a bit and found that Infosys was awesome and released open source versions of their DB connectors. In order to get this to run with the free self managed version, is it as simple as extending the docker compose file: https://github.com/camunda/camunda-platform/blob/1ea70db6fe03eb309ed4000d2050e2ce0e65fc1c/docker-compose.yaml by adding another container for one of these DB connectors? I’m a java newbie but i ran mvn clean package and have a jar with dependencies file. Is it as simple as copying the connector bundle section?
Location of the DB connectors: GitHub - Infosys/camunda-connectors: The solution helps users to connect to various Databases & business applications like ERP's from the camunda business
Hi @crench92
Yes we can extend the docker compose file of self managed for running the additional connectors
One Important thing to notice here is that connectors bundle is package of connectors avalable as a docker image in dockerhub
But when it comes to partner connectors we need to create the images on our own before running the compose file
Instead of extending the compose file of self managed you can add the connector runtime container into the network of the self-managed platform so that it can communicate with components
In order to run a connector in docker runtime in container
If you do mvn clean package you will get the jar with dependencies
You need to build an image for the jar using Docker file
FROM camunda/connectors:0.21.3
COPY target/your-jar-with-dependencies.jar /opt/app/
Use this command to build the image in the Dockerfile directory
docker build -t your-connector:1 .
Then If you want to run a single connector in container You can follow this approach
If you want to use the more connectors to run in a single go you can create a docker compose file to run them
services:
your-connector:
image: your-connector:latest
container_name: yourConnector
environment:
- ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS=zeebe:26500
- ZEEBE_CLIENT_SECURITY_PLAINTEXT=true
networks:
- camunda-platform-828_camunda-platform
networks:
camunda-platform-828_camunda-platform:
external: true
You can add some environment configurations in the compose file based on your requirement like below for an outbound connector, For more configurations refer this
- CAMUNDA_CONNECTOR_POLLING_ENABLED=false
- CAMUNDA_CONNECTOR_WEBHOOK_ENABLED=false
- SPRING_MAIN_WEB-APPLICATION-TYPE=none
- OPERATE_CLIENT_ENABLED=false
Hope this helps
Please let me know if you need further help
Regards
Praveen
Thanks for the response Praveen. It looks like I got a docker container running with the java runtime running but what’s odd is that I’m not seeing any errors but nothing is happening either. I’ve attached several images of what I’m seeing, and the docker compose configuration I used.
Hi @crench92
Can you please share the entire logger details of only required connector
Before running the compose file you need to make sure the image of connector which should be built on top of the camunda/connectors:latest should be avialable to the container to run
Instead of giving image name as camunda/connectors:latest give image name that is built for the connector by you in local
Please try this way and provide additional information of connector logs
So i thought all i had to do was provide the jar in the correct location and the connector would start correctly. There’s a log stating MS SQL seemed to start.
tandard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
. ____ _ __ _ _
/\ / ’ __ _ () __ __ _ \ \ \
( ( )__ | '_ | '| | ’ / ` | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
’ || .__|| ||| |_, | / / / /
=========||==============|/=////
:: Spring Boot :: (v3.1.1)
2023-07-25T02:51:38.938Z INFO 1 — [ main] i.c.c.r.app.ConnectorRuntimeApplication : Starting ConnectorRuntimeApplication v0.21.3 using Java 17.0.7 with PID 1 (/opt/app/connector-runtime-application-0.21.3-with-dependencies.jar started by root in /)
2023-07-25T02:51:38.992Z INFO 1 — [ main] i.c.c.r.app.ConnectorRuntimeApplication : No active profile set, falling back to 1 default profile: “default”
2023-07-25T02:51:51.235Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.zeebe.spring.client.annotation.processor.AnnotationProcessorConfiguration’ of type [io.camunda.zeebe.spring.client.annotation.processor.AnnotationProcessorConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:51.272Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.connector.runtime.outbound.OutboundConnectorRuntimeConfiguration’ of type [io.camunda.connector.runtime.outbound.OutboundConnectorRuntimeConfiguration$$SpringCGLIB$$0] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:51.897Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘zeebe.client-io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties’ of type [io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:51.917Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.zeebe.spring.client.configuration.ZeebeClientAllAutoConfiguration’ of type [io.camunda.zeebe.spring.client.configuration.ZeebeClientAllAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:51.933Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.zeebe.spring.client.configuration.ExecutorServiceConfiguration’ of type [io.camunda.zeebe.spring.client.configuration.ExecutorServiceConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.005Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusMetricsExportAutoConfiguration’ of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusMetricsExportAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.027Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘management.prometheus.metrics.export-org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusProperties’ of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.050Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘prometheusConfig’ of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusPropertiesConfigAdapter] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.087Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘collectorRegistry’ of type [io.prometheus.client.CollectorRegistry] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.089Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘org.springframework.boot.actuate.autoconfigure.metrics.MetricsAutoConfiguration’ of type [org.springframework.boot.actuate.autoconfigure.metrics.MetricsAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.092Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘micrometerClock’ of type [io.micrometer.core.instrument.Clock$1] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.385Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘prometheusMeterRegistry’ of type [io.micrometer.prometheus.PrometheusMeterRegistry] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.537Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘zeebeClientThreadPool’ of type [io.camunda.zeebe.spring.client.jobhandling.ZeebeClientExecutorService] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.561Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘commandExceptionHandlingStrategy’ of type [io.camunda.zeebe.spring.client.jobhandling.DefaultCommandExceptionHandlingStrategy] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.564Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.zeebe.spring.client.CamundaAutoConfiguration’ of type [io.camunda.zeebe.spring.client.CamundaAutoConfiguration$$SpringCGLIB$$0] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:52.588Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.connector.runtime.OutboundConnectorsAutoConfiguration’ of type [io.camunda.connector.runtime.OutboundConnectorsAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:54.628Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘objectMapper’ of type [com.fasterxml.jackson.databind.ObjectMapper] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:54.630Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘zeebeJsonMapper’ of type [io.camunda.zeebe.client.impl.ZeebeObjectMapper] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:54.640Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘io.camunda.zeebe.spring.client.configuration.ZeebeActuatorConfiguration’ of type [io.camunda.zeebe.spring.client.configuration.ZeebeActuatorConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:54.892Z INFO 1 — [ main] i.c.z.s.c.a.MicrometerMetricsRecorder : Enabling Micrometer based metrics for spring-zeebe (available via Actuator)
2023-07-25T02:51:54.892Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘micrometerMetricsRecorder’ of type [io.camunda.zeebe.spring.client.actuator.MicrometerMetricsRecorder] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:54.915Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘jobWorkerManager’ of type [io.camunda.zeebe.spring.client.jobhandling.JobWorkerManager] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:55.143Z INFO 1 — [ main] org.camunda.feel.FeelEngine : Engine created. [value-mapper: CompositeValueMapper(List(org.camunda.feel.impl.JavaValueMapper@7ee8130e)), function-provider: io.camunda.connector.runtime.core.feel.FeelConnectorFunctionProvider@4afbb6c2, clock: SystemClock, configuration: Configuration(false)]
2023-07-25T02:51:55.987Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘outboundConnectorFactory’ of type [io.camunda.connector.runtime.core.outbound.DefaultOutboundConnectorFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.032Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘springSecretProviderAggregator’ of type [io.camunda.connector.runtime.core.secret.SecretProviderAggregator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.040Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘outboundConnectorManager’ of type [io.camunda.connector.runtime.outbound.lifecycle.OutboundConnectorManager] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.103Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘annotationProcessor’ of type [io.camunda.connector.runtime.outbound.lifecycle.OutboundConnectorAnnotationProcessor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.123Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘deploymentPostProcessor’ of type [io.camunda.zeebe.spring.client.annotation.processor.ZeebeDeploymentAnnotationProcessor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.147Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘propertyBasedZeebeWorkerValueCustomizer’ of type [io.camunda.zeebe.spring.client.properties.PropertyBasedZeebeWorkerValueCustomizer] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:56.152Z INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘zeebeWorkerPostProcessor’ of type [io.camunda.zeebe.spring.client.annotation.processor.ZeebeWorkerAnnotationProcessor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2023-07-25T02:51:58.739Z INFO 1 — [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2023-07-25T02:51:58.789Z INFO 1 — [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-07-25T02:51:58.809Z INFO 1 — [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.10]
2023-07-25T02:51:59.611Z INFO 1 — [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-07-25T02:51:59.629Z INFO 1 — [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 19928 ms
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2023-07-25T02:52:03.261Z WARN 1 — [ main] c.c.r.c.i.DefaultInboundConnectorFactory : No inbound connectors discovered
2023-07-25T02:52:03.346Z INFO 1 — [ main] z.s.c.c.ZeebeClientProdAutoConfiguration : Creating ZeebeClient using ZeebeClientConfiguration [ZeebeClientConfigurationProperties{broker=Broker{gatewayAddress=‘zeebe:26500’, keepAlive=PT45S}, cloud=io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties$Cloud@255d9277, worker=Worker{maxJobsActive=32, threads=1, defaultName=‘null’, defaultType=‘null’, override={}}, message=Message{timeToLive=PT1H}, security=Security{plaintext=true, overrideAuthority=‘null’, certPath=‘null’}, job=Job{timeout=PT5M, pollInterval=PT0.1S}, interceptors=[], requestTimeout=PT10S}]
2023-07-25T02:52:05.166Z INFO 1 — [ main] org.camunda.feel.FeelEngine : Engine created. [value-mapper: CompositeValueMapper(List(org.camunda.feel.impl.JavaValueMapper@2864f887)), function-provider: io.camunda.connector.runtime.core.feel.FeelConnectorFunctionProvider@4b07cad0, clock: SystemClock, configuration: Configuration(false)]
2023-07-25T02:52:07.725Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:12.753Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:17.757Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:22.760Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:27.769Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:32.791Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:37.798Z WARN 1 — [ main] s.c.c.OperateClientProdAutoConfiguration : An attempt to connect to Operate failed: io.camunda.operate.exception.OperateException: org.apache.hc.client5.http.HttpHostConnectException: Connect to http://operate:8080 [operate/172.23.0.5] failed: Connection refused
2023-07-25T02:52:44.980Z INFO 1 — [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path ‘/actuator’
2023-07-25T02:52:45.281Z INFO 1 — [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ‘’
2023-07-25T02:52:45.606Z INFO 1 — [ main] i.c.z.s.c.jobhandling.JobWorkerManager : . Starting Zeebe worker: ZeebeWorkerValue{type=‘com.infosys.camundaconnectors.db:mssql:1’, name=‘MS SQL’, timeout=null, maxJobsActive=null, requestTimeout=null, pollInterval=null, autoComplete=true, fetchVariables=[databaseConnection, operation, data], enabled=null, methodInfo=null}
2023-07-25T02:52:45.677Z INFO 1 — [ main] i.c.c.r.app.ConnectorRuntimeApplication : Started ConnectorRuntimeApplication in 69.824 seconds (process running for 71.154)
2023-07-25T02:53:05.530Z INFO 1 — [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet ‘dispatcherServlet’
2023-07-25T02:53:05.535Z INFO 1 — [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet ‘dispatcherServlet’
2023-07-25T02:53:05.550Z INFO 1 — [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 3 ms
2023-07-25T03:06:40.205Z INFO 1 — [pool-2-thread-1] i.c.c.r.c.outbound.ConnectorJobHandler : Received job 2251799813686071
2023-07-25T03:11:48.854Z INFO 1 — [pool-2-thread-1] i.c.c.r.c.outbound.ConnectorJobHandler : Received job 2251799813686071
2023-07-25T03:17:44.117Z INFO 1 — [pool-2-thread-1] i.c.c.r.c.outbound.ConnectorJobHandler : Received job 2251799813686774
2023-07-25T03:29:53.629Z INFO 1 — [pool-2-thread-1] i.c.c.r.c.outbound.ConnectorJobHandler : Received job 2251799813687655
2023-07-25T03:35:19.643Z INFO 1 — [pool-2-thread-1] i.c.c.r.c.outbound.ConnectorJobHandler : Received job 2251799813687655
I tried creating an image and i get the same issue where the process just sits there. The connector seems to start with this log:
2023-07-25T13:27:28.362Z INFO 1 — [ main] i.c.z.s.c.jobhandling.JobWorkerManager : . Starting Zeebe worker: ZeebeWorkerValue{type=‘com.infosys.camundaconnectors.db:mssql:1’, name=‘MS SQL’, timeout=null, maxJobsActive=null, requestTimeout=null, pollInterval=null, autoComplete=true, fetchVariables=[databaseConnection, operation, data], enabled=null, methodInfo=null}
Hi @crench92,
Logs look fine but it is unable to Execute the job
Not sure why it’s not executing, @cma your view on this
I have tried extending the connectors setup from another compose file(not in compose file of camunda) by connecting to camunda docker network, It worked fine for me
@Praveen_Kumar_Reddy Looks good to me. I do it almost exactly like you describe, except that I build my own connector runtime which includes connectors through dependencies and then I make an image from that jar.
Check your bpmn file, this is the only thing I can think of, is the task really waiting to be executed by this connector or another one?
1 Like
mssql_diagram_1.bpmn (79.3 KB)
I attached the BPMN file. I’m wondering if there some way to turn on any kind of debugging just to see what’s happening.
Silly question, but…
Can your connectors container actually reach the MSSQL instance?
Does the MSSQL Instance allow connections from the address that the connector will be coming from?
One of your early error messages indicates:
Network error: Software caused connection abort
Naw not silly. This message was from my SSH session to the VM that’s running Camunda docker containers. Has nothing to do with my issue.
How should i upload this file? It looks like the forum removed it due to sharing an IP. But the IP wasn’t a public IP but just a local private IP
Maybe there’s a way to enable debugging so I can dive deeper?
What do the logs from the Connector container say?
The Camunda logs indicate that it dispatched the job to the connector… so there’s possibly an issue on the connector end.
Sometimes forum readers mistake Usernames/Passwords/Addresses as being confidential information. Try blanking out the information and posting again.
What logs should i post other than what i have above?
The logs are only saying the job was dispatched. I tried this: Logging | Camunda Platform 8 Docs but I’m getting that the endpoint doesn’t exist.
My config in the bpmn isn’t much. Just indicating trying to create a new DB on the MS SQL server.
But that’s for the Zeebe side, not the connector itself.
You have to think of a connector as an external worker… it has its own container (maybe logging to stdout and available from docker log)
Though according to the code camunda-connectors/connector-mssql/src/main/java/com/infosys/camundaconnectors/db/mssql/service/CreateDatabaseService.java at 360bebf9745dfb62112a36ae8bac30936f7e43b5 · Infosys/camunda-connectors · GitHub
It’s still using slf4j so you should be able to configure the logging of the connector (within its container)
1 Like