Camunda Consumer Connector

Camunda 8 Topics
1

I am using kafka producer connector in camaunda to produce a message on a topic “test_2” on confluent cloud.

An independent application not related to camunda reads that message from topic “test_2” process the message and put its response on another kafka topic “topic_5”

now i am using camunda kafka consumer connector to read that message from topic “topic_5” but unable to do so.

when i run the process producer works fine–>

  1. i can see message in topic “test_2”
  2. i can also see my application read the message from “test_2” topic processed it and push it on the topic “topic_5”
  3. i can see the final message message on topic “topic_5”
  4. but my camunda kafka consumer connector is not able to read the finalMessage from topic “topic_5”

My kafka credentials are fine. I tested it → if i produce a message using camunda kafka producer in “test_2” topic and read from the same topic using kafka consumer connector on camunda it works fine

Any suggestions please .

kafkaflow
kafkaflow1884×763 62 KB

2

@BerndRuecker request to please help with the suggestion

3

@hulk - is it possible the correlation key is incorrect or missing? If you check for running processes in Operate, do you see additional processes being created when the topic_5 message is written?

4

Hey @nathan.loding

I am able to produce and consume the message successfully through camunda connectors when producer is producing message on topic_a and consumer connector is reading the message from the same topic_a. the it works fine.

but with my below given flow goes it doesn’t work
start camunda process with request parameters—> produce the message on topic_a through camunda producer connector–> an independent application reads the message from topic_a and produce its response topic_b.now on camunda my consumer kafka connector needs to read that message from topic_b.
But it is unable to do so. I am able to see the message on topic_b on confluent.

I even hardcoded the correlation key(PAYLOAD )=“abc” and correlation key(PROCESS)=“abc” still it doesnt work with different topics.

DO camunda intermediary kafka consumer event work on same topics?

ALso related to correlation key

there are two parameters correlationkey(process) i pass userID process variable
in correaltionKey(payload) i pass value.userID →

my understanding is correlationKey(process) and correlationKey(payload) are compared if they are equal the message is consumed.

5

Hey I was able to consume the message now through kafka camunda consumer connector

difference i figured out was
###format of the message value in kafka which i was able to consume
through kafka camunda consumer connector using correlationKey(payload)=value.userID

{userID=BPIO10001BrijeshMishra, requestFrom=PG10001}

##but same message in json format didnt work I was not able to consume using
correlationKey(payload)=value.userID

{
“userID”: “BPIO10001Brij93500”,
“requestFrom”: “PG10001”
}

ANy idea how to fetch correlation when the value of the message in kafka is is json format ?

6

@hulk I was not able to reproduce the issue.

  1. Created a bpmn process with a Kafka consumer connector as an intermediate event to consume messages from topic_0
  2. Started this process with {“corrKey”:“BPIO10001Brij93500”} process variable
  3. Produced message in json format to topic_0
{
"userID": "BPIO10001Brij93500",
"requestFrom": "PG10001"
}
  1. Kafka consumer read the message from topic_0 and correlated it successfully

Attached the bpmn I created for testing.
KAFKA_INBOUND.bpmn (5.5 KB)

You are right about the two correlation keys, they are resolved and compared when the connector reads a message.

Could you please check in Operate if you have the right Correlation key(process) value in your process instance?