Consuming Messages from Queue

Hello Fellas,

Need your help in understanding if Camunda has a in-built OOTB connector which can consume SWIFT messages and create case ?

1 Like

Hi @aqaadi,

Could you please clarify what are SWIFT messages? Camunda provides a http-connector and a soap-http-connector 1.

Cheers,
Roman

Here is a link - I had some experience. They can be complex. Since I wasn’t aware of an off-the-shelf (inside Camunda) adapter, I didn’t reply.

But, depending on the application server, was thinking this was more a feature of built-in (app’server) integration adapter - not something Camunda has off-the-shelf.

Appreciate your response Gary.

We are definitely using Tomcat8.

Could please share your experience on these lines ?

From my experience, the SWIFT standard is somewhat big and complex. And, a message transporting a SWIFT payload can be huge. Because of its size and complexity, it’s a good idea to create a specialized adapter responsible for ingesting the payload and then provide discrete services per specific process/task requirements.

So, my experience… the team initially tried to manage the entire SWIFT message as one big type/object. It was a huge object regardless of the specific elements needed by their BPM process and associated task implementations. This lead to a serious problem in computing resource requirements and problematic configuration management. And, because the SWIFT adapter, ORM, and instance all shared space alongside the BPM engine - we ended up with a serious performance bottleneck and inability to both scale and manage change/revisions.

The team also didn’t plan for potential change requirements as they effected process-dependent information. For example, a slight change on a SOAP interface broke the system. If the interface designer simply provided extended version and field requirements per operation - well, that would have spared us some major issues. Some brief notes:

Isolate system boundaries:
Only use the standard where it’s required. Only use what’s needed.

example - what we needed: (imagine this as: Java API, JAX-WS/RS)
SWIFT.getCustomer(interface compatibility or version #, field requirements, customer #)

we instead got:
SWIFT.getTheWholeDamnObjectAndTheKitcheSink(customer #)
This lead to serious issues in transformation and IO overhead. In fact, transformation grew into a project anti-pattern. And the entire system stack spiked when marshalling these phat objects. And, for example, some developers internalized the schema up-and-into the WEB-UI! This caused trouble… with the management of changes by locking the entire data-definition into a single schema (though it was good to have a centralized data-dictionary… the reward wasn’t realized - at a cost of serious project delays).

We needed a specialized host tuned for managing SWIFT messages:
Ended up off-loading and mocking up the SWIFT adapter rather than waiting for the integration team to factor/re-factor their services.

Set-up feature/performance expectations for the SWIFT adapter:
Using the mock/skeleton adapter, we then moved forward with performance benchmarks - goal was to isolate adapter computing and performance costs. This lead to an SLA with the then separated SWIFT message provider.

Added robust message verification/validation - and this can be switched to varying levels of enforcement and reporting:
Obviously, we needed to know what was going on with inbound schema and data requirements.

NOTE: I’m just listing these quickly - hope this helps.

Also, though Tomcat provides a decent platform… why not use Wildfly? (the community supported version of JBoss). You’re giving up CDI, etc…

1 Like

These points are too good for our start.
Thanks alot Gary !!

No harm in using wildfly. The problem being the client mandates Tomcat :frowning:

Tomcat constraint - workarounds/options?

Messaging and AWS services aren’t necessary saying “no” to Tomcat.

I was thinking:

  • Apache Camel running as a plain-old Java application. Though I’ve always relied on Wildfly/Fuse add-on, I think you can run this as its own application.
  • Amazon (AWS) hosting your messaging and light-weight processing needs via Lambda.
  • ESB - Though this is pushing the constraint given that some ESB services are, or were, just specialized application containers.

When I build event-based, messaging systems… I typically bridge the inbound/outbound messages between local JMS implementations (ActiveMQ, Rabbit, etc) and enterprise/cloud-base sysetms (AWS, MQ, etc). Using a sort-of “store-and-forward” architecture. Provides local horizontal robustness and availability while keeping hooked into external communication channels.

Do you feel Kafka, AWS , Tomcat, Camunda a good fit.

Acknowledging your Tomcat constraint… Because, this necessarily drives your services into alternate territories. And, even though you can set-up a pool of Tomcat instances, clustering these servers is honestly pushing the envelope - meaning that advanced infrastructure (scaling, availability, etc) similarly demands advanced application and hosting services.

In an ideal world I’d recommend:

- Camunda: clear winner for BPM/Case/Rules
- Wildfly v10+: Community supported with current technology and vast documentation/howto articles. And, it’s open-source.
- Wildfly add-ons: If you’re going to use Wildfly then you really must add-on Apache-Camel for the sake of its EIA libraries. Also, very well documented with loads of examples.
- NGINX: Load balancing and my personal choice for this open-source combination.
- Kafka, Artemis, ActiveMQ, Messaging/Event (etc): Based on requirements… Kafka is excellent. But, not sure you’re planning to use its advanced features. If you’re looking at basic messaging, then I’d lean towards Artemis/ActiveMQ (noting that Wildfly has Artemis built-in).
- AWS for cloud services: Noting that cloud-service will tend to drive up on-going hosting fees. But, it’s a balance trading off internal/externally supported hosting requirements. AWS is excellent. But, I’d hold-off on this decision until it becomes necessary - while keeping AWS infrastructure requirements/patterns in mind to better support the migration when the timing is right.

Now, restricting the platform to Tomcat:
- Camunda: Same as above
-Tomcat: This is our constraint (no choice)
- NGINX: Same as above. Fail-over/load balancing might be a little slower - but, the difference could be a non-issue (based on needs).
- AWS: Provides some workarounds and options for service infrastructure technologies. Before taking the leap into AWS… recommend a quick POC. Though AWS is excellent - it has its pitfalls and annoyances. I just wrapped up an AWS-hosted project and was somewhat disappointed with the overhead/workarounds for infrastructure support I’d previously taken for granted on ESXi (VMWARE) hypervisor. AWS SDKs/integration are quickly evolving - this means you’ll see a lot of API updates. An example is their jump to Jackson JSON support - generally a good thing. But, could have provided some options. So, develop locally and migrate to AWS rather than building in early hard-dependencies.
Kafka: Ok… I’m honestly a little jealous. I’ve heard really good things about Kafka but was required to use Artemis/ActiveMQ due to built-in support and product maturity. You’ll need to ramp up on: Streams, advanced JDK 8 features (functional patterns), and event-driven architecture (rxJava, etc). And this is damn fun technology - but, plan for a little extra ramp-up. Your dev’ team will likely love the new technology - keeping morale very high throughout the project.

Thanks Gary.

Very much appreciate your thoughts. Will keep you posted on what will be the outcome.

Hi aqaadi - can you share your experience with camunda - Kafka integration ? I am trying to use that for one of the enterprise applications would like to hear from the community on the success stories or failures …