The Role of the Workflow Engine

First of all big up for your tremendous initiative on pushing microservices orchestration using a process engine. A perfect fit.

After reading documentation of the latest version carefully, I’m wondering about an essential gap regarding the role of the workflow engine communicated in the blog and the latest documentation.

In The Microservices Workflow Automation Cheat Sheet: The Role of the Workflow Engine Bernd is mentioning the third architecture alternative Work distribution by workflow engine, which puts the engine in the middle of the architecture as central piece where:

Microservices can subscribe to certain work of a workflow and get tasks via some kind of queue.

So from my understanding no additional communication channels (beside the workflow engine) are required in that ideal form of architecture, where you have one workflow spanning different microservices (implemented as workers / message publishers in different applications)

I’m wondering why this kind of architecture is not highlighted or even mentioned explicitly in the documentation / best practices section. Instead the best practices implies the ‘one workflow over multiple applications’-approach only as exceptional:

Typical applications will include multiple pieces of glue code in one codebase. […] As discussed in writing good workers, you typically will bundle all workers within one process solution, but there are exceptions where it makes sense to have single workers as separate application.

What are these exceptions from your perspective (only scaling requirements), and why shouldn’t that be the preferred design to think of first when getting a PoC started?

In a greenfield project the Work distribution by workflow engine alternative seems the most intuitive architecture to me, because it allows to avoid plenty of glue code and additional communication channels (also on network level) to get different microservices integrated throughout a workflow. By avoiding REST endpoints / Kafka integration etc. in every application it keeps the overall architecture of the core worflow small and simple. Of course developers have to take care about the service task / messaging semantics of the workflow engine, of versioning when a process evolves over time, and also to not put too much data into the process context. But I would expect these points as exceptions for thinking of other communication channels on top to the workflow centric architecture, but not the other way around. Is that an too much exotic approach?

I also don’t understand why the major con in the blog (Workflow engine becomes a central piece of the architecture and needs to be operated appropriately;) does not apply to the other architectures as well. Workflows won’t be executed in any architecture, unless the engine is operated properly.