Deleting workflows

Camunda 8 Topics Discussion & Questions
1

Hi,

We have a use case, where we would be creating series of steps dynamically, which needs to be executed in flow,

we are planning to use zeebe for this use case, where we would deploy workflows dynamically based steps provided, once this sequnence is done, we are planning to terminate,

we would like to know that, would zeebe be the right option for this, when data grows,
we are expecting the step might grow over million in an year, we would need to clean up the workflows, post execution of workflow

Thanks,
Prasanna

2

Hi @Prasanna and welcome,

Currently undeploying (or deleting) a process is not yet possible. There’s an open issue to add support for this here.

However, Zeebe cleans up what it no longer needs when it comes to process instances. So if a process instance has completed or terminated, Zeebe will clean up its data associated to it. As long as process instances are continuously completed or terminated, Zeebe can execute millions of process instances.

Would it be possible to model your dynamic workflow in a single process? For example, using exclusive gateways to choose whether or not to execute a task (step). In addition, sub processes and multi-instance come to mind to deal with large numbers of steps (i.e. some variable list to iterate over). Also have a look at our FEEL expression support to evaluate expressions on the available data.

Please let me know if this helps.

Best,

Nico

1 Like
3

Thanks @korthout, we are going through these docs, Also we have one more use case, where we would like to have exponential backoff retry strategy, as the upstream system that we rely on has higher latency, we would like to retry with a exponential delay

Thanks,
Prasanna

4

Hi @Prasanna,

In your follow-up question it is not clear to me what you would like to retry. I assume that you mean retrying of jobs.

So a job worker has activated a job, is processing it and needs to fail the job because the upstream system is unresponsive. Failing the job can be done with the FailJob RPC, which immediately makes the job available again to job workers. A retry backoff for failed jobs is not yet available, but proposed here. Another proposal exists here which is about timed-out jobs instead of explicitly failed jobs.

A workaround is available by modeling the retry loop, using a FEEL expression as the duration of the timer boundary event and using output mappings to initially set and increase the exponential multiplier.

Something like:

Screen Shot 2021-06-18 at 11.12.16
Screen Shot 2021-06-18 at 11.12.161276×764 21.3 KB

Hope that helps!

Nico

2 Likes
5

Thanks @korthout , this helps us. as you suggested, we could come up with a workflow using exclusive gateway and sub process, And we will be using multi Instances as well

We are evaluating the dependencies for Zeebe,
We understood that the zeebe brokers doesn’t rely on rdbms, it maintains the stream of as append only log.

We have few queries on monitoring Zeebe
Do we need zeebe operate for production?
Does zeebe operate supports integration with kafka exporter?
Do we any other zeebe broker state, and workflow instances?

Thanks,
Prasanna

6

Hi @Prasanna,

Yes, Zeebe uses streams (append only logs; each partition is 1 stream; technically zeebe uses a multi-RAFT implementation forked from atomix) and builds a view of the current state in RocksDB (an embedded database commonly used in streaming applications).

Do we need zeebe operate for production?

For technical monitoring, Zeebe provides metrics for Prometheus and Grafana dashboards (docs). For business operations we recommend using Operate. Please reach out to our sales team here if you are interested in using Operate.

Does zeebe operate supports integration with kafka exporter?

Zeebe exports its streams using exporters (e.g. elastic-search exporter). Operate consumes elastic search and cannot be connected to kafka. However, you can use one of the exporters maintained by the community (awesome exporters), or build your own custom exporter. A kafka exporter maintained by the community is available here.

Do we any other zeebe broker state, and workflow instances?

I’m unsure what you’re asking here, please explain your question in more detail.

And as always: Hope this helps :slight_smile:

Nico

PS: To help other users find answers to their questions, please open a new topic for questions you have that are unrelated to the original question (deleting workflows/processes).

4 Likes
7