How do I avoid sending massive payloads in workflow variables?

Op: @Josh Wulf I’ve refactored my code to stop storing huge variables and storing just the ID now instead…
the input to the workflow is still this huge payload.
which has to go as a ‘variable’…
wanted to know if there is any other way I can pass an input other than as a variable, from where the worker could pick the input? (edited)

Josh Wulf: A URL, and when the worker gets the URL, the server returns JSON

Op: but this JSON is something that the UI/client forms and sends…
the above approach which you mentioned would mean doing some work (storing it somewhere and make it accessible via URL) on the server before calling the workflow

this is exactly what I was thinking of doing, but wanted to know if this can be avoided

Josh Wulf: Where does the UI/client send it?

Op: to a node service that calls the workflows…

Josh Wulf: You could either use Redis or a similar store, or else just write it to a JSON file on the server.
and write an endpoint to kick off the workflow and store the data on disk
How do you collect the outcomes?

Op: redis, pubsub
nodejs services creates a pattern using workflowname-workflowinstance id for the topic name and subscribes to it… and down the line (whereever i would want the response) i have a publish outcome box which publishes to the above pattern

Josh:You could put it in Redis then
and have the workers retrieve it from there using the key that gets passed as the dataKey variable, or whatever

Note: This post was generated by Slack Archivist from a conversation in the Zeebe Slack, a source of valuable discussions on Zeebe (get an invite). Someone in the Slack thought this was worth sharing!

If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.

1 Like