Form Builder (Drag and Drop) + Form Server Validations

[StephenOTT]

So this is some details about a project we are working on to provide Camunda with a Form Builder.

This has been raised by others in the past: Redirecting to Google Groups.

The purpose is to solve two problems:

1. Ability to easily create the equivalent of Camunda Embedded Forms using a “Drag and Drop Form builder”, but also allow those same forms to be built using pure JSON.

2. Provide a single language (javascript) to create Client Side and Server Side validations (so the server can validate any client side work without any extra effort on your part).

To solve these problems we have been working with Form.io · GitHub, more specifically:
Form Builder: GitHub - formio/ngFormBuilder: The Form.IO Form Builder Application
Form Renderer: GitHub - formio/ngFormio: JSON powered form rendering library for AngularJS + Form.io.
Form Server: GitHub - formio/formio: A Form and Data Management Platform for Progressive Web Applications.

Overall goals:

1. Do not build something that places Camunda Form Building in a “island”. We want the tools used to be based on a large user base, standards, and easily extendable. You should be able to technically take the same solution and use it in other non-Camunda based apps. Why? Because we help to ensure more community support and easier extendability rather than being stuck on the ‘island’.

2. Nothing overly complicated, but can still increase in complexity because of its ability to extend.

3. As a starting point allow the Form to integrate with Camunda’s current Embedded Forms / CamForm SDK JS. This means leveraging the camunda html attributes and some relatively small adjustments to Tasklist.

4. Allow Server Side Validations with the same system that provide client side validations

5. (Eventually) allow Form draft Saving/persistence of data to DB.

So what does this look like?

Well Formio is a Angular based Form builder and Renderer but more so it is a standard to follow for building forms (JSON), and you can rebuild the renderer in any lang (such as Formio’s React renderer).
We allow the user to build and store a angular generated form using a drag and drop form builder, have the json of that form structure stored independently of the BPMN and camunda server, and use the Form server for server side validations (bonus: the server is Node so we can write client side and server side validations in the same language, and often in one place).

This setup will require some small modifications to the Tasklist to support a new type of FormKey format, and a extension to the Camunda server to proxy data to formio (nodeJS server), and only accept data into Camunda once the formio server side validations are accepted.
Requests for the Form Structure and the Form Structure JSON response that is passed between Tasklist and Formio would also be proxied by Camunda Server.

From modeler perspective the key would be something like ‘formio:FormSystemName’. This way forms could be used across deployments in a very reusable way, and you could technically update forms without having to redeploy. (of course if we wanted the aspects of having to redeploy when a form changes, this could be easily supported as well).

I see this formio scenario sort of a mix between Embedded Forms and External Forms. In Formio form config we would still add the Camunda attributes to fields so when the JSON is loaded through the Formio Directive in Tasklist, Tasklist will still be able to pull or push variables.

Submission to Camunda Server will still be the same as the Camunda attributes, as they will be maintained as part of camFrom API/SDK.

The Form builder is highly themeable so it could easily be added as a new page into Admin, Cockpit, or Tasklist.

Here is how we have been thinking about the data flow (very draft):

Would love feedback thoughts, use cases, issues, etc

[rpeinl]

Hi Stephen,
I really love that project. It looks like exactly what I’m currently planning for the next year, but seems you are much quicker. I was planning to use Orbeon Form Builder, but I don’t see any advantages of it compared to your proposal. I saw that form.io does also support more advanced input elements like a date picker and a select field with the possibility to bind to a URL. That’s awesome.
I’m totally in and can imagine to support the project also with code contributions.
Keep up the good work
René

[StephenOTT]

@rpeinl

I saw that form.io does also support more advanced input elements like a date picker and a select field with the possibility to bind to a URL

With the formio “engine” you can create any field (called components) you wish. They are just HTML that is configured and templates through with JSON. So if a field is missing, there would not be extensive effort to add the functionality. It is just JS, Node, and HTML. Very likely the code already exists in some example and you only need to do some ninja copy and paste.

We are working on our schedule at the moment. But if anyone has interest and wants to have the timelines let me know, and can see what we can do.

[StephenOTT]

Great prototype implementation with Java:

Dockerized example with Form.io form builder and Camunda

Hi! I made an example as proof of concept trying to contribute Camunda BPM with Form.io form builder. I wanted to share it: Demo is here: http://unctad.redfunction.ee/ Source code: Bitbucket On this example you can: draw your BPMN diagram; define DMN and inject into BPMN rule task; model any user form or wizard and inject it into BPMN user taks test it All microservices and applications are dockerized. If you want to run example on your computer, first install Docker, download source co…

[Greg_Svitak]

Thanks for the post. We would interested in helping contibute to your efforts.

Just curious… how do you think Orbeon compairs to Form.io. It seems that Orbeon has been around a little longer and wondering if it has some additional features.

[StephenOTT]

@Greg_Svitak Hey
We originally did not go with orbeon because it was a “beast of a software” to use and manage.

formio was providing a fairly lightweight, new generation of standards (node, json, JS, Angular, React, etc), and if you needed features that are paid, the pricing model was just way to high. (It also has a nice fit with Camunda’s Angular based Tasklist).

Also many of the paid features of orbeon are just standard OSS / expected features in formio: embedding, SQL, web hooks, services and actions, Internationalization, Signatures, Permissions, etc.

Overall orbeon “felt” like a much more closed system that places you in a smaller ecosystem. With formio we have the entire Node, Angular, and React communities to pull from.

Our OSS release as a symfony bundle should be out in coming days.
There is also this example that was posted above: Dockerized example with Form.io form builder and Camunda.

Edit: In Terms of contributions I had imagined that there was a few different levels depending on your use cases:

1. someone building a web app to work on the Camunda server, and building a modification to the Tasklist so this functionality is supported out of the box (this whole task is on our todo as a R&D gov paid project).

2. Build out other middleware implementations: would be used depending on the application you are using.

3. Build additional formio components that are useful to Camunda space, but can also be used by rest of formio community.

[Greg_Svitak]

@StephenOTT Thank you very much for your opinion and the great work.

We will be starting work on this type of project soon. Please feel free to drop me a DM or an email if you would like some help.

One question…based on your profile, it looks like you work for a SAAS offering. Do you need to pay for the form.io service or could you run their code base in production just by cloning.

Thanks,
Greg

[StephenOTT]

@Greg_Svitak

One question…based on your profile, it looks like you work for a SAAS offering. Do you need to pay for the form.io service or could you run their code base in production just by cloning.

We use the Open source versions of Formio: https://github.com/formio. We try as much as possible to contribute back to that community because it benefits us the more the community grows and builds more robust form components.

[StephenOTT]

Realized i did not update this:

So if you would like a base to build off of:

check out:

GitHub - DigitalState/Platform: The DigitalState Platform

and look at the dev folder.

If you deploy: Authentication, Identities, Services, Formio, and Camunda you get the previously discussed deployment, along with JWT security, field level security, Services directory, etc, etc. Everything is microservices based, with the UIs running with Angular (ng2-Admin)
All self deploying with the the scripts in the dev folder (windows and linux/unix versions)

(note that when i took those screenshots there is a layout bug on the two columns layout of first name and last name)

Enjoy!

[josecristhian]

Hi @StephenOTT , i do not know much about docker, I tried to build it but I was not able to do it.

Can you explain a little bit how did you integrate camunda and form.io? How do you call a form from within a task or init form?

Thks in advance.

BR,
Jose

[StephenOTT]

@joechromo in the specifics of our build, Start Forms and Task Forms are activated through the Portal Angular App. This is a sort of partial replacement for Camunda Tasklist. All features of tasklist are not replaced (and thats not the goal). The current build releases “Services”. A service has a BPM Definition ID. The None Start Event of that process definition has a form key such as formio:myFormName. When you Activate the Service, the Service looks up the BPM Definition and gets the form key, and then gets the Form schema from Formio, and the formio.js renders the form based on the schema.

On submit of the form, the data is sent to Formio for validation, and if valid, the data is submitted into Camunda as a JSON process variable along with other process variables that are sort of “metadata” about the submission.

In a very soon to be OSS release we have “Data Resolvers” which are pre-population on the server side of Formio form fields based on data in Camunda and data in other Microservices. Data Resolvers work with Camunda Tasks, where the Task Form Key is used to get the Formio Form.

[StephenOTT]

@josecristhian did you follow: https://github.com/DigitalState/Sdk/tree/master/dev ?

[josecristhian]

Thks @StephenOTT , I was not able to build the docker using the instructions in the GitHub, always I get an error but for sure is my zero experience doing that.

What do you mean by Service? How do you use it?

So you built and app and formio does not work inside the tasklist cockipt, right? Did you use any camunda library in your app or just web services?

Thks in advance.

[StephenOTT]

@josecristhian

What do you mean by Service? How do you use it?

So you built and app and formio does not work inside the tasklist cockipt, right? Did you use any camunda library in your app or just web services?

A “Service” is a business word (in our context) for government service / a “service” offered by an organization to a individual, another organization, or anonymous person. A Service is something like “Pay your Taxes”, “Report a Pot Hole”, “Make a Complaint”, “Request a refund”.
Services have “Service Scenarios”: a scenario typically the channels that a service is available on, but is also can be used for any sort of variation in the service. Example:

→ Service: Pay your Taxes:
—> Scenario: Online
—> Scenario: In-Person
—> Scenario: By Phone
—> Scenario: By Letter Mail

There are Scenario Types: BPM, Information, and External. Different Types(Scenario Strategies) have different fields and requirements. BPM executes Camunda/Formio. Information is only a html field to display static or dynamic widgets/information. And External is similar to information, but includes a “link” field which is a URL to the service (think of a directory that links to another website).

---

There is nothing that was built in tasklist. Tasklist is untouched in the deployment.

We are using the Camunda REST API, but we rebuilt the PHP SDK in more modern frameworks compared to the current camunda php sdk. We did the same for the Formio Rest API.
Camunda / any BPM engine we connect to with the app is treated as “another microservice” / generic system that we push data into. In the OSS stuff on Github it is not released as prod ready. You need to add layers of security in places (such as securing camunda).

You can see the symfony components for BPM, BPMCamunda, and Formio here:
https://github.com/DigitalState/Services/tree/develop/src/Ds/Component

[StephenOTT]

Wanted to share a update on this that might be useful for some:

We recently ported over our data resolvers and tasks.
This is a partial replacement (with some additional features) for Tasklist. Its not designed to be a formal replacement for tasklist. More of a specific use case replacement:

See the following images. There is a Portal and Admin app. Portal is (client side)(citizens in our context), and Admin is administration (Staffers in our context).

All tasks that are submitted get logged into Camunda as well as into a “Submissions” entity incase Camunda is offline (Example you do a offline form submission and your form was cached), or you want to process the data to multiple locations or other camunda instances.

client side is send the form schema to render, and we also support Default values that are pulled from the user account, any other microservice or Camunda:

ds[bpm].task.variable.none_start_event_form_data.firstname

Portal loading tasks:

All fields support field level security. All security is handled at our microservice level with Multiple identities, business units/groups etc with ability to set custom actions
Everything is configurable.

All runs on Angular (latest)

Tasks can be viewed and completed in portal and admin UIs

is fully multilingual, where the languages are loaded from the CMS (GitHub - DigitalState/Cms: The DigitalState Cms Microservice)

examples:

Oh you can also support fully read-only fields, partial read only (read only on client side), etc. Basically all the fancy enterprise features.

Lots of other fun features tucked away. But last one i will mention: When you build a Form and attach it to the Task (or start event), it doubles as a fully server side validation for the form.

So you can do something like:

{{tasks}}{{environment}}/tasks/a210161b-9ae9-11e7-92b2-0242ac110002/submission

and have a body of:

{
	"data": {
        "firstName": "Bob"
    }
}

and it will validate it against the formio server before submission into camunda into the task.

Tasks are submited into camunda as JSON variables with the name of the variable being the task_form_data_{{activityId}}.

Once Local Variables are supported on Camunda’s API for task completions, we will convert to using that.

Can find examples at DigitalState · GitHub, postman API SDK at: GitHub - DigitalState/Platform: The DigitalState Platform.
MIT License.

Enjoy!

[user334]

@StephenOTT Please look at this i think you can help me :

How can Use **Form Builder** (drag and drop) replacement embedd forms?

Basically i want to use Form Builder drag and drop to give user more powerful to build form . it’s possible add Form Builder in Cammunda Modeler ? and when user start task fill the form as embedd form not “Open external form” ?

[Stefan_Houtzager]

I am using ngFormbuilder at the moment with camunda’s BPMN and DMN modelers. But I am looking to switch to https://github.com/formio/formio.js which is written in plain js. I needed to customize ngFormbuilder, which is doable but not so easy not knowing angular 1. Moreover I do not want to depend on a js framework. Btw the backend (WIP) of my workflow apparatus is built in elixir, using the phoenix framework. Workflows are running via websockets (the needed customization in ngFormbuilder is easy), form json’s are sent from the server when a next usertask is needed, forminput goes back via the same socket. Validations in the json can be executed serverside by, for js validations, calling node from elixir. Workflow state is persisted in one actor (lightweight proces/green thread; trademark of erlang since the eighties - the language under elixir) per running workflow (in key-value pairs) and can be persisted in a db when needed.

[rSalois]

Any plans to let such forms be exported to PDF? No idea how to realize that, just didn’t see such a thing with json before… I more used to use this one https://online-form-builder.pdffiller.com/ yet there’s no way it would work well with Camunda in the same smooth fashion. I wonder if it would be experiencing the same

[StephenOTT]

Formio has a PDF generator for its form. Submitting a filled PDF would be the process to upload the PDF and process the fields back into a submission

[Abhishek_Manocha_Sof]

@StephenOTT I did follow up the steps in dev env. Running the dev env through docker.

With one difference being, I used release/0.16.0 branch instead of tags/0.8.0. Thought the documentation is old.

My question is - I can login to http://api.formio.dev.ds but not any of the other like http://portal.dev.ds/ gives me 404 page not found.

Something amiss?
What is the full URI?
How can I find for others, like admin.dev.ds and others api.camunda.dev.ds/camunda