Open source RPA playground for Camunda

Community Community Extensions
1

With @Noordsestern we’ve noticed RPA (Robotic Process Automation) being mentioned often at Camunda events lately. Mostly in the context of exclusive commercial vendors. Strictly limited to communicating with legacy systems.

We have both been using open source Robot Framework for RPA for years with success *). Without need to worry about license limitations, we might have been able to use RPA for wider range of use-cases than just for legacy systems.

Because isn’t any proper RPA solution eventually indistinguishable from a generic “remote code execution service” or just a FaaS (Function as a Service)? Once you have a such system in place, any problem starts looking like an “RPA problem”. Maybe not all glue code is worth of their own dedicated microservice after all?

For the last few weeks, we have been trying to figure out, what could be an approachable enough solution for trying out Robot Framework RPA with Camunda. Here’s the latest try:

Robot Framework RPA playground is Docker Compose -environment, with pre-configured Camunda with a deployed example process, Jupyter Lab IDE for editing and manual execution of the bots, all connected with containerized Firefox for browser automation with noVNC-provided visual.

The example process is about using one bot to do a search for web content and then required amount of additional bots for fetching the results for manual review. Task logs and results are reported back to Camunda, of course.

rpa-playground-worker-03
rpa-playground-worker-031364×766 56.7 KB

rpa-playground-camunda-04
rpa-playground-camunda-041364×766 106 KB

For a start, it would be fun to know, if anyone else can actually get that running. Seeing is believing, hopefully. Next, if someone gets interested, we’d be happy to point to more examples and resources. There are plenty of materials available and also a commercial cloud provider with their own IDE and bot scheduling service.

Also, it is not a coincident that the playground uses Jupyter Lab as its IDE. It happens to be that Jupyter is also the de-facto standard working environment for data science and machine learning…

*) Disclaimer. From the two of us, only Markus is already orchestrating RPA bots with Camunda, including scheduling them using Kubernetes. I’m still relying on other systems triggering jobs with Jenkins, and only designing & implementing Camunda treatment for our future bots.

8 Likes
2

Thank you, @datakurre ! I have no idea, how you created this composition. This is is hot! :fire: :fire: :fire:

I installed docker-compose and followed your instructions in the repository. Since I only have 1 screen i had to switch to selenium-grid-tab very quick otherwise I would not see anything live running, because the web-ui-automation is so damn fast. :+1:

This is a great all-in-one demonstration. :star:

3

This is super cool!
I’m gonna give it a try right now! :sweat_smile:
Thanks for sharing it.

1 Like
4

Another disclaimer. I have never used the popular commercial RPA tools, so I have no idea how convenient and fun their development tools are to use. That said, I am quite happy what we achieved with Jupyter Lab for Robot Framework RPA a few of years ago:

(And I don’t mean only that convenient browser automation auto-completion element picker, which sometimes works better sometimes worse, but the overall fast feedback loop with Jupyter Notebook and Jupyter Lab, that encourage explorative approach, discovery and learning by doing.)

3 Likes
5

Hi @datakurre
I tried your playground :grinning:.

Works perfect! Only small issue I had:

  • you used your user name to clone. Correct is: git clone https://gitlab.com/atsoukka/robot-rpa-playground.git

Thanks for this excellent work!

6

Thanks. I fixed the clone instructions in README. True that the prior one required some GitLab account configured for git.

Ideas for future improvements and examples are welcome. I should be even using that myself for internal training in early June.

Note that the playground does not include Markus’ Robot Framework CamundaLibrary as such, but a smaller set of similar keywords, because the approach is a bit different (to be able to submit robot log files back to Camunda). But that should just underline, how little code is required to integrate Robot to anything.

1 Like
7

Do you also have also an External Task Worker - that runs this RPA Code automatically?
If not that would be maybe an idea for a next step in integrating with Camunda :grinning:.

8

It depends.

I have a concept with a PoC for our “hashistack” based on-premise infrastrucure: My dream open source RPA orchestrator with Camunda and Nomad

But the best solution really depends on what you already have. For example, for a low volume RPA, a GitLab CI with bot per repository would work. Jenkins is also usable with proper extensions. Robocorp cloud is a commercial option. Markus is building one for their Kubernetes cluster…

And also the execution model varies. Markus is running bots with fixed schedules and one bot execution may execute multiple tasks from Camunda, but there may also be “empty runs”. My concept has worker, which at first locks task from Camunda, then schedules bot at Nomad and the keeps polling Nomad and keeps the task locked as long as bot is running (or has completed the task). Markus’ approach is better for a big amoutn of small tasks. My use case is relatively low volume, but some tasks may take up to minutes to complete.

1 Like
9

Thanks for the explanation, I am a beginner - and I thought all that is missing now would be a simple external task worker that starts the RPA jobs. So I think I have to do some reading :laughing:.

10

Again. It depends. If your use case is simple, the worker could be simple too :slight_smile: Maybe I should think about adding the simplest possible NodeJS based external task worker there as an example.

Well, need to think about that. In its current shape, that “playground” is suitable for trying out Robot Framework with Camunda and even for limited bot development. For me, adding a simple example worker there makes sense only, if it is suitable for some real use case.

For bigger picture, we do have also an open source standalone desktop installer for Robot Framework Jupyter Lab IDE https://robots-from-jupyter.github.io/ and it have had tools also for (Windows) desktop automation. There has not been new release for that for a while, but I’m getting help from a nice Jupyter developer to make a new release for us. Once our release pipeline works again, adding some Camunda support next to the other learning materials there could be possible.

To get an idea about, try out this interactive Robot Framework tutorial based on that: Binder

11

I had a chance to play a day with AWS APIs, so the playground has now a branch with a new example that uses AWS Textract to extract form fields from a submitted form-like document (or just image of a document).

aws-textract
aws-textract1918×1077 252 KB

This might be a bit more complex to run than the original example, because this requires saving suitable AWS credentials for robot and I have only covered my own needs for ways to submit those credentials. Also the results may vary after the documents you submit to the robot.

This demo use Robocorp’s RPA Framework library for Robot Framework to parse AWS Textract response into an easy to access model.

I love the simplicity of the required Robot syntax to read the extracted form fields from that model:

*** Tasks ***

AWS.Textract.Form
    Init textract client  region=eu-central-1
    ${result}=  Get external task variable  result
    ${model}=  Convert textract response to model  ${result}
    ${form}=  Create dictionary
    FOR  ${page}  IN  @{model.pages}
      FOR  ${field}  IN  @{page.form.fields}
        IF  ${field.value}
          Set to dictionary  ${form}  ${field.key.text}  ${field.value.text}
        END
      END
    END
    Set external task variable  form  ${form}  type=JSON
3 Likes
13

We have now a yet another take on open source RPA playground for Camunda. This time with Zeebe:

https://datakurre.github.io/automation-playground/

The virtual desktop is available for Vagrant as described at:

https://datakurre.github.io/automation-playground/vagrant.html

2 Likes