Automated Camunda Deployments - CI for process files - Shared Engine

Camunda 7 Topics Discussion & Questions
1

I put together a working example of Process Deployment using Jenkins for a sort of CI for Process Files.

Use Case: When you are not using Java WAR deployments, and you deploy directly to Camunda using the Rest API, and you still want to have the benefits and structure of CI.

Looking to get feedback from the community on the structure and style. Anything that was missed or ideas that would make this better.

For this first iteration I put this together using Jenkins. See the following Jenkins File: https://github.com/StephenOTT/ProcessProjectTemplate/blob/master/Jenkinsfile.

You can deploy jenkins locally with: docker run -p 8081:8080 -p 50000:50000 jenkins

See the following for notes about how the deployment process works: https://github.com/StephenOTT/ProcessProjectTemplate/blob/master/docs/deployment.md

TL;DR: the Jenkins file reviews the following file inside of your SCM Repository such as Github:

You set your Camunda /deployment/create arguments in the deployment object, and any files you are uploading as part of the deployment in the deployment.files object.

You can set this to work with Master, Branches, Releases, etc.

The Repo: GitHub - StephenOTT/ProcessProjectTemplate: Template project for Process (BPMN, CMMN, DMN) / Camunda Projects is a working example you can connect your jenkins system to. In this repo there are some other conventions that are documented. These are purely for ease of use/repeatability for large number of “projects”. Feel free to provide feedback.

Some top level thoughts that I would be interested to hear peoples thoughts on:

  1. How can Tests be added? I am looking at jUnit + Cucumber at the moment. In the ProcessProjectTemplate model there are not java delegates or other java code you upload/deploy. Everything is in your Project as Scripts and related files. So the number of unit test use cases are reduced in some respect.

  2. Thoughts on Migrations?

  3. A few common helper scripts are documented at: https://github.com/StephenOTT/ProcessProjectTemplate/blob/master/docs/helpers.md, would be interested to know of other common scripts that people use.

  4. The folders in the root of of the project(BPMN, DMN, and CMMN) were added for a more clean separation of files to make sorting through large projects easier. But would be very interested to hear back experiences on this, where you have projects with large numbers of process files.

  5. In this current model, all files would be redeployed. Meaning that we are not taking advantage of redeployments, and other selective process file deployment strategies. Interested to hear peoples thoughts on how this could be addressed.

1 Like
2

added support for Basic Auth. Its just a simple first iteration as pipeline parameters

3

Add multi-environment support example:

We saw the need to support allowing the process projects to be deployed to different instances for testing.

In these examples, we wanted to make the process files (BPMN, CMMN, and DMN) to be un-aware of their environments. Any values that change per env are stored in config.json, and any different files that need to be deployed (such as different config.json files per env) are deployed based on deploy_[ENV].json.



you setup your CI to use a specific deploy_[ENV].json file based on your needs.

for us we use a jenkins pipeline.

for example

...
pipeline {
  agent any
  parameters {
    string(name: 'CAMUNDA_URL', defaultValue: 'http://camunda_url', description: 'URL of Camunda Instance.')
    choice(name: 'Camunda_ENV', choices: 'dev\nstage', description: 'What Config Environment Files should be used?')
    booleanParam(name: 'USE_BASIC_AUTH', defaultValue: false, description: 'Check this box if you want to use Camunda Basic Auth.')
    string(name: 'CAMUNDA_USERNAME', defaultValue: 'default_username', description: 'Camunda Basic Auth Username: Must not be empty if you use Basic Auth.')
    password(name: 'CAMUNDA_PASSWORD', defaultValue: 'default_password', description: 'Camunda Basic Auth Password: Must not be empty if you use Basic Auth.  WARNING: Passwords are exposed in the console output of this build script!!')
  }
...

and then anytime we need the the camunda env is looks like this in the pipeline:

...
 echo "Reading for deploy_${params['Camunda_ENV']}.json"
          try {
            deployConfig = readJSON file: "deploy_${params['Camunda_ENV']}.json"
          } catch (Exception e) {
            sendErrorMessageToHipchat("Cannot read deploy_${params['Camunda_ENV']}.json file.<br>Error:<br>${e}")
            error("Cannot read deploy_${params['Camunda_ENV']}.json file\nError:\n${e}")
          }
          echo "-------------------------------------------------------"
          echo "Reading deploy_${params['Camunda_ENV']}.json's deployment object:"
          try {
            deploymentObject = deployConfig['deployment']
          } catch (Exception e) {
            sendErrorMessageToHipchat("Cannot read deploy_${params['Camunda_ENV']}.json property: deployment<br>Error:<br>${e}")
            error("Cannot read deploy_${params['Camunda_ENV']}.json property: deployment\nError:\n${e}")
          }
...
3 Likes
4

Hi Stephen,

Thanks for providing your code. I really like the idea. Some thoughts on your questions:

Sounds like a good approach. Maybe it is possible to enable writing steps in Nashorn javascript. Then people can use use your template with very little pure Java knowledge. You could also provide a separate library with common step definitions, for example one that sets up a Mock REST endpoint or similar.

There is already a community extension in the BDD area, see GitHub - camunda-community-hub/camunda-bpm-jbehave: camunda BPM community extension providing support for JBehave testing framework.

I haven’t got much to say here. Just be aware that making multiple REST requests (one for deployment, x for migrations) means that the process will not be atomic anymore.

Here as well, it could be nice to publish these scripts as a separate Maven artifact that people can depend on.

Cheers,
Thorben

5

Yes i was looking at these and thinking the same, but my Java skills are not there to build that myself :stuck_out_tongue:

Would be very interested to collaborate with someone on this.

6

I have updated the jenkins file with a more robust version with extra error handling, file checks, more robust camunda error handling, and if you use HipChat, Build Status and error outputs (all hipchat code has been commented out in the jenkins file):

The HipChat output looks like this:
The Error message is the JSON response from the Camunda when a deployment fails.

Jenkins-HipChat-BuildStatuses.png843Ă—648 86.3 KB

and what it looks like from Jenkins UI:

jenkins.png803Ă—477 38.4 KB

7

@thorben if you have ideas on how the automated testing could work using nashorn scripts, I would be VERY interested to discuss!

8

Hi Stephen,

I am not really familiar with how cucumber-java works. My idea is that if cucumber is extensible in the sense how it loads step definitions, then it could be possible to plug a nashorn script into that. Unfortunately I can’t look into that myself for now.

Cheers,
Thorben

9

There is also cucumber-rhino, but apparently with zero documentation.

10

Ya the entire cucumber-java/rhino libs seem to have very little docs…

The jbehave is more of the “spaceship” model when we really just need a basic sedan.

When i did some testing using the cucumer-java lib, i could get the scripts to execute, but could not get the engine fully bootstrap and would get a error related to no process engine id.

11

Feel free to push some code to github (ideally in a fresh repo) and I can take a look when I have some spare time.

For cucumber-rhino, there are some unit tests on github that may give an idea how it is supposed to work, see https://github.com/cucumber/cucumber-jvm/tree/master/rhino/src/test.

12

Part of the issue at the time was trying to force the @rule to function inside cucumber:

…Initially when I started looking at Spring enabling the test coverage plugin, I was investigating how to be able to write Camunda tests with Cucumber. As I remember it, it is impossible to make Cucumber work together with JUnit Rules. …

13

Hey Stephen,

Great work here… I’m trying to get this working and get up to par to help…

I’m getting a bit lost on how to set up the Jenkins Job.

Do you have any documentation to help that I’m missing?

Thanks!

14

The Jenkins job is just a wrapper around the Camunda rest API: so you will need to adjust it for your needs. Otherwise you can use any pipeline tooling to run the deployment rest api call.

1 Like
15