Shell Script Invoked by Camunda

I am using mysql database , the below is output of ACT_HI_JOB_LOG for the service task (which has name UserTask_076y3m5)

ID_, TIMESTAMP_, JOB_ID_, JOB_DUEDATE_, JOB_RETRIES_, JOB_PRIORITY_, JOB_EXCEPTION_MSG_, JOB_EXCEPTION_STACK_ID_, JOB_STATE_, JOB_DEF_ID_, JOB_DEF_TYPE_, JOB_DEF_CONFIGURATION_, ACT_ID_, EXECUTION_ID_, PROCESS_INSTANCE_ID_, PROCESS_DEF_ID_, PROCESS_DEF_KEY_, DEPLOYMENT_ID_, SEQUENCE_COUNTER_, TENANT_ID_

‘c32f4fbd-e425-11e7-9a0d-0242ac140003’, ‘2017-12-18 19:00:49’, ‘77e3f381-e425-11e7-9a0d-0242ac140003’, NULL, ‘3’, ‘0’, NULL, NULL, ‘2’, ‘44311700-e425-11e7-9a0d-0242ac140003’, ‘async-continuation’, ‘async-before’, ‘UserTask_076y3m5’, ‘77e37e50-e425-11e7-9a0d-0242ac140003’, ‘61eaf364-e425-11e7-9a0d-0242ac140003’, ‘XXX:1:4430efd8-e425-11e7-9a0d-0242ac140003’, ‘XXX’, ‘441662f5-e425-11e7-9a0d-0242ac140003’, ‘3’, NULL
‘77e41a92-e425-11e7-9a0d-0242ac140003’, ‘2017-12-18 18:58:43’, ‘77e3f381-e425-11e7-9a0d-0242ac140003’, NULL, ‘3’, ‘0’, NULL, NULL, ‘0’, ‘44311700-e425-11e7-9a0d-0242ac140003’, ‘async-continuation’, ‘async-before’, ‘UserTask_076y3m5’, ‘77e37e50-e425-11e7-9a0d-0242ac140003’, ‘61eaf364-e425-11e7-9a0d-0242ac140003’, ‘XXX:1:4430efd8-e425-11e7-9a0d-0242ac140003’, ‘XXX’, ‘441662f5-e425-11e7-9a0d-0242ac140003’, ‘1’, NULL

@Michal_S see the above edit.

Correct, my service task is Java Class which implements JavaDelegate.
Thanks for the assumption, unfortunately I think it is not exactly what I am trying to achieve
My process has a businessKey which is enough unique for a correlation of REST message as I expect to have only one running instance of a shell script per running Camunda Bpmn Instance, more then a one means failure.

Could you please explain a little bit more what executors limits are?
“this also assumes you are running a shell command outside of the executors limits - not tested. Thinking says probably not…):”

Thank you very much for your help! It is very appreciated :slight_smile:

See the details here where Thorben goes through the executor: Shell Script Invoked by Camunda

1 Like

@Michal_S here is a script for you to test.

I have not tested against the timeouts of the executor to see what is actually is occurring, but from a cockpit perspective is looks like its running outside of the executor:

So look at this:

shell_test

I run the following javascript:

with (new JavaImporter(org.apache.commons.exec)) {

  var myString = 'echo "hello Steve!" && sleep 15s && curl --request POST --url http://localhost:8080/engine-rest/message --header "Accept: application/json" --header "Content-Type: application/json" --data \'{"messageName":"myMessage"}\' && echo "hello Stephen!"'

  var shellCommand = new CommandLine("sh").addArgument("-c")
  shellCommand.addArgument(myString, false)

  var resultHandler = new DefaultExecuteResultHandler()
  var watchdog = new ExecuteWatchdog(5 * 60000)
  var executor = new DefaultExecutor()
  executor.setExitValue(1)
  executor.setWatchdog(watchdog)
  executor.execute(shellCommand, resultHandler)
}

Output would be something like this:

camunda_1  | hello Steve!

… 15 seconds later…

camunda_1  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
camunda_1  |                                  Dload  Upload   Total   Spent    Left  Speed
100    27    0     0  100    27      0    393 --:--:-- --:--:-- --:--:--   397
camunda_1  | hello Stephen!

In this scenario above, curl output and the hello Stephen! echo is from the script that executed in the “Run Shell” task, but it occurred while the engine was at the “Get Background” task.

What appears to happen is the background job is created outside of the Camunda Executor.
and then we run the Localhost:8080 curl to message back to the engine.
I also installed curl on the camunda server to run the command in the sh script.

Would be interested to hear from @camunda / @thorben about some likely issues with doing this ;).

Note:
The script was based on: java - execute shell command with org.apache.commons.exec.DefaultExecutor - Stack Overflow

1 Like

@StephenOTT
indeed, this looks interesting! the sad part is that I am not JavaScripter but definitely I have got homework to look at it :smile: and perhaps to learn some basics during holidays, could you please share your BPMN file?

Thanks

Michal

My code is just java code but written as JS. You can just change the “var” into their proper types and remove the first line and last line and you will have java. See the stack overflow link in my previous post.

I will not try to amplify Stephen’s excellent and comprehensive answers, I grovel before his knowledge. You might consider learning and using Groovy script as I’ve found it offers a wide range of functionality and “feels” similar to Java. That said, I have done the vast majority of my work in shell scripts in the past, so I don’t blame you for continuing to use them.

Hello Stephen,

Could you please explain how you made the script run in background ? was it set as an asynchronous task?

@Rajiv_Cj see the explanation from:

and note the last line that shows where my logic came from: java - execute shell command with org.apache.commons.exec.DefaultExecutor - Stack Overflow

The snippet of Javascript is basically executing apache commons exec creating a command line argument. We then use the Execute Watchdog feature of the exec lib to run the process as a background process/async process. Apache Commons Exec 1.3 API

This has does not have anything to do with Camunda’s “async” feature.

just to people whom it may help, using jsch

with (new JavaImporter(com.jcraft.jsch, java.util)) {

var jsch = new JSch();

var  config = new java.util.Properties(); 
config.put("StrictHostKeyChecking", "no");

var session = jsch.getSession('name','address',22);

session.setConfig(config);
session.setTimeout(20000);
session.setPassword('pass');
session.connect();

var channel = session.openChannel('exec');

channel.setCommand("cd /home/yyy/xxx && ./test.sh");

channel.connect();

channel.disconnect();
session.disconnect();


}
1 Like

@Michal_S can you code quote your code snippet?

@Michal_S, your example would be blocking, correct? Meaning that the duration of your execution on the remote session would take up the job executor. Has your requirement changed from your original post?

My requirements has not changed much. The code I have posted is just usage example. In my case I have decided to use Jsch and external task
vimeo video

as it is much more easier to implement.

The Camunda I am using lives in a container on a separate server. In order to run a script the way you have described, I would need to use swarm and docker secrets and many ssh tunnels to get to different servers.

Currently, I am looking how to store secretly passwords in Camunda, and it seems that I will need to use Docker Secretes anyway :smile:

Thank you

:+1:


Note that docker secrets only functions (in its designed secure way) when using Docker Swarm. So keep that in mind. If you are using Kuber, then use kuber secret volumes or one of the other options.

Docker Secrets will work when using Docker-compose (in non-swarm mode) because it was added for testing and development purposes, but the secrets are not actually secured in the way they are described in the docs when you are not using Swarm.

TLDR: If you are not using Docker Swarm, then Docker Secrets are not secure.

Hi Camunders,

Just exploring a new way how to execute a shell script.
I have found that Nashorn should be able to execute shel scripts
the documentation
https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html#sthref25
I have tried to execute commands locally via Nashorn
e.g.

#!/usr/lib/jvm/java-1.8-openjdk/jre/bin/jjs  -scripting
$EXEC("ls -l")
print($OUT)

which works fine.
But when inserted as a Inline Script (javaScript) into Modeler and executed I get
“…Unable to evaluate script while executing activity…”

Do u have any experience using jjs?
Thx.

M

Nashorn has different modes: the shell scripts are in scripting mode which is not the mode being used by camunda.

Have you tried using the shebang feature?

Hello guys,

had a chance again to work a little bit on this.
So I tried to implement the jsch solution in groovy

so far I came to the following with help of internet :

import com.jcraft.jsch.JSch
import com.jcraft.jsch.Session
import com.jcraft.jsch.UserInfo
import com.jcraft.jsch.Channel
import com.jcraft.jsch.ChannelExec

import java.util.Properties

import org.camunda.bpm.engine.delegate.DelegateExecution;
import org.camunda.bpm.engine.delegate.JavaDelegate;


def sshHost = '1111.12.12.11'
def sshUser = 'vqweq
def sshPass = 'wfdssx'
def sshPort = 22

println "Opening connection to ${sshUser}@${sshHost}:${sshPort}"
Properties config = new Properties()
config.put("StrictHostKeyChecking", "no")
JSch jsch = new JSch()


Session sshSession = jsch.getSession(sshUser, sshHost, sshPort)
sshSession.setPassword(sshPass)
sshSession.setConfig(config)
sshSession.connect()
println "Connected"

// Could use "shell"
Channel channel = sshSession.openChannel("exec")
// Let's just get the hostname
((ChannelExec)channel).setCommand("cd /home/.... && ./CMND_test.sh")
// Spew errors to console
((ChannelExec)channel).setErrStream(System.err)
// We're not sending anything
channel.setInputStream(null)
// Get the input stream
InputStream is = channel.getInputStream()
// Connect
channel.connect()
// This could be written better and groovier...
byte[] tmp = new byte[1024]
// Uh oh. We really need a better way out of the loop...
while (true) { 
  // But it's just an example... :-)
  while (is.available() > 0) {
    int i = is.read(tmp, 0, 1024)
    if (i<0)
      break
    System.out.print(new String(tmp, 0, i))
  }
  if (channel.isClosed()) {
    // All done.
    System.out.println("exit-status: " + channel.getExitStatus())
    def Validity = execution.setVariable('Valid', channel.getExitStatus());
if ( Validity == 1 ) {
 execution.setVariable('Valid', true)
} else if ( Valid != 1) {
   
     execution.createIncident("someType", "someConfiguration", "someMessage");
 // throw new org.camunda.bpm.engine.delegate.BpmnError('Error')
}
    break
  }
  // Ugly: You might want to change this
  try{Thread.sleep(1000);}catch(Exception ee){}
}
// Close channel and session
channel.disconnect()
sshSession.disconnect()

which gives image
which is similar to Error creating incident via javascript - #7 by StephenOTT
which I don’t like as it creates incident and token continues in my process flow
I have tried to use @StephenOTT context but with no luck

If u have an idea how to keep incident “token” on an error task then thank you for advice.

If you want to keep the incident, then use the latest API to throw the incident(execution.createIncident(), or something like that). Or throw just throw a regular error and it will be caught as a incident and stop on the task.