This may be a bit “hackish”?
But could you just create a script on the Start Event: Start Listener:
var instances = execution.getProcessEngineServices().getRuntimeService().createProcessInstanceQuery().processDefinitionKey("concurrentTest").active().count();
execution.setVariable("instances", instances);
if (instances > 1) {
throw "too many instances are running";
}
and if you run through tasklist:
and through the API:
Note: the variable being set in the script is just for testing and seeing what was going on.