Automatic process deployment not working in fat jar

Hi, I have tried to run camunda engine as spring boot app as fat jar, and the processes are not deployed. Everything works fine when i run it from ide or via maven, problem is only with fat jar. There was issue for this last year, Auto-deployment doesn't work for Fat-Jar · Issue #106 · camunda/camunda-bpm-spring-boot-starter · GitHub, however this seems to be another issue, as the code changed in as fix for that incident seems not to be being used.

I think I have identified the possible problem to method ClassPathProcessApplicationScanner.scanUrl(URL url, String paResourceRootPath, boolean isPaLocal, String[] additionalResourceSuffixes, Map<String, byte[]> resourceMap)

  protected void scanUrl(URL url, String paResourceRootPath, boolean isPaLocal, String[] additionalResourceSuffixes, Map<String, byte[]> resourceMap) {

    String urlPath = url.toExternalForm();

    if(isPaLocal) {

      if (urlPath.startsWith("file:") || urlPath.startsWith("jar:") || urlPath.startsWith("wsjar:") || urlPath.startsWith("zip:")) {
        urlPath = url.getPath();
        int withinArchive = urlPath.indexOf('!');
        if (withinArchive != -1) {
          urlPath = urlPath.substring(0, withinArchive);
        } else {
          File file = new File(urlPath);
          urlPath = file.getParentFile().getParent();
        }
      }

    } else {
      if (urlPath.startsWith("file:") || urlPath.startsWith("jar:") || urlPath.startsWith("wsjar:") || urlPath.startsWith("zip:")) {
        urlPath = url.getPath();
        int withinArchive = urlPath.indexOf('!');
        if (withinArchive != -1) {
          urlPath = urlPath.substring(0, withinArchive);
        }
      }

    }

    try {
      urlPath = URLDecoder.decode(urlPath, "UTF-8");
    }
    catch (UnsupportedEncodingException e) {
      throw LOG.cannotDecodePathName(e);
    }

    LOG.debugRootPath(urlPath);

    scanPath(urlPath, paResourceRootPath, isPaLocal, additionalResourceSuffixes, resourceMap);

  }

imho, the problem is on the line

int withinArchive = urlPath.indexOf('!');

and it should be

int withinArchive = urlPath.lastIndexOf('!');

I have deduced this from the fact, that when run the application via maven spring boot plugin, the url is “file:/C:/somePathToDependencyJarInMaven/omnichannel-poc-processes-1.0-SNAPSHOT.jar!/META-INF/processes.xml”

and when i run it as fat jar, it is “jar:file:/C:/somePathToFatJar/omnichannel-poc-boot-app.jar!/BOOT-INF
/lib/omnichannel-poc-processes-1.0-SNAPSHOT.jar!/META-INF/processes.xml”

So in result(in fat jar case), the processes will be looked for in “file:/C:/somePathToFatJar/omnichannel-poc-boot-app.jar” instead of in “file:/C:/somePathToFatJar/omnichannel-poc-boot-app.jar!/BOOT-INF
/lib/omnichannel-poc-processes-1.0-SNAPSHOT.jar”

Unfortunately, I am not able to verify my hypothesis, as when I override the class in my project, the camunda-engine original class is still being used for whatever reason. Could somebody verify my hypothesis and raise an issue for this.

Hello, of course the moment I hit post button I realized, why I can’t override the original class.
My guess, where the problems is, was probably correct.

Unfortunately, the fix won’t be that easy as suggested above, as the real path is jar archive inside jar archive, so whole code will have to be refactored. The method

protected void handleDirectory(File directory, String rootPath, String localPath, String paResourceRootPath, boolean isPaLocal, String[] additionalResourceSuffixes, Map<String, byte[]> resourceMap) {

will have to get on more parameter with the path to the processes jar insided the main jar, and if the parameter is not null, the method will have to look the processes one level deeper inside the zip file.

Or am I doing something wrong and the deployment is working correctly?

Hi @tomorrow,

how do you build your fatjar?

I think you should be using https://maven.apache.org/plugins/maven-shade-plugin/ and extract all lib jars, then there will be no problem.

Cheers,
Askar

Hi, I am using spring boot, therefore I am using maven spring boot plugin for assembling fat jar. According to this article, I could probably use spring boot plugin together with maven shade plugin. What are the benefits of having the lib jars extracted in the fat jar?

Hi @tomorrow,

your code will be working, that’s the main advantage I would expect. :slight_smile:

Cheers,
Askar

so I’ve managed to get the fat jar working with following code change in ClassPathProcessApplicationScanner. Using of lastIndexOf mentioned earlier is used to, and together with this code it works.

protected void scanPath(String urlPath, String paResourceRootPath, boolean isPaLocal, String[] additionalResourceSuffixes, Map<String, byte[]> resourceMap) {
    if (urlPath.startsWith("file:")) {
        urlPath = urlPath.substring(5);
    }
    String outerUrlPath = urlPath;
    if (urlPath.lastIndexOf('!') > 0) {
        outerUrlPath = outerUrlPath.substring(0, urlPath.lastIndexOf('!'));
    }

    File file = new File(outerUrlPath);
    if (file.isDirectory()) {
        String path = file.getPath();
        String rootPath = path.endsWith(File.separator) ? path : path + File.separator;
        handleDirectory(file, rootPath, paResourceRootPath, paResourceRootPath, isPaLocal, additionalResourceSuffixes, resourceMap);
    } else {
        int metaFileArchiveLevel = StringUtils.countMatches(urlPath, "!");
        if (metaFileArchiveLevel > 0) {
            File innerFile = extractInnerArchive(file, urlPath.substring(urlPath.lastIndexOf("!") + 2));
            if (innerFile != null) {
                try {
                    handleArchive(innerFile, paResourceRootPath, additionalResourceSuffixes, resourceMap);
                } finally {
                    if (!innerFile.delete()) {
                        //TODO use logger instead
                        System.out.println("Could not delete " + innerFile);
                    }
                }
            }

        } else {
            handleArchive(file, paResourceRootPath, additionalResourceSuffixes, resourceMap);
        }
    }
}

protected File extractInnerArchive(File file, String innerArchivePath) {
        File tempFile;
        FileOutputStream tempOut = null;
        try (ZipFile outerZipFile = new ZipFile(file);) {
            tempFile = File.createTempFile("tempFile", "zip");
            tempOut = new FileOutputStream(tempFile);
            IOUtils.copy(
                    outerZipFile.getInputStream(new ZipEntry(innerArchivePath)),
                    tempOut);
            return tempFile;
        } catch (IOException e) {
            throw new IllegalStateException(
                    "Error occured while creating temp file, deployment cannot be finished", e);
        } finally {
            IOUtils.closeQuietly(tempOut);
        }

    }

As @aakhmerov mentioned, the problem should be solvable with extracted lib jars inside the fat jar, but with this sollution, it would work independently of the way the fat jar is built.

1 Like

Well, of course, that is true, but then, there should either be some mentioning in the docs(I haven’t found nothing like the here https://camunda.github.io/camunda-bpm-spring-boot-starter/docs/current/index.html), that using camunda in spring boot app requires that way of building the fat jar, or camunda’s deployment of processes should work no matter the fat jar is built.

Btw, isn’t there similar problem with wars/ears? Does this require to extract the lib jars too?

@tomorrow, I think that wars\ears are deployed in shared engine and therefore do not require any extra packaging steps.

About spring boot community extension, please feel free to prepare a pull request or contact maintainers.

Cheers,
Askar.

@aakhmerov thank you, I haven’t realized camunda spring boot extension maintained by another people than camunda process engine.

Hi @tomorrow,

Maybe you could create a PR with your code changes to the camunda-bpm-platform project as the ClassPathProcessApplicationScanner is part of the core engine.

Cheers,
Christian

1 Like

Hi I certainly could, however as I haven’t done anything opensource before, also I am new to git, the time needed for me to learn this stuff, my company currently doesn’t have. So if somebody more educated would prepare the PR, I would be grateful, as that would bring the fix/change to some official release much sooner.

One more question, wouldn’t extracting the lib jars introduce problem, when there are more processes lib jars inside fat jar, with different processes.xml, as the current code would find all *.bpmn files with the first metamodelfile? This would mean that only one processes.xml could be used, and that would be probably a step back in modularization of the application.

@tomorrow, theoretically, yes. But in that case I would assume one would like to use annotations instead of .xml configuration file.

If one wants to use .xml descriptor, then packaging and split into modules has to be setup manually and probably will be more tricky.

Does that answer your question?
Askar

Sorry, partially a lie, you could configure different locations for .xml using annotation.

Cheers,
Askar

Hi, I probably understand what’s you point.
In @ProcessApplication enumerate different processes.xml and each processes.xml has specified some unique subdirectory, where the processes are, right?

@tomorrow, em probably not enumerate, but just have multiple classes annotated each one with separate subfolder location for .xml configuration file and unique name.

Cheers,
Askar

Ok now I hope I understand, you’re considering situation, where there are multiple process engines annotated with @ProcessApplication probably. Currently I have only one process engine deployed in one fat jar and hypotheticaly, I could have multipe jars with processes for one engine. Would you advise to have all the processes in one jar for one process engine?

I guess this is related to the way spring boot changed its executable jar layout: http://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html

if you use @EnableProcessApplication and processes.xml it should still work as expected … the spring-classpath scan lead to other problems in the past (forms not accessible for example), so process application is the preferred way.

I guess its possible to tell the spring boot maven plugin to use the old layout, but I am not sure where I read about this.

An engine PR won’t be the right solution imho because it is spring boot specific, so we should fix this in the extension (or disable classpath scan) …

Hi, using @EnableProcessApplication was not enough. I am using it(and was before) and I needed to do the patch mentioned above to get it to work. Or am i using the application incorrectly?
This is my spring boot application.

@SpringBootApplication
@EnableProcessApplication
@EnableConfigurationProperties
@EnableEurekaClient
@EnableFeignClients(basePackages = {"sk.slsp.integration.osb.impl.netflixfeign"})
public class POCApplication extends OmnichannelProcessEngineBootApplication {

    public static void main(String[] args) {
        SpringApplication.run(POCApplication.class, args);
    }
}

As mentioned before, everything works ok when run from ide or from maven. The problem is only with the fat jar because of how the process deployment is implemented.

I know this is two years old, but the issue still exists and I found this discussion when looking for a solution. Just wanted to add: Instead of switching to maven-shade-plugin, you can also keep using the spring-boot-maven-plugin and configure to unpack the JARs containing the process definitions:

            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <requiresUnpack>
                    	<requiresUnpack>
                    		<groupId>com.example.myApp</groupId>
                    		<artifactId>my-example-artifact</artifactId>
                    	</requiresUnpack>
                    </requiresUnpack>
                </configuration>
1 Like