Logback logging Camunda Engine

[kai]

Hi there,
I am working with Camunda for few months now and I want to utilize the logs a little bit more.

What do I want to achieve?
Logging Camunda Engine calls and behavior from categories like “org.camunda.bpm.engine.jobexecutor” as written in the docs, to a file.

What did I do?
I consulted the docs (Logging | docs.camunda.org) how to configure logging. I think logback is the way I could use best, so I tried to do like it is documented.

→ There seems to be a missing link in the docs to “SLF4J Documentation”.

However, it’s not working as expected. There was no different logging output at all.

Through search in the forum, I found the thread where thorben was giving a good answer and OP said “It worked”.

Configuring logback for shared engine in tomcat

Sounds like a good plan. At least for those log statements that the engine makes and that are not made in the classloading context of your application, logback.xml in your application will not be useful I think. I just tried the following things myself and was able to get my logback configuration to work: Add logback-core and logback-classic jars to global lib folder Remove slf4j-jdk14 Edit ${TOMCAT_HOME}/conf/catalina.properties and add the following property: logback.configurationFile=<path…

At least the Engine outputs all the categories in the JUL log (Java Window). Nevertheless, I want that to be written in a file. But the logback.xml seems to be ignored.

My Steps

1. Download fresh Camunda Community Tomcat distribution
2. Copy logback-classic & logback-core to global lib in server/apache-tomcat/lib
3. Create a logback.xml inspired by Camunda docs (Copy-Paste) and add a file appenter

<configuration>

  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <!-- encoders are assigned the type
         ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

	<appender name="FILE-AUDIT"
		class="ch.qos.logback.core.rolling.RollingFileAppender">
		<file>${catalina.base}/logs/file.log</file>
		<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
			<Pattern>
				%d{yyyy-MM-dd HH:mm:ss} - %msg%n
			</Pattern>
		</encoder>

	</appender>
  
  <!-- camunda -->
  <logger name="org.camunda" level="info" additivity="false">
	<appender-ref ref="FILE-AUDIT"/>
  </logger>

  <!-- common dependencies -->
  <logger name="org.apache.ibatis" level="info" />
  <logger name="javax.activation" level="info" />
  <logger name="org.springframework" level="info" />

  <root level="debug">
    <appender-ref ref="STDOUT" />
  </root>

</configuration>

4. Delete slf4j-sdk14.jar and add jul-to-slf4j.jar to redirect JUL logging to slf4j

5. Add logback bridge to logging.properties
   handlers = org.slf4j.bridge.SLF4JBridgeHandler

Questions

Has someone already achieved the logging this way?
Did I miss something in my steps to produce a file output of the engine logs?

After adding slf4jBridgeHandler to properties error " java.lang.NoClassDefFoundError" is thrown. Seems as jul-to-slf4j is not found.

I really would appreciate some hints if some already managed to do this stuff.

[thorben]

Hi,

The easiest way on Tomcat is to configure Tomcat’s JULI logging which the engine integrates with by default. So you would use the Tomcat distribution as is and add your logging configuration to ${TOMCAT_HOME}/conf/logging.properties For example:

org.camunda.bpm.engine.jobexecutor.level=FINE
org.camunda.bpm.engine.jobexecutor.handlers=1catalina.org.apache.juli.AsyncFileHandler

edit: Details on logging in Tomcat here: https://tomcat.apache.org/tomcat-8.0-doc/logging.html

Cheers,
Thorben

[kai]

Hi Thorben,

thank you for this advise.

I tried it and it’s basically what I want to achieve (without logback but doesn’t matters) .
I extended the entry with a seperate FileHandler, that’s easier to parse later on.

5jobexecutor.org.apache.juli.AsyncFileHandler.level = FINE
5jobexecutor.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
5jobexecutor.org.apache.juli.AsyncFileHandler.prefix = jobexecutor.
5jobexecutor.org.apache.juli.AsyncFileHandler.maxDays = 90

org.camunda.bpm.engine.jobexecutor.level = FINE
org.camunda.bpm.engine.jobexecutor.handlers = 5jobexecutor.org.apache.juli.AsyncFileHandler

Thank you so much.

[Ilya_Malyarenko]

Hi Kai,

We recently enabled logging using Tomcat JULI as mentioned by Thorben below. In order to create a separate log file for each of business processes (distributed as war-files) we added a logging.properties file in /resources folder of each of the processes (please don’t forget to remove .txt ext):

logging.properties.txt (973 Bytes)

The only missing part of the equation is ability to define custom logging format like below as JULI does not take those definitions into account:
java.util.logging.SimpleFormatter.format=[%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL] %4$s: %5$s %6$s %n

The following groovy script dumps variable contents at each process step (if added into execution listener of the task) via Tomcat JULI:

import org.apache.juli.logging.Log;
import org.apache.juli.logging.LogFactory;

class AppLogger {
private Log log = LogFactory.getLog(AppLogger.class);
private def scope;
private String bk;
public AppLogger(def scope)
{
    this.scope = scope
};
def execute() {
      log.info ("*** " + scope.getCurrentActivityId() +": "+ scope.getCurrentActivityName() + " REF# " + scope.getProcessBusinessKey());
      log.info('==> Variables:');
      Map<String,Object> variables = scope.getVariables();
      variables.each{
        try
        {
            String variableName = it.getKey();
            def variableObject  = it.getValue();
            String variableClassName = variableObject.getClass().getSimpleName();

            log.info ("> " + variableName + " ["+variableClassName+"]: " + variableObject.toString());
            log.info('');
        } catch (err) {  };
    }
  }
}

AppLogger al = new AppLogger(execution);
al.execute();

Best regards,
Ilya

[kai]

Hi Ilya,

thanks for the script. This is very helpful for future stuff.

I just tried to implement a SimpleFormatter format by the example you linked like this

java.util.logging.SimpleFormatter.format = %1$tFT%1$tT.%1$tL%1$tz %4$s %2$s %5$s %n

and extended my jobexecution logger

5jobexecutor.org.apache.juli.AsyncFileHandler.formatter = java.util.logging.SimpleFormatter

At least it’s working fine for the AsyncFileHandler and now prints ISO dates as the timestamp (got it from here: http://looseconnection.tumblr.com/post/104589234906/iso-8601-date-and-time-with-javautillogging).

Regards
Kai

[Ilya_Malyarenko]

Hi @kai ,

Thank you for your email. Formatting works well if you define it Tomcat-wide, however, is not taken into account on per process basis. We are a bit limited in a way we can change configuration of the distribution,so this is still something to explore.

Best regards,
Ilya