Different results for IncidentQuery list and count operations

nvanbelle · September 6, 2017, 2:06pm

Hi all,

I’m using Camunda 7.5.0 and I’m getting different results for both calls which is odd. I’m wondering if this is a bug or not.

RuntimeService().createIncidentQuery().count(); // returns 4
RuntimeService().createIncidentQuery().list(); // returns 1 Incident object

When I call the REST-service instead http://localhost:8080/engine-rest/incident I get returned all 4 objects…

The contents of my act_ru_incident table can be found here, as it’s messy in markdown.

Below is the result of the list() operation:

[{
    "revision": 1,
    "id": "1ccaefb7-7293-11e7-b7ca-c60a45aabea2",
    "incidentTimestamp": 1501136182691,
    "incidentType": "failedJob",
    "executionId": "9ca1e7da-7291-11e7-b7ca-c60a45aabea2",
    "activityId": "CreateNotification",
    "processInstanceId": "9ca1e7da-7291-11e7-b7ca-c60a45aabea2",
    "processDefinitionId": "productOrderProcess:32:a1f12cc7-1acb-11e7-bb8c-c60a45aabea2",
    "causeIncidentId": "1ccaefb7-7293-11e7-b7ca-c60a45aabea2",
    "rootCauseIncidentId": "1ccaefb7-7293-11e7-b7ca-c60a45aabea2",
    "configuration": "9ca39597-7291-11e7-b7ca-c60a45aabea2",
    "incidentMessage": "ENGINE-03051 There was an exception while invoking the TaskListener. Message: 'Error while evaluating expression: ${newNotification.create(task)}. Cause: javax.ejb.EJBTransactionRolledbackException: HTTP 502 Bad Gateway'",
    "jobDefinitionId": "a1f12cc9-1acb-11e7-bb8c-c60a45aabea2",
    "referencedEntityIds": [
        "1ccaefb7-7293-11e7-b7ca-c60a45aabea2"
    ],
    "persistentState": {
        "executionId": "9ca1e7da-7291-11e7-b7ca-c60a45aabea2",
        "processDefinitionId": "productOrderProcess:32:a1f12cc7-1acb-11e7-bb8c-c60a45aabea2",
        "activityId": "CreateNotification",
        "jobDefinitionId": "a1f12cc9-1acb-11e7-bb8c-c60a45aabea2"
    },
    "revisionNext": 2
}]

thorben · September 6, 2017, 3:03pm

Can you provide a test case that reproduces this?

nvanbelle · September 7, 2017, 6:27am

Hi thorben, can you specify what you mean by test case because

 @Inject
 private ProcessEngineServices engineServices;

public List<Incident> getIncidents() {
	List<Incident> incidents;
		
	try {
	        incidents = engineServices.getRuntimeService().createIncidentQuery().list();	
	} catch (Exception ex) {
		incidents = Collections.emptyList();
	}

	return incidents;
}

is the only thing I’ve done.

thorben · September 7, 2017, 7:18am

A JUnit test where you deploy an example process model, start one or more process instances and then bring them into the state that the queries return different results would be great. You can use the unit testing template to get started: https://github.com/camunda/camunda-engine-unittest

nvanbelle · September 7, 2017, 9:11am

I’ve created a unit test before to test the happy path / sunshine scenario, but never failures. I honestly wouldn’t know how to fail jobs programatically -I can only execute(job()) or complete(job())- or otherwise insert those 4 records into the H2 test db in a unit test.

thorben · September 7, 2017, 9:26am

You could write a delegate that throws an exception based on a process variable. As an alternative to reproducing this via testing, you can also set the log level for the logger org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity to DEBUG or INFO or TRACE. This will log the SQL statements that are executed by the queries which should show if there are any differences.

nvanbelle · September 7, 2017, 11:21am

Okay so he has found the 4 records, but for some reason the marshalling to JSON throws an NPE for one of the objects.
I hope this is due to missing records in other tables (I manually cleared some other tables a couple of months ago) but I don’t think so as no other queries are being launched. Or perhaps it tries to link some info from missing cache entries due to my db cleanup?

13:10:02,237 FINE  [org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity.selectIncidentByQueryCriteria] (org.apache.ibatis.logging.jdk14.Jdk14LoggingImpl.debug(Jdk14LoggingImpl.java:51)) ==>  Preparing: select distinct RES.* from ACT_RU_INCIDENT RES order by RES.ID_ asc LIMIT ? OFFSET ? 
13:10:02,237 FINE  [org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity.selectIncidentByQueryCriteria] (org.apache.ibatis.logging.jdk14.Jdk14LoggingImpl.debug(Jdk14LoggingImpl.java:51)) ==> Parameters: 2147483647(Integer), 0(Integer)
13:10:02,243 FINE  [org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity.selectIncidentByQueryCriteria] (org.apache.ibatis.logging.jdk14.Jdk14LoggingImpl.debug(Jdk14LoggingImpl.java:51)) <==      Total: 4
13:10:02,245 ERROR [io.undertow.request] (io.undertow.servlet.api.LoggingExceptionHandler.handleThrowable(LoggingExceptionHandler.java:80)) UT005023: Exception handling request to /dashboard/web-rest/process-instances/incidents/: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception

Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity["execution"])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177)
	at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:199)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:683)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:157)
	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1387)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:889)
	at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:635)
	at com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider$Proxy$_$$_WeldClientProxy.writeTo(Unknown Source)
	at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131)
	at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60)
	at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120)
	at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145)
	at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124)
	at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100)
	at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124)
	at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:466)
	... 33 more
Caused by: java.lang.NullPointerException
	at org.camunda.bpm.engine.impl.persistence.entity.IncidentEntity.getExecution(IncidentEntity.java:387)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:654)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:675)
	... 51 more

thorben · September 7, 2017, 12:02pm

Is this a custom REST endpoint that uses Jackson to marshall Incident objects?

nvanbelle · September 7, 2017, 12:14pm

@thorben Yes, that is correct.

thorben · September 7, 2017, 12:19pm

I believe that approach doesn’t work well with the API interface implementations. Jackson uses reflection to serialize the values of all getter methods. That works fine for the getters of the Incident interface, but fails for some non-API getters (like getExecution in this case). Instead of marshalling Incident directly, I recommend to write a DTO type class that contains exactly those fields that you want to have in the response, like this: https://github.com/camunda/camunda-bpm-platform/blob/master/engine-rest/engine-rest/src/main/java/org/camunda/bpm/engine/rest/dto/runtime/IncidentDto.java

nvanbelle · September 7, 2017, 12:22pm

Yes, I was now looking at the IncidentEntity implementation and I realised this was a recipe for disaster.

public ExecutionEntity getExecution() {
    if(executionId != null) {
      return Context.getCommandContext()
        .getExecutionManager()
        .findExecutionById(executionId);
    } else {
      return null;
    }
  }

As always, thanks for your efforts investigating and guiding me