Can HTTP-Connector Receive Binary Data?

Camunda 7 Topics Discussion & Questions
1

Working through some use cases using http-connector, and noticing that if you do a retrieve a file using http-connector and parse it using groovy with something like:

import org.camunda.bpm.engine.variable.value.FileValue
import org.camunda.bpm.engine.variable.Variables

String filename = "testFile.pdf";

InputStream stream = new ByteArrayInputStream(response.getBytes());

FileValue typedFileValue = Variables.fileValue(filename).file(stream).mimeType("application/pdf").create()

typedFileValue

The structure of the file is returned, but its essentially white pages, or other odd behaviour.

If you do the same but with a text file with something like:

import org.camunda.bpm.engine.variable.value.FileValue
import org.camunda.bpm.engine.variable.Variables

String filename = "testFile.txt";

String sampleString = "sample string 123";
InputStream stream = new ByteArrayInputStream(sampleString.getBytes());

FileValue typedFileValue = Variables.fileValue(filename).file(stream).mimeType("text/plain").create()

typedFileValue

The file downloads and loads as expected.

Am i missing something obvious here? My thoughts that are HTTP-Connector is returning response which is stripping out data when the response is binary data? Or am i just not parsing this correctly?

Thanks

2

Hi,

Perhaps given its a binary response, the sender has base64 encoded the content? I have successfully retrieved images (jpeg) from a camera server using a Camunda connector, however the images were base64 encoded…

regards

Rob

3

I am just preforming a regular GET on a URL pointing to a PDF file.

HTTP-Connector’s response returns a java.lang.String, and can convert that string into byte to send into FileValue, but does not seem to work out :wink:

4

Hmmm seems to be different results per PDF

This works: http://presidencycollege.edu.in/images/default/sample.pdf with the following script:

import org.camunda.bpm.engine.variable.value.FileValue
import org.camunda.bpm.engine.variable.Variables

String filename = "test11.pdf";

FileValue typedFileValue = Variables.fileValue(filename).file(response.getBytes("utf-8")).mimeType("application/pdf").create()

typedFileValue

But if you use the same script but with this PDF: http://www.pdf995.com/samples/pdf.pdf it does not work… (the PDF downloads and opens, but all the pages are blank)

5

other interesting result i am seeing if with file overwrite on download: On Safari Mac (latest), when you download the file from Cockpit in the process variable viewer, there are times when the file will overwrite the existing file with the same name, and other times it will create a new file with a incremented number appended (file.pdf, file-2.pdf, file-3.pdf, etc), but not sure whats causing this difference in behaviour…

6

Which Camunda version are you on?

7

@thorben, version 7.6

8

@thorben this is similar behaviour to what i am seeing when I download the “bad” pdfs:

9

In the first code snippet of your first post, what is response for an object?

10

response is the HTTP-connector’s standard response variable. Running response.getClass() tells me it is a java.lang.String

11

Ok, apparently the http connector is not really good at handling non-textual responses, as it always transforms them into a String, see

Anyway, if you’re lucky, you can change the invocation response.getBytes() to response.getBytes("utf-8") in your first code snippet and that way recover the binary response. That would also be in line with the explanation given in the stackoverflow post you referenced.

edit: you also don’t need a ByteArrayInputStream.

12

I have been toying with:
(I had dropped the inputstream)

FileValue typedFileValue = Variables.fileValue(filename).file(response.getBytes("utf-8")).mimeType("application/pdf").encoding("UTF-8").create();

and was the same issue/result as before.

Edit: Also when you remove the ByteArrayInputStream and just do something like: InputStream stream = response.getBytes("utf-8"), you get a Cast error:

Unable to evaluate script: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object '[B@1ed57915' with class '[B' to class 'java.io.InputStream'
13

Oh, I see. I guess then that converting the pdf to String and back is not lossless and I’m afraid there’s not much you can do without fixing the connector.

14

Yeah, sorry, I meant this Variables.fileValue(filename).file(response.getBytes("utf-8")) by not needing the stream :slight_smile:

15

In order to check my theory, could you write a unit test that loads the failing pdf (independent of Camunda), converts it into a String, then back, and then assert the equality of the binary representation?

16

It’s weird… cause it seems like it should be lossless because java.lang.String is not encoded to anything, so running getBytes should give the “true” / original data… (at least that was my understanding of how String and getBytes works :slight_smile:

17

kk. (have to write the test when back in the office)

Looks like your assumption about the String being the problem is a good bet: http://haacked.com/archive/2012/01/30/hazards-of-converting-binary-data-to-a-string.aspx/

Possible future fix: Provide a additional optional input to receive the response as a Base64.

18

Hey all

okay with the additional work i did with Jsoup, I have made a update to this problem.
See the below code snippet for a Jsoup solution to this problem.

Reference for Camunda docker with Jsoup: Replacing Http-Connector with Jsoup usage

cockpit
cockpit1153Ă—506 51.6 KB

binaryFileDownload-Jsoup.bpmn (3.8 KB)

create a downloadFile.js file and deploy it along with the above bpmn file.

The downloadFile.js has the following content:

function downloadFile(fileUrl)
{
  with (new JavaImporter(org.jsoup, java.io.BufferedInputStream))
  {
    var doc = Jsoup.connect(fileUrl)
                    .method(Java.type('org.jsoup.Connection.Method').GET)
                    .timeout(30000)
                    .ignoreContentType(true)
                    .execute()

    var bodyStream = doc.bodyStream()

    return bodyStream
  }
}

function saveFile(fileStream, fileName, mimeType)
{
  var file = Java.type('org.camunda.bpm.engine.variable.Variables')
                 .fileValue(fileName)
                 .file(fileStream)
                 .mimeType(mimeType)
                 .create()

  execution.setVariable(fileName, file)

}

function downloadAndSaveFile(fileUrl, fileName, mimeType)
{
  var file = downloadFile(fileUrl)
  saveFile(file, fileName, mimeType)
}

downloadAndSaveFile('http://www.pdf995.com/samples/pdf.pdf', 'mypdf.pdf', 'application/pdf')

The downloadAndSaveFile() function is a helper function hat saves you some time. and lets you download the file and save it as a Camunda File type all in a single function.

Lots of fun use cases!

3 Likes