We are having the same issue here: get a list of historical process instances sorted by the value of a specific variable (the name of the variable to sort by can vary). This should be the same as with “normal” process instances.
I have read about the possibility of creating a custom query but I cannot find the correct way to implement that and get it working in the Camunda engine and have it mapped to a REST-Url.