Clarification for MetaXAP::parse required

Question

Does MetaXAP::parse handle UTF-16 encoded xml? I've knocked up a sample UTF-16 packet and libxmptk is throwing XAP_ERR_BAD_XML. I've traced this error through which is occurring deep within xmlparse.c.
At this stage it's likely that my UTF-16 sample is ill formatted. However there is no mention in the documentation whether MetaXAP::parse will handle UTF-16 or not. If not, then I presume we just convert the xml buffer to UTF-8 before sending to MetaXAP::parse?

Answer 1

Do you try to use the XAPDumper sample to verify your UTF-16 sample ?
In this sample, we can read :
Recognizing the byte order marker is a surprisingly messy thing to do. It can't be done by the normal string matcher, there are no intervening nulls. There are 4 transitions after the opening quote, the closing quote or one of the three encodings. For the actual BOM there are then 1 or 2 following bytes that depend on which of the encodings we're in. Not to mention that the buffer might end at any point.
The intervening null count done earlier determined 8, 16, or 32 bits per character, but not the big or little endian nature for the 16/32 bit cases. The BOM must be present for the 16 and 32 bit cases in order to determine the endian mode. There are six possible byte sequences for the quoted BOM string, ignoring the differences for quoting with ''' versus '"'.
Thus, I think UTF-16 is recognized by MetaXAP::parse but maybe with some precaution.