XmlLite with unicode content

Question

Hello,
I am writing a little example of XML with some Unicode (Chinese text) in it. If I write to a file it works fine:
<?xml version="1.0" encoding="UTF-8"?><message>世界您好</message>
If I read from memory I get this:
㼼浸敶獲潩㵮ㄢ〮•湥潣楤杮∽呕ⵆ∸㸿洼獥慳敧隸闧ꢂꗥ㲽洯獥慳敧
Here is my simple code:
CComPtr<IStream> stream;
std::shared_ptr<IXmlWriter> writer;
COM_VERIFY(CreateStreamOnHGlobal(nullptr, false, (LPSTREAM*)&stream), L"CreateStream");
COM_VERIFY(CreateXmlWriter(__uuidof(IXmlWriter), (void**)&writer, nullptr), L"CreateXmlWriter");
COM_VERIFY(writer->SetOutput(stream), L"SetOutput");
COM_VERIFY(writer->WriteStartDocument(XmlStandalone_Omit), L"StartDocument");
COM_VERIFY(writer->WriteStartElement(0, L"message", 0), L"StartMessage");
COM_VERIFY(writer->WriteString(originalText.c_str()), L"OriginalText");
COM_VERIFY(writer->WriteEndElement(), L"EndMessage");
COM_VERIFY(writer->WriteEndDocument(), L"EndDocument");
COM_VERIFY(writer->Flush(), L"FlushWriter");
STATSTG ssStreamData = { 0 };
COM_VERIFY(stream->Stat(&ssStreamData, STATFLAG_NONAME), L"GetSize");
SIZE_T cbSize = ssStreamData.cbSize.LowPart;
LPWSTR pwszContent = (WCHAR*) new(std::nothrow) BYTE[cbSize + sizeof(WCHAR)];
LARGE_INTEGER position;
position.QuadPart = 0;
COM_VERIFY(stream->Seek(position, STREAM_SEEK_SET, nullptr), L"SeekPosition");
SIZE_T cbRead;
COM_VERIFY(stream->Read(pwszContent, cbSize, &cbRead), L"Read");
pwszContent[cbSize / sizeof(WCHAR)] = L'\0';
What am I missing?
Thanks,
Pedro

Answer 1

Since the default output encoding is UTF-8, you cannot reinterpret the data as Unicode using the
pwszContent pointer. For debugging purposes you can cast it:
    LPCSTR p = (LPCSTR)pwszContent;
Then you will see a more valid content in Visual Studio, excepting the non-Latin letters.
In fact, the buffer (pwszContent) should be declared as
LPSTR instead of LPWSTR, since it is UTF-8.
In order to cast to Unicode:
    WCHAR ws[1000];
    MultiByteToWideChar( CP_UTF8, 0, p, -1, ws, 1000 );
Then you can assign it to wstring. To work with any length, see
MultiByteToWideChar.
In order to change the output encoding, consider the features related to
IXmlWriterOutput.