Non-English characters not displaying correctly - Serious Issue

My corporate email is on a Lotus Domino server with Lotus Traveler installed.
I have set my PlayBook (with OS 2) up to syncronize with the corporate email trough Active Sync (see http://alturl.com/qh3nn), which works perfectly.
I have however noticed that in some emails special non-english characters are displayed correctly but in some emails special non-english characters are displayed as a black diamond with a question mark inside.
This is of course a serious issue as most non English speaking countries use some special characters.
When trying to understand this problem how can I analyse the emails and see what character set is being used?
And of course better; has someone solved this?

I am having the same problem. Is there any update available?

Similar Messages

Problems with Turkish characters not displaying correctly

A user is having problems with certain Turkish characters in particular fonts not displaying correctly (as shown below).
The two fonts (Gill Sans and Helvetica LT CondensedLight) are used for English language marketing material. Ideally, for continuity, the same font would be preferred for our Turkish language marketing material rather than changing it to a similar font.
Helvetica LT CondensedLight
Turkish character “ İ ” does not display correctly in either Word or InDesign.
Gill Sans
Turkish characters “ ş ” and “ ğ ” display correctly in Word but not InDesign.
My questions are:
1. For Gill Sans, why are the characters not displaying in InDesign but are in Word?
2. I've noticed here: http://www.webtype.com/font/gill-sans-family/#glyphs-tab that the required characters are available within the Gill Sans font. Is this because the font is newer and includes the additional characters? Would this work in InDesign?

One of the big differences between InDesign and Word is that Word will happily substitute glyphs without telling you. So, unless your installs of Word and of InDesign are using different versions of Gill Sans (which is possible, BTW) then your s-with-cedilla and your g-with-macron are actually not in Gill Sans but in some other font. I'm guessing Arial.
It's possible that you have more than one Gill Sans installed (e.g. one in your Document Fonts folder and another in your system fonts folder) that have different glyph complements. Gill Sans has been around a long time, and the version I have from the 90s has no Turkish support whatsoever.
As far as the capital-I-with-dot, you can check your glyph coverage in a variety of ways. In both InDesign and Word, you can open up a window that shows you all of the glyphs in a font. In InDesign, find it in Type -> Glyphs. In Word, it's called "Insert Symbol" and you can find it in Word 2010 by going to Insert -> Symbol -> More Symbols. I am guessing that the answer to this question is simply that your cut of Helvetica LT has no cap-I-with-dot. You'd need to pick a font that actually has support for Turkish. On WIndows, I use a very full-featured freebie called BabelMap to check font coverage. They have a Web version here:
http://www.babelstone.co.uk/Unicode/babelmap.html
but I personally prefer the downloadable .exe file.

Thai characters not displaying correctly within App

Hi everyone,
I have a Adobe Flex Mobile app displays Thai text, however this is not displaying correctly. While developing the App on my Windows machine and testing using the Adobe Flash Builder mobile emulator, the font/characters look correct.
However when running the app on either the MAC Adobe Flash Builder emulator, the iPhone simulator or on an actual iPhone, the font/characters are incorrect.
First I thought that it was maybe an encoding issue (I am using locale properties files to replace text), so followed instructions at this website Flash Builder, Flex language bundles showing strange symbols. Not utf8 encoded.Sefol however this did not solve the issue.
Looking more closely at the text rendered, it looks as though all of the characters for each word are actually being rendered, but they are just being rendered in correctly. Where as if it was a encoding issue then it would be missing characters completely.
So if you look at the first character, in the correct version it is built up of 3 components. If you look at the second image, you can see that these three components are present but just not all appearing within one character.
Is this a font issue? or is it actually an encoding/rendering issue?
My code for testing this is just a simple label tag
<s:Label text="ชื่อผู้ใช้" fontFamily="Arial" fontSize="12" />
Many thanks.

Ok so after doing a bit more of an investigation into the Font being the issue and not the encoding type, it seems as though if I use a specific font which I know has the Thai characters, then they are displayed correctly.
I have embedded the font into the App using the code:
[Embed(source="../assets/Kinnari.ttf", mimeType="application/x-font", embedAsCFF="true", fontFamily="ThaiFont")]
private var ThaiFont:Class;
Then on my UI I have added the font:
<s:Label text="ชื่อผู้ใช้" fontFamily="Arial" fontSize="12" fontFamily="ThaiFont" />
I was able to find this font at a Hawaiian Educational website! Thai fonts :: Thai Language Program, University of Hawai'i, Manoa
Thanks @tooMuchTrouble for your help.

German Special Characters not displayed correctly in RTF using code

Hi ,
In my code we are using webdynpro method
WDResourceFactory.createResource(
byte[] data, String resourceName,WDWebResourceType
Type)
Here in our code we are implementing this as
ITemplateElement templateEl = wdContext.currentTemplateElement();
WDResourceFactory.createResource(
templateEl.getReportData(),
reportName.substring(0, reportName.lastIndexOf('.')),
WDWebResourceType.RTF); reportName.lastIndexOf('.')),
Here templateEl.getReportData() returns a set of bytes which has some
german special characters.
We are generating the Bytes using String.getByes(),Just before
String text = new String(in);
collector.putBusinessObject(boName, bo);
reportDocTemplateParser(collector, text);
collector.removeBusinessObject(boName);
String generatedText = collector.generateRTF();
out = (null != generatedText) ? generatedText.getBytes() : null;
The out put is the if i am giving a word with german special characters for eg:
Betriebsübersichten it first gets converted to bytes and then passes through method WDResouseFactory.createResourse(.....) which creates an RTF file and finally in the RTF file it appears as Betriebsbbersichten the special character is not displayed correctly.
i came to knw that while converting into bytes we have to make it RTF supported encoding.ie for eg generatedText.getBytes('cp1252').i even tried with other charactersets like ISO-8859,cp1253 and so on but none of them worked.
It would be really great if you could suggest the needful.
Thanks and Regards
Neeta

I soved this by using get_data function of response object. Then converting this into ISO-8859-1 charset.
See code below.
DATA : lv_encoding   TYPE abap_encoding,
          lv_conv       TYPE REF TO cl_abap_conv_in_ce,
          lv_x_string   type xstring.
lv_x_string = pv_http_client->response->get_data( ).
    lv_encoding = '1100'.
    lv_conv = cl_abap_conv_in_ce=>create(
                          encoding = lv_encoding
                             input = lv_x_string ).
    lv_conv->read( IMPORTING data = pv_result ).

Webservice: Special characters not displayed correctly

Hi,
I'm facing a problem when retrieving informations via a webservice. I'm able to use it and data is retrieved but when there are special characters into the response they are not displayed correctly. Repsonse of the webservice is XML formatted.
It seems to be a charcter set problem, strange thing is that the response is ISO-8859-1 formatted, this charset should normally display special characters like (é à ...) correctly.
In my code I simply use an if_http_client object to use the webservice.
pv_result = pv_http_client->response->get_cdata( ).
Xml repsonse :
<?xml version="1.0" encoding="ISO-8859-1" ?>
<![CDATA[Eetcaf�/Steakhuis Baskent]]>
Should be Eetcafé
How can I specify a charset for the response object?
I did the same in .NET, where I can bypass this issue specifying the charset like this.
Dim reader As New StreamReader(oResponse.GetResponseStream(), System.Text.Encoding.Default)
Thanks a lot for your help.
Edited by: dom___35 on Dec 21, 2009 3:30 PM

I soved this by using get_data function of response object. Then converting this into ISO-8859-1 charset.
See code below.
DATA : lv_encoding   TYPE abap_encoding,
          lv_conv       TYPE REF TO cl_abap_conv_in_ce,
          lv_x_string   type xstring.
lv_x_string = pv_http_client->response->get_data( ).
    lv_encoding = '1100'.
    lv_conv = cl_abap_conv_in_ce=>create(
                          encoding = lv_encoding
                             input = lv_x_string ).
    lv_conv->read( IMPORTING data = pv_result ).

Characters not displayed correctly

Hi,
After support package installation, chararcters are not displayed correctly when i use application server,
for example when i write internal table into a file in text mode and read it after into attachment in binary mode
but characters are not correct. And after mail sent, the half or file is sent, not all data available.
What could be the reason?
Char exp in debug of attachment: ㅔ䠰㘹㌲〰㤰㘰㌍ਲ䙈〶㤳㈰〹
I get above char after i read the file, code like following:
do.
clear attch.
read dataset filename into attch.
if sy-subrc <> 0.
append attch.
exit.
endif.
append attch.
enddo.
Thanks.

I think this problem is some what related to Unicode and you should use Encoding statements whith ur open dataset stetement as follows.
open dataset ....................Encoding UTF-8.
Encoding UTF-8 is for unicode environment.
Hope this will help u.

Non-breaking hyphen not displayed correctly

The non-breaking hyphen (\u2011, &#8209) is not displayed
correctly - the wrong glyph is shown.
To reproduce, do the following:
1. Start the TLF demo editor
http://labs.adobe.com/technologies/textlayout/demos/
2. Import the markup below
Results:
- Line wrapping is correct: no line break at the hyphen
- Times New Roman on Windows does have the glyph defined - it
should look like an ordinary hyphen.
This was seen in Build 3291.
Cheers
David
Markup:
<?xml version="1.0" encoding="utf-8"?>
<flow:TextFlow whiteSpaceCollapse="preserve" xmlns:flow="
http://ns.adobe.com/textLayout/2008"><flow:p
marginRight="5"><flow:span fontSize="60">
2011 non-breaking hyphen Macro‑media
</flow:span></flow:p></flow:TextFlow>

Hi Chris,
you're right, Times New Roman does not contain the glyph. I
was seeing 2013 whlie I type 2011 -- my mistake, sorry.
However, it's a pity non of the OTFs implement this glyph
(also Myriad Pro doesn't, for instance). Open Office 3 seems to
replace it by a hypen (which is greyed), and InDesign CS4 also
seems to replace the character and does no word-wrap at this
position.
Of course it would be very convenient if Flash could handle
it the same way..
Btw, character map is the same as my "chart table
application", at least it's my translation from the German
"Zeichentabelle". :-)
So, sorry again for the confusion...
david

Filename in non-english are not saved correctly

hi
I recently (last week) installed archlinux.
I come from manjaro so I am to some extent familiar with archlinux.
now I am trying to solve some issues that have poped up in arch that there were not in manjaro( mostly because manjaro was pre-configured).
when I use jdownloader to save youtube videos the files names are displayed correctly in jdownloader but not in konsole or dolphin file manager. (they are shown as "????" characters).
this happens for Persian and Russian characters.
I thought this was jdownloader issue but this doesn't happen on my pc that has a manjaro installed on it.
and another thing that makes it weird is that if i create files in dolphin with say Persian language they are shown and treated correctly. but the ones that are created by jdownloader have this issue.
and another related symptome is that some files that I download from subtitle websites that are in zip format and have some rare European characters in them (like the French and German ones) I cant open them or extract them in dolphin file manager.
I had a similar issue like this in manajro but in manajro I used file-roller to open the files and that app would work with this files correctly.
but in archlinux even file-roller has this issue.
can somebody help me with this?

Lone_Wolf wrote:
rezad,
it was a long time ago i had that problem and locale handling is like a black box to me.
It's easy to reverse if it doesn't help though, why don't you try it ?
I have tried it.
adding other encoding before the famous en-US,UTF-8 one gives warning for default lang and falling back.
I see these error when running apps from command line.
adding other encodings after that encoding doesn't show this warning but i have a suspicion that it mean that other encoding are suppressed,
i have to try it with a file with a wired but known encoding and then adding that encoding to config and testing again.
my problem is how do i find the filename encoding?(not find content encoding)
this is the file in question. can somebody tell me what is the filename encoding?
https://www.sendspace.com/file/fwmxpc
EDIT: my findings on this matter.
even adding all the langs to /etc/locale.gen and then using locale-gen command doesnt fix this. cause this are different issues.
this problem as a whole doesn't have a solution, cause linux unlike windows doesn't have a default inbuilt filename encoding and it all depends on your environment.
linux just treat the filename in the encoding you tell it. so it cant detect anything cause there is nothing to detect.
from its perspective filename are just binary data ready to be interpreted as wished by the user.
widows on the other hand uses UTF-16.
plz if anybody have another contradicting point ,then reply but plz dont say I have to use correct encoding. the only way to fix it is using convmv util and that requires finding the correct original encoding.
thanks for all the replys.
Last edited by rezad (2015-04-20 14:45:14)

Utf8 characters not displayed correctly

I have a database created with collation of latin1_swedish_ci
which contains big5. I realise that I have to create the db using
utf8 in order to display correctly. However, due to the db was
created long ago, I could not change it. Is it possible to do a
conversion before the characters are displayed?
default charset of apache is utf-8
connection string is added
"useUnicode=true&characterEncoding=UTF-8"

dynason wrote:
> I have a database created with collation of
latin1_swedish_ci which contains
what "database"? what datatypes?
> big5. I realise that I have to create the db using utf8
in order to display
if the columns holding your text datatypes aren't unicode
capable, then you'll
never get the data out except the same way it went in.
depending on the db, datatypes, app server, it's a good bet
that the data's not
big5 but some corruption of it.

Non-English fonts not rendering correctly in iMovie 10.0.3?

I use xenotype Tibetan font in iMovie and it has always rendered correctly until I updated from 10.0.2 to 10.0.3. Now many of the stacked characters do not render correctly in any of the titles. Even projects that I made three months ago with 10.0.2 when opened in the new version render the font incorrectly. Is there a reason for this change? Is there any adjustment I can do? Line-height adjustment? iMovie 10.0.3 seems to be the only Mac software currently made by apple that doesn't render the font correctly.

Hello,
Welcome to the discussions. There have been some transition problems reported, although maybe not so many problems as you are experiencing.
I had the black/flash with the transition cross-dissolve in one of my movies recently. I found the suggestions in this thread to resolve that: http://discussions.apple.com/message.jspa?messageID=3842119#3842119
Have a look and see if it can help you.

Thai language characters not displaying correctly in browser

Hello all
I am starting to develop a multilingual application which will include both english and thai.
The problem Im having is that the browser displays only garbage when Im fetching a report on a table that includes Thai characters in a column.
What Im seeing is something like this:
Language Id     Short     Name
0     ENG     English
1     THA     à¸à¸à¹à¸à¸¢
Things that I have checked:
- The database character set is AL32UTF8
- The same is set for NLS_LANG when starting the database, listener, OHS
- The same is configured in dads.conf
- The browser is set to Unicode as well ( and can display normal thai websites without any problems)
I have tested on a windows 7 client, as well as locally to the database on the linux box using firefox, both show the same result.
In xterm I have no problems inserting and querying thai characters. Its all shown correctly.
I tried with chinese characters as well, and I see the same result.
Any ideas please let me know
Cheers
Stefan

Hi Stefan,
Sorry to interrupt you in this thread, Me too having the saming requirement of displaying my apex application in both Arabic and English.
Hence i have accomplished of displaying the page name, labels, and all the other stuff related to page in arabic expect the data, im not sure of displaying the data in arabic soon after the user select the Arabic language.
From your thread i can see that you are seeing your data in Thai it seems, if you dont mind can you share the stesp you did for showing the data in thai so that i can try the same for my Arabic too.
Also sorry my post couldnt help your question.
Thanks in advance.
Brgds,
Mini

German characters not displaying correctly when redaing from URLConnection

I have a java file reading the contents of a generated jsp page and writing to an html file and I have the code below doing that. However a couple of German characters like �, � and � are being changed to � in the html file. I tried to set the encoding for the reader, the OutputStreamWriter and also for the URLConnection, but it does not seem to help.
The data is fine in the database.
Any inputs on resolving this will be greatly appreciated.
public void makeLiveHTMLfile(String uri, String HTMLFileName, String siteUrl, String siteDir) throws
Exception {
//Local vars
int i = 0;
try {
getConnection(uri);
Reader rd = getReader(uri);
FileOutputStream output_file = new FileOutputStream(
SystemProps.getInstance().getProperty( WEBTOOL_HOME_DIR )+"/html/" + HTMLFileName);
OutputStreamWriter output = new OutputStreamWriter(output_file, "UTF8");
while (i != -1) {
i = rd.read();
if (i != -1) {
output.write(i);
rd.close();
output.close();
this.sendFile(HTMLFileName, siteDir, "html", siteUrl);
} catch (Exception e) {
throw e;
private Reader getReader(String uri) throws IOException {
if (uri.startsWith("http:") || uri.startsWith("https:")) {
// Retrieve from Internet.
return new InputStreamReader(conn.getInputStream(), "UTF8");
} else {
// Retrieve from file.
return new FileReader(uri);
private Writer getWriter(String uri) throws IOException {
if (uri.startsWith("http:") || uri.startsWith("https:")) {
// Retrieve from Internet.
return new OutputStreamWriter(conn.getOutputStream(), "UTF8");
} else {
// Retrieve from file.
return new FileWriter(uri);
private void getConnection(String uri) throws IOException {
if (uri.startsWith("http:") || uri.startsWith("https:")) {
// Retrieve from Internet.
conn = new URL(uri).openConnection();
conn.setRequestProperty("Content-Type", "text/html; charset=utf-8");
conn.setRequestProperty("Accept-Encoding", "utf-8");
conn.setRequestProperty("Accept-Charset", "utf-8");
conn.setDoOutput(true);
conn.setDoInput(true);
Thanks,
Monisha

This web page says that the correct encoding code is "UTF-8", not "UTF8".
http://java.sun.com/j2se/1.3/docs/api/java/lang/package-summary.html#charenc
Maybe there are aliases, and it recognises both.... but worth a try.
Also, when you open the web page with a browser and view the source, do you see Utf8 mentioned in the webpage ? Maybe it's not in utf8, and you're trying to force your app to read it as such.
Ps. we won the war... tell them to start speaking English !
regards,
Owen

Random text characters not displaying correctly

Hello,
I'm using Firefox 29.0.1 and I'm having a problem with words displaying incorrectly on Facebook.
http://i1343.photobucket.com/albums/o795/hatsunemii/ss_zpsa166e384.jpg
It's a strange problem with the fonts. As you can see in the screenshot, all the text characters should have the same style and font, but one of words is displaying incorrectly. I zoomed in to that word and you can see it's in bold??? In large paragraphs, there will be words like that all over the place and it's annoying... I tried reinstall FF, update my fonts, play with font settings in FF, but no luck...

Many site issues can be caused by corrupt cookies or cache.
* Clear the Cache
Press the '''<Alt>''' or '''<F10>''' key to bring up the tool bar.
Followed by;
Windows; '''Tools > Options'''
Linux; '''Edit > Preferences'''
Mac; ''application name'' '''> Preferences'''
Then '''Advanced > Network > Cached Web Content: Clear Now'''
and
* Remove Cookies
Press the '''<Alt>''' or '''<F10>''' key to bring up the tool bar.
Followed by;
Windows; '''Tools > Options'''
Linux; '''Edit > Preferences'''
Mac; ''application name'' '''> Preferences'''
Then '''Privacy.'''
Under '''History''', select Firefox will '''Use Custom Settings'''.
There is a button on the right side called '''Show Cookies'''.
If there is still a problem,
'''''[https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode Start Firefox in Safe Mode]''''' {web link}
While you are in safe mode;
Press the '''<Alt>''' or '''<F10>''' key to bring up the tool bar.
Followed by;
Windows; '''Tools > Options'''
Linux; '''Edit > Preferences'''
Mac; ''application name'' '''> Preferences'''
Then '''Advanced > General.'''
Look for and turn off '''Use Hardware Acceleration'''.
Poke around safe web sites and see if there is still a problem. Then restart.

[Solved] Characters not displaying correctly in some applications

When I read things in applications like Firefox I've noticed that sometimes the " symbol is displayed wrong. Sometimes it is shown as what I can only describe as two forward-ticks stuck together (it won't let me paste it), and other times as two back-ticks ``. When I try to paste code over to the rxvt-unicode terminal the forward-tick symbol is displayed as a question mark, as are some other special symbols like maths characters. So I have two questions: are there any terminals that support such special characters? And is my character display problem related to setting up my locales correctly (which I think I already have)?
I followed the steps on the wiki for setting up locales. I uncommented "en_GB.UTF-8 UTF-8" and "en_GB ISO-8859-1" in /etc/locale.gen, ran locale-gen as root, and added LOCALE="en_GB.utf8" to /etc/rc.conf and "export LANG=en_GB.utf8" to my .zshrc. locale -a shows that the correct locales are installed.
Last edited by slyson (2009-06-16 15:20:18)

See this pdf for fixes: unix.derkeiler.com/pdf/Mailing-Lists/Fr … g00564.pdf.
Type 'locale' on the console to find out your locale. Mine is en_GB.utf8.
Now add the following to your shell profile or ~/.xinitrc.
I already had this in my .zshrc:
export LANG=en_GB.utf−8
But I think the LC variable was overriding it. I Changed my LC_ALL to:
export LC_ALL=en_GB.utf8
(It was set to LC_ALL=C before).
The above fixes copy/paste to most terminal apps, but some still misbehave. For example, I had to add this to ~/.emacs:
;; Set language environment for MULE.
(set−language−environment 'UTF−8)
;; My customization for text modes
(defun my−text−mode−hook ()
(auto−fill−mode 1)
(show−paren−mode t)
(activate−input−method 'rfc1345) ; Good input method for UTF−8
(add−hook 'text−mode−hook 'my−text−mode−hook)
More examples are given on page 3 of the pdf.
Last edited by slyson (2009-06-16 15:26:31)

Reports with output to excel - special characters not displayed correctly.

Hi,
I have a portal with data from 5 continents. In many places special or national characters are used in names. My portal can show most of these characters correctly and so can my version of Microsoft Office Excel. When text with these characters are copy/pasted from the screen to an excel sheet, they are also shown correctly in excel. The problem arises in portal reports, where output is set to Excel, then special characters are often displayed as strange double characters, for example á is shown as Ã¡ . Does anybody have a solution to that?
Thanks
Arne

No answers have been posted yet and now I have found a solution myself. The portal report is run and output set to Excel. The resulting file is opened, and a new row is inserted in the top of the sheet. In the first field in this row is written the characters *\xEF\xBB\xBF*. It is called a UTF-8 BOM, and when written in the start of a text file, it will tell the program that opens it, that UTF-8 encoding is used.<br><br>
The file is saved with the file type CSV (semicolon separated) (*.csv) and excel is closed. When you click on the CSV-file, it is opened by excel, and now all characters with accents, circumflexes and umlauts are shown correctly.<br><br>
The solution was found here: http://www.roosmaa.net/importing-utf-8-csvs-in-excel/

Non-English characters not displaying correctly - Serious Issue

Similar Messages

Maybe you are looking for