ASCII Characters Display As ????

hi,
i seem to be having an issue displaying ASCII characters, all i seem to get is black diamonds with white triangles inside, this happens on gnome-terminal, on xterm i get nothing at all. also some fonts do not appear to render in my web browser (chromium or firefox) i seem to get hollow squares instead of text.
$ locale - a
C
en_GB
en_GB.iso88591
en_GB.utf8
POSIX
$ locale
LANG=en_GB.UTF-8
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC=en_GB-UTF-8
LC_TIME=en_GB-UTF-8
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY=en_GB-UTF-8
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER=en_GB-UTF-8
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT=en_GB-UTF-8
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=
# print ascii chars (program prints number followed by ascii conversion)
180 ?
181 ?
182 ?
183 ?
184 ?
185 ?
186 ?
thank you for looking, i would appreciate any help possiable with this.
if you need more info please let me know.

wiggly wrote:if you want to see the C Source code i can show you, alough it was not really relevant except for the fact that it demenstrates my issue.
I know, it's just so that people could 1. test it themselves easily, 2. be sure that everyone is talking about the same thing. (In fact I have written the exact same program after reading your first post.)
wiggly wrote:unless you include the extended characters that run from 128 - 255.
But those are not "real" ASCII — those are various encodings, and the interpretation of those characters depends on which encoding you want to use to read them. They are incompatible with UTF-8.
On my machine (running with i from 32 to 254):
$ ./a.out
180 ´
181 µ
182 ¶
183 ·
184 ¸
185 ¹
186 º
But:
$ ./a.out | less
180 <B4>
181 <B5>
182 <B6>
183 <B7>
184 <B8>
185 <B9>
186 <BA>
$ ./a.out | iconv -t utf-8
128 iconv: illegal input sequence at position 662
$ ./a.out >a; file a
a: data
The question: why do you care about / want to use non-UTF-8 encodings?

Similar Messages

IPod Nano Notescast displays photos as 3 ascii characters

I created a 110 page Notescast and loaded it into Notes on an iPod 2G and 3G. Links within the copy in the Notescast link to 50 different photos, and they reside within a folder titled "Photos" within the Notes folder. Both Nanos recognized the photo files, because the links in the Notescast copy are highlighted indicating all is set up correctly, yet when I click on any of the photo links, the Nano loads and displays a screen with 3 ascii characters in the upper left hand corner.
The photos assigned to the Notescast also reside in the Photos folder located in the main iPod menu screen, (automatically loaded there by the iPod) and these photos display just fine via the slideshow when I'm in that folder. It's just when I try to access the same .jpg photo files in the newly created Photos folder within the Notes folder from the Notescast copy that the photos don't display properly.
The photos are all 50Kb to approx. 100Kb.
All of the above does work fine on iPod Videos and Classics.
Any thoughts?
Thank you.

Have you had a chance to look at this troubleshooting document? It may be of some help:iPod shows up in Windows but not in iTunes

Display extended ascii characters as question mark in xml file

I am creating a XML file with encoding as UTF-8. Some tag values contain some extended ascii characters. When i run the java program to create the file in windows, the extended ascii characters are display correctly. But in linux it is displaying as ?(question mark).
i am not able to rectify this. can anyone help me....
Its urgent
Thanks in advance.
Message was edited by:
Rosy_Thomas@Java

Probably the locale is not set for the shell you are running in. The default 'C' locale uses the ASCII encoding which defines only 128 characters. See if giving the commandexport LC_CTYPE=en_US.UTF-8before starting the program fixes the issue.

Cannot view OWA email - after upgrade to Firefox 5.0, Outlook Web Access displays ASCII characters (garbage) in message body. Same issue with multiple servers.

Email body of HTML or RTF messages are rendered in ASCII characters after upgrading to Firefox 5 when viewing email in MS Outlook Web Access (OWA) light from MS Exchange 2007 servers. Issue is repeatable with two entirely different Exchange systems.
Text email renders OK. Work around is to forward email (in use MS-IE).

You are welcome. I'm glad you got it back up.
(1) You say you did the symbolic link. I will assume this is set correctly; it's very important that it is.
(2) I don't know what you mean by "Been feeding the [email protected] for several weeks now, 700 emails each day at least." After the initial training period, SpamAssassin doesn't learn from mail it has already processed correctly. At this point, you only need to teach SpamAssassin when it is wrong. [email protected] should only be getting spam that is being passed as clean. Likewise, [email protected] should only be getting legitimate mail that is being flagged as junk. You are redirecting mail to both [email protected] and [email protected] ... right? SpamAssassin needs both.
(3) Next, as I said before, you need to implement those "Frontline spam defense for Mac OS X Server." Once you have that done and issue "postfix reload" you can look at your SMTP log in Server Admin and watch as Postfix blocks one piece of junk mail after another. It's kind of cool.
(4) Add some SARE rules:
Visit http://www.rulesemporium.com/rules.htm and download the following rules:
70sareadult.cf
70saregenlsubj0.cf
70sareheader0.cf
70sarehtml0.cf
70sareobfu0.cf
70sareoem.cf
70sarespoof.cf
70sarestocks.cf
70sareunsub.cf
72sare_redirectpost
Visit http://www.rulesemporium.com/other-rules.htm and download the following rules:
backhair.cf
bogus-virus-warnings.cf
chickenpox.cf
weeds.cf
Copy these rules to /etc/mail/spamassassin/
Then stop and restart mail services.
There are other things you can do, and you'll find differing opinions about such things. In general, I think implementing the "Frontline spam defense for Mac OS X Server" and adding the SARE rules will help a lot. Good luck!

Email can not display Extended Ascii Characters

Hi every One,
In my Project there is a One option in that i can send Product Information as well as Customer Address to specify Email id , In Customer Address Have some Ascii Characters .After Sending a Mail when i Check Email that Ascii Character replace by ? .
I am trying that one , still i dont know where is Going wrong, Please any one Know Regading this Problem let me Know
with Regards
Asif

I sure hope you get that sorted out. sounds annoying
right, back to the java.....

Convert smart quotes and other high ascii characters to HTML

I'd like to set up Dreamweaver CS4 Mac to automatically convert smart quotes and other high ASCII characters (m-dashes, accent marks, etc.) pasted from MS Word into HTML code. Dreamweaver 8 used to do this by default, but I can't find a way to set up a similar auto-conversion in CS 4. Is this possible? If not, it really should be a preference option. I code a lot of HTML emails and it is very time consuming to convert every curly quote and dash.
Thanks,
Robert
Digital Arts

I too am having a related problem with Dreamweaver CS5 (running under Windows XP), having just upgraded from CS4 (which works fine for me) this week.
In my case, I like to convert to typographic quotes etc. in my text editor, where I can use macros I've written to speed the conversion process. So my preferred method is to key in typographic letters & symbols by hand (using ALT + ASCII key codes typed in on the numeric keypad) in my text editor, and then I copy and paste my *plain* ASCII text (no formatting other than line feeds & carriage returns) into DW's DESIGN view. DW displays my high-ASCII characters just fine in DESIGN view, and writes the proper HTML code for the character into the source code (which is where I mostly work in DW).
I've been doing it this way for years (first with GoLive, and then with DW CS4) and never encountered any problems until this week, when I upgraded to DW CS5.
But the problem I'm having may be somewhat different than what others have complained of here.
In my case, some high-ASCII (above 128) characters convert to HTML just fine, while others do not.
E.g., en and em dashes in my cut-and-paste text show as such in DESIGN mode, and the right entries
    –
    —
turn up in the source code. Same is true for the ampersand
    &
and the copyright symbol
    ©
and for such foreign letters as the e with acute accent (ALT+0233)
    é
What does NOT display or code correctly are the typographic quotes. E.g., when I paste in (or special paste; it doesn't seem to make any difference which I use for this) text with typographic double quotes (ALT+0147 for open quote mark and ALT+0148 for close quote mark), which should appear in source code as
    “[...]”
DW strips out the ASCII encoding, displaying the inch marks in DESIGN mode, and putting this
    "[...]"
in my source code.
The typographic apostrophe (ALT+0146) is treated differently still. The text I copy & paste into DW should appear as
    [...]’[...]
in the source code, but instead I get the foot mark (both in DESIGN and CODE views):
I've tried adjusting the various DW settings for "encoding"
    MODIFY > PAGE PROPERTIES > TITLE/ENCODING > Encoding:
and for fonts
    EDIT > PREFERENCES > FONTS
but switching from "Unicode (UTF-8)" to "Western European" hasn't solved the problem (probably because in my case many of the higher ASCII characters convert just fine). So I don't think it's the encoding scheme I use that's the problem.
Whatever the problem is, it's caused me enough headaches and time lost troubleshooting that I'm planning to revert to CS4 as soon as I post this.
Deborah

Replacing non-ASCII characters with HTML charcter references

Hi All,
In Oracle 10g or greater is there a built-in function that will convert a string with non-ASCII characters like this
a b č 뮼
into an ASCII string with HTML character references like this?
a b & # x 0 1 0 D ; & # x B B B C ;
(note I had to include spaces between each character in the sample code for message to prevent the forum software from converting my text)
I tried using
utl_i18n.escape_reference( val, 'us7ascii' )
but for some reason it returns
a b c & # x B B B C ;
Note how it converted the Western European character "č" to its unaccented counterpart "c", not "& # x 0 1 0 D ;" (is this a bug?).
I also tried a custom solution using regexp_replace and asciistr (which I can't include here because the forum software chokes on it) but it only returns the correct result for values <=4000 characters long. Unfortunately asciistr doesn't appear to accept CLOB values larger than 4000 characters. It returns an error message like
(ORA-22835: Buffer too small for CLOB to CHAR or BLOB to RAW conversion (actual: 30251, maximum: 4000) ).
I'm looking for a solution that works on CLOB data of any size.
Thanks in advance for any insight you can provide.
Joe Fuda

So with that (UTF8) in mind, let's take another look.....
As shown below, I used a AL32UTF8 database.
Note: I did not use a unicode capable tool for querying. So I set console mode code page to 1250 just to have č displayed properly (instead of posing as an è).
Also, as a result of using windows-1250 for client character set, in the val column and in the second select's ncr column (iso8859-1), è (00e8) has been replaced with e through character set conversion going from server back to client.
Running the same code on a database with a db character set such as we8mswin1252, that doesn't define the č (latin small c with caron) character, would yield results with a c in the ncr column.
C:\>chcp 1250
Aktuell teckentabell: 1250
C:\>set nls_lang=.ee8mswin1250
C:\>sqlplus test/test
SQL*Plus: Release 11.1.0.6.0 - Production on Fri May 23 21:25:29 2008
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the OLAP option
SQL> select * from nls_database_parameters where parameter like '%CHARACTERSET';
PARAMETER              VALUE
NLS_CHARACTERSET       AL32UTF8
NLS_NCHAR_CHARACTERSET AL16UTF16
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'us7ascii') NCR from dual;
VAL NCR
č e c e
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'we8iso8859p1') NCR from dual;
VAL NCR
č e &# x10d; e     <- "è"
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'ee8iso8859p2') NCR from dual;
VAL NCR
č e č &# xe8;
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'cl8iso8859p5') NCR from dual;
VAL NCR
č e &# x10d; &# xe8;In the US7ASCII case, where it should be possible for all non-ascii characters to be escaped, it seems as if the actual escape step is skipped over.
Hope this helps to understand whether utl_i8n is usable or not in your case.
Message was edited by:
orafad
Fixed replaced character references :)

Non US-ASCII characters in download file names

I am trying to implement a simple file download in a JSP, and trying to get IE, Firefox and Opera to all display and handle non US-ASCII characters in the suggested download file name. Only concerned with Windows platform for now. Here's the code I am currently using:
String agent = request.getHeader("USER-AGENT");
if (null != agent && -1 != agent.indexOf("MSIE"))
String codedfilename = URLEncoder.encode(cfrfilename, "UTF8");
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + codedfilename);
else if (null != agent && -1 != agent.indexOf("Mozilla"))
String codedfilename = MimeUtility.encodeText(cfrfilename, "UTF8", "B");
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + codedfilename);
else
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + cfrfilename);
}This URL encodes the file name if the browser is IE, MIME encodes it if the browser is Mozilla, and sends plain UTF-8 (the encoding of the JSP) for all other browsers. I get "cfrfilename" from translated properties files, and the string can contain characters from any character set - Chinese, Thai, Korean, etc.
This code works correctly for IE - the file name is displayed correctly in the file Save as dialog, and it is saved correctly on disk, no matter which character set is used.
For Firefox, the file name is displayed correctly in the file Save as dialog, but it is only saved correctly to disk if the file name is in a character set supported by the system locale. This seems to be a known Firefox bug (not fully using the Windows Unicode APIs), so nothing I can do about that.
Nothing seems to work for Opera, however - I cannot get the file name to display correctly in the file Save as dialog, no matter which method I use (I have tried URL encoding and MIME encoding in addition to the plain UTF-8).
Has anybody implemented something similar that works for at least these 3 browsers?

I tested your code today,
                     dialog           save           open
Firefox 1.5          OK                 OK               OK
IE 6.0                OK                 OK                NGdailog: filename show in download popup dialog
save: save to disk from dialog
open: open directly from dailog

Cannot rename file with non-ASCII characters when using the

My application moves files from one directory to another by calling File[] srcFiles = srcDir.listFiles() to get a list of files in the source directory, and then calling srcFiles.renameTo(destFile) to rename each file.
This does not work (renameTo returns false and the file is not moved) under the following circumstances:
- the file's leaf name contains non-ASCII characters, for example "�"
- the OS is Solaris 9
- the LANG and LC_* environment variables are unset, i.e. the C locale is being used
If I set the LANG environment variable to, for example, en_GB.UTF-8 then the rename succeeds.
I have tried calling srcFiles[index].getName().getBytes("UTF-8") and the non-ASCII characters are being replaced with ? (0x3f) characters when LANG is unset.
Is this a bug in the JRE? I would argue that since my code does not actually manipulate the filename (I just use the File object that File.listFiles() gives me) then the rename should succeed. Of course I would not expect the file name to be displayed correctly if I printed it out.
I have reproduced this behaviour with JDK 1.4.2_05 and 1.5.0_04 on Solaris 9.
Francis

Thanks for the info Alan.
I considered setting the locale in the environment (this sounds like the "correct" fix to me and we might implement it later), but this application shares a WebLogic server with many other applications so we would have to do a huge amount of testing to make sure that the locale change wouldn't break the other apps. In the end I worked around the problem by making the code that generates the filenames in the first place strip out any non-ASCII characters (the names of the files are not critically important).
Looking forward to JSR-203, in the meantime perhaps a note about this behaviour in the java.io.File javadoc would be useful.

Validation for non-ASCII characters

Hi all,
Requirement: I have to apply a validation on on fields like Name and Address in applicationdefination.xml. When a user types non-ASCII characters and navigates to next page then it should display the error message. Thus, I have to restrict my user to ASCII values only.
Present Situation: I'm using regular expression for this problem. In Jheadstart there is an option regular expression under the heading Validation. I have written following values in regular expression and Regular Expression Error Message options.
Regular Expression
^\s*[\w\.\,\-\_\#\'\/\\\ u0022\u0026\*\;\:\s]+\s*$
Regular Expression Error Message
It is important to note that foreign characters are not accepted on our system. Please ensure only standard English letters are entered
Since, i was getting error in jspx page due to double quotes(") and ampercent(&), So i have replaced the double quotes(") and amprecent(&) by their unicodes. Thus, the expression has become like ^\s*[\w\.\,\-\_\#\'\/\\\u0022\u0026\*\;\:\s]+\s*$.
This expression is validating many characters like Ã,µ,Ç,Ï,Ö,§,¥,{,} but not all non ASCII characters like ѓ є ѕ ї Њ Щ Ώ Ω Ϊ Ά Ή Θ Λ Ξ Π τ ẫ ờ Ỡ Ứ Ỷ ự Ẁ ỹ ị Ọ ň ũ ť ţ Έ Ϊ ﻍ. Thus, its not fulfilling the requirement.
Please suggest some valid solution to this problem. It’s very urgent.

Hi,
The validation seems to be performed in Java or Javascript depending on the layout (I'm sorry I can't remember the exact details). The expression suggested above by theEternalStudent works very well in Java, but not in Javascript.
We came up with an expression which works in both. It rejects strings which contain &# by doing a lookahead before the main pattern - you might want to expand this to look for &#nnn; but for our purposes &# is enough.
Here is the "platform neutral" solution:
(?!.*\u0026#.*)^[\w\.\,\-\_\#\'\/\\\u0022\u0026\*\;\:\s]+$
I think in future we will write a javascript function and amend the templates to call it directly.
thanks,
Michael

Normal string ASCII characters are not shown after using string reversal.

In a VI that I have written, there is a point where I take a simple 5 character string and input it to a reverse string node. I also have an indicator on the output of the reverse string node so I can look at the reversed string. Both the input and output indicators are in normal display mode. For the input indicator I can see the input string with no problem. But on the indicator at the output of the reverse string node I see nothing. If I then put each of the indicators in HEX display mode, both indicators display the hex equivalent characters of the original ASCII characters, and the output indicator shows the characters reversed from that of the input indicator, as expected. So why when I am in normal display mode, are the characters in the output indicator not showing up? Thanks.

What are the ASCII values that you are trying to display? Perhaps they are non-printable ASCII values.
Perhaps you can post a simple VI with the values in a string control and saved as a default.
EDIT: After seeing Rod's message, that seems very likely, especially considering that there is likely an EOL character if you are doing serial communication with some other device.
Message Edited by Ravens Fan on 03-23-2009 12:04 PM

Question on ascii characters and bytes

Is there a difference between an 8 bit ascii character and a byte.
I need to display characters to an LCD display - the chip on the display will accept 8 bit ascii characters, but I need to send them to the serial port in a byte array, does this mean they are not in 8 bit ascii format when they reach the chip on the display.
If they are different is it possible to send the ascii characters to the serial port as actual ascii characters.
Cheers David

you should be just able to create a byte array, fill the array with downcasted char's and then it off to the serial port via the outputstream.
byte[] tmp = new byte[2];
tmp[0] = (byte)'H';
tmp[1] = (byte)'I';
os = serailport.getOutputStream();
os.write(tmp);
(i think that's it)
an ascii character is just a particular representation of a byte string. it doesnt really mean anything. so you could rewrite the above using the hex/decimal values from the ascii char set.
a winn

Why does non-ASCII text display improperly?

One of the things that has long baffled me about OS X is the occasionally improper display of text on web sites. Sometimes, though less than before, the Mac still can't properly diplay non-ASCII characters. Today, for instance, I bought a GPS from Amazon, and the word nüvi has junk characters where the umlaut "ü" should be, as the text image below should show. Why is this? Is there a setting that corrects the problem?

Hi Yawder, do you want to file a bug report on the problem that when Firefox generates the faux bold face for Droid Sans Mono it is doing a bad job compared with other browsers?
You can submit that here: https://bugzilla.mozilla.org/

Username with ascii characters

Hello, i'm having and html form and i would like the user in
the username field to type ONLY ascii characters.
For example, in other fields of the form i
would like the user to type his mother language but
as far as the username and password fields are concerned
the characters have to be ascii.
How am i supposed to check when the username is accepted/correct (*consists of ascii characters*)?
and which are the desirable characters a username must have (e.g. *?* is a desirable character in a username , *:* this one?)
Thanks, in advance!

g_p_java wrote:
How am i supposed to check when the username is accepted/correct (*consists of ascii characters*)?ASCII characters are the Unicode characters whose code points are between 0 and 127.
and which are the desirable characters a username must have (e.g. *?* is a desirable character in a username , *:* this one?)I don't understand this. You have already said they must be ASCII. You have other requirements? Fine, go ahead and program them and ask questions if you have problems with that. Personally I don't think that requiring somebody to have a question mark in their user name is a good idea -- but probably you didn't mean it when you suggested that.

Non ascii characters being sent from a parameter in a form

Hi!
I have seen many topics posted on passing non ascii characters through parameters from one servlet to another and converting them into whatever format is necessary.
However, I have not seen anyone answer the following question. I have a jsp page (html) with the character encoding set to utf-8. The user inputs some data in to a text field which is inside a form. The data could be in non ascii characters such as hebrew or arabic. This form is then sent to another jsp where i try to retreive the data from teh text field. No matter what i do, i cannot get the data presented correctly. It is either question marks or other wierd symbols.
I have tried every permetation of encoding of the actual html page, the ecoding of the string from request.getParameter etc but it still is not presented on the new html page correctly.
Can anyone help??
Spencer

Ok, I solved the problem.
I had to put at the top request.setCharacterEncoding("utf-8");
Spencer

ASCII Characters Display As ????

Similar Messages

Maybe you are looking for