Non-ASCII charatcers converted

I am working for a section that is one small part of the user base for Oracle in my company. Another section is inserting strings with the letter u with the Germanic umlaut. The problem is that the DB settings don't allow that character, or presumably, any other character unusual to the English language. Instead, it converts that character to the letter c and a closing bracket 'c]'. We can't change whatever setting limits the insert. What we would like ot know is if we can simply find the 'c]' and replace it with the u-with-the-umlaut again. Is this a reliable conversion, or will it occasionally convert otherwise? Thanks.

You can change the settings if you want. Check what the Database Character Set is:
select parameter,value from nls_database_parameters;
Check what the PC (client) they are using to insert the data is using
From the Registry (HKEY LOCAL MACINE -> SOFTWARE -> ORACLE) , look for NLS_LANG.
Ensure you have the correct Character Set for Germany GERMAN_GERMANY.WE8MSWIN1252
If your Database Server Charater set already supports German, then you will be lucky and they will only need to check the Client. Else, you may have to do more than what I have just mentioned above.
If your Application support Unicode, use UTF8.

Similar Messages

Non ASCII characters are converted to '?' or ASCII characters

Non ASCII symbols like æ ø in the xml file have been converted to ? or other ascii characters.
What could be the reason behind this.

Mayil wrote:
This file we are loading through the Flex application in the front end.
Through java class file we are making changes to this city.xml file and adding and deleting this information in the city.xml.
Now suddenly, i dont know what happen.. 'ø' in the city name has replaced with the '?'
If we try to chaange this to 'ø' also, it again changes to '?'.
I dont know how to rectify this error.I would suggest you start by finding out when it happens. Does it happen as soon as you change the XML through this mysterious "java class file"? Or does it happen when Flex reads it? And is the underlying file actually changing, or are you just seeing those question marks after Flex handles the file?
In short a much better problem description is necessary.

Non ascii characters being sent from a parameter in a form

Hi!
I have seen many topics posted on passing non ascii characters through parameters from one servlet to another and converting them into whatever format is necessary.
However, I have not seen anyone answer the following question. I have a jsp page (html) with the character encoding set to utf-8. The user inputs some data in to a text field which is inside a form. The data could be in non ascii characters such as hebrew or arabic. This form is then sent to another jsp where i try to retreive the data from teh text field. No matter what i do, i cannot get the data presented correctly. It is either question marks or other wierd symbols.
I have tried every permetation of encoding of the actual html page, the ecoding of the string from request.getParameter etc but it still is not presented on the new html page correctly.
Can anyone help??
Spencer

Ok, I solved the problem.
I had to put at the top request.setCharacterEncoding("utf-8");
Spencer

Replacing non-ASCII characters with HTML charcter references

Hi All,
In Oracle 10g or greater is there a built-in function that will convert a string with non-ASCII characters like this
a b č 뮼
into an ASCII string with HTML character references like this?
a b & # x 0 1 0 D ; & # x B B B C ;
(note I had to include spaces between each character in the sample code for message to prevent the forum software from converting my text)
I tried using
utl_i18n.escape_reference( val, 'us7ascii' )
but for some reason it returns
a b c & # x B B B C ;
Note how it converted the Western European character "č" to its unaccented counterpart "c", not "& # x 0 1 0 D ;" (is this a bug?).
I also tried a custom solution using regexp_replace and asciistr (which I can't include here because the forum software chokes on it) but it only returns the correct result for values <=4000 characters long. Unfortunately asciistr doesn't appear to accept CLOB values larger than 4000 characters. It returns an error message like
(ORA-22835: Buffer too small for CLOB to CHAR or BLOB to RAW conversion (actual: 30251, maximum: 4000) ).
I'm looking for a solution that works on CLOB data of any size.
Thanks in advance for any insight you can provide.
Joe Fuda

So with that (UTF8) in mind, let's take another look.....
As shown below, I used a AL32UTF8 database.
Note: I did not use a unicode capable tool for querying. So I set console mode code page to 1250 just to have č displayed properly (instead of posing as an è).
Also, as a result of using windows-1250 for client character set, in the val column and in the second select's ncr column (iso8859-1), è (00e8) has been replaced with e through character set conversion going from server back to client.
Running the same code on a database with a db character set such as we8mswin1252, that doesn't define the č (latin small c with caron) character, would yield results with a c in the ncr column.
C:\>chcp 1250
Aktuell teckentabell: 1250
C:\>set nls_lang=.ee8mswin1250
C:\>sqlplus test/test
SQL*Plus: Release 11.1.0.6.0 - Production on Fri May 23 21:25:29 2008
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the OLAP option
SQL> select * from nls_database_parameters where parameter like '%CHARACTERSET';
PARAMETER              VALUE
NLS_CHARACTERSET       AL32UTF8
NLS_NCHAR_CHARACTERSET AL16UTF16
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'us7ascii') NCR from dual;
VAL NCR
č e c e
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'we8iso8859p1') NCR from dual;
VAL NCR
č e &# x10d; e     <- "è"
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'ee8iso8859p2') NCR from dual;
VAL NCR
č e č &# xe8;
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'cl8iso8859p5') NCR from dual;
VAL NCR
č e &# x10d; &# xe8;In the US7ASCII case, where it should be possible for all non-ascii characters to be escaped, it seems as if the actual escape step is skipped over.
Hope this helps to understand whether utl_i8n is usable or not in your case.
Message was edited by:
orafad
Fixed replaced character references :)

Non-ASCII character in Email field

Hi Guys,
I am trying to enter non-english characters in Email field of user form, but OIM throws an error that "A non-Ascii character has been entered". I have also tried to turn off the AppFirewall Filter in xlConfig.xml file but no help. Is there any way thay I can enter non-Ascii characters in Email field?
Regards,
Rahul

.oO(surfinIan)
>I have a script that converts a ms word document to text
then uploads that to a
>blob field on a mysql db.
> During the conversion some characters my not be
recognised. When i then call
>up the blob for display on the browser...those characters
show up as unknown
>characters with a ? or box. Is there a way to
preg_replace those unknown
>characters before displaying them.
What about fixing the encoding problem instead? If chars get
lost during
such a transfer
document->script->database->script->browser it's always
an encoding problem somewhere down the road.
The recommendation these days is to use UTF-8, which avoids
most of
these old problems. You just have to make sure that your
documents are
properly stored as UTF-8 in the database and delivered as
such to the
script and the browser, then you don't have to worry about
special chars
anymore.
That's just the general idea. I can't be more specific, since
I don't
know your conversion script or the database structure.
Micha

Non-ASCII Characters in AppleWorks

New to the forums and hope I'm not duplicating a prior post ... eons back, I used ClarisWorks on an old Mac SE/30, then moved over to AppleWorks which I got to use on my PC because I wanted to go back and work on something I had from years ago. The document in question contained non-ASCII characters. Anyway, I imported it into AppleWorks and what was once a Cyrillic font, was converted to ASCII (now a bunch of gibberish). I fully recognize I need to find the original font I used for the Cyrillic, but before I go that route, it looks like AppleWorks doesn't support non-ASCII characters ... ergo, I'm wasting my time. Am I wrong?
Thanks.
Windows 2.8 GHz Windows XP Pro

You may need to embed the font:
http://livedocs.adobe.com/flex/3/html/help.html?content=fonts_04.html
If this post answers your question or helps, please mark it as such.

Java / Linux - why ??? instead of non-ascii symbols

Hi All,
I've run into the problem during creating a new file from Java program on Linux gentoo.
All non-ascii characters in the file name are converted into ??? symbols.
I've tried to run the program with -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8, and set LC_ALL='UTF-8' - but the problem still exists.
At the same time I can create the files with non-ascii characters in the file manager.
Could you advise how can I fix this issue?
Regards,
Boris

Is it related to File name or File content ?
If it is file name related then Linux might have restriction on File name character !!.
About File Content : Make one File reader program using Java and check are you able to read it back. if yes then It will be fault of your FileManager which is not correctly interpret content of your file.

Displaying non-ascii numerals (digits) in native language

Hello
I have been looking through the forums and docs on OTN and haven't been able to determine whether or not Oracle has builtin support for converting numerals.
I know that is has support for Date and Time formats, Monetary and Numeric formats (position of decimal and comma), and Calender formats. Also, according to the NLS FAQ...
What does the language part of NLS_LANG control?
The languages used for Oracle messages, of day and month names etc.
...Oracle can even automatically translate messages and day/month names. My question is, can it also automatically translate numerals. If my database character set is set to UTF-8 and I insert the number 1285 into a NUMBER datatype column, can I pull that number out in a different language than what it was inserted as? If a user of my site is using a Thai locale for instance, I want him to see digits 0-9 not as 0123456789, but digits in Thai language.
If it doesn't support that, then how do multi-language applications deal with numerals? Do they always store numerals in char or varchar2 datatypes? If non-ascii numerals are stored as char/varchar2, how are mathematical equations done? For instance, how would a multi-language calculator work?
I'd appreciate any insight that can be offered.
Thanks
-Matt

There is some support for numeric month conversion for Asian languages but not for Thai as far as I know.
Arabic, Thai, and Indic languages have classical shapes for numbers that are different from the conventional western digits most often used on computers.
Unicode provides separate code points for each digit shape. So it would be straight forward to substitute a Thai number for numbers 0-9. Displaying these with the appropriate shape is subject to the available fonts and usually means the front end GUI must be able to handle Unicode.

Security error: Cannot authorize operation for invalid non-ASCII URL

I have an error that popped up with Flash Player 10.1.85.3 or 10.1.82.76.
In our application, we load users' avatars with their original filename. Some filenames have non-ASCII characters (i.e. Hebrew, accents, or umlauts) and I can no longer load those files from a different subdomain.
For example, I'm trying to load this image (with the Loader class and a LoaderContext):
http://photos.myawesomedomain.com/images/awesöme.jpg (o with umlaut)
It comes to me encoded in UTF-8:
http://photos.myawesomedomain.com/images/awes%C3%B6me.jpg (umlaut converted to %C3%B6)
When I try to load the file from http://myawesomedomain.com/myawesome.swf, I get this error:
*** Security Sandbox Violation ***
Connection to http://photos.myawesomedomain.com/images/awesÃ¶me.jpg halted - not permitted from http://myawesomedomain.com/myawesome.swf
Error: Cannot authorize operation for invalid non-ASCII URL http://photos.myawesomedomain.com/images/awesÃ¶me.jpg
If I try this with a player earlier than 10.1.82.76, it works. It also works if I move the file to the same domain as the SWF (http://myawesomedomain.com/images/awesöme.jpg) -- but that's not an option for me. Files without unusual characters work regardless of the player version.
The error occurs when I try to access the bitmap data of the loaded image. I've tried encoding the URL differently, but Flash always reports it as "http://photos.myawesomedomain.com/images/awesÃ¶me.jpg" - with the umlaut converted to strange characters. The cross domain file allow-access-from "*.myawesomedomain.com"
Has anyone run into this? Is there a way I can fix it without renaming the users' photos (we have tens of thousands of these)?
I feel like I must be missing something obvious; hardly anything comes up in Google for this, but I don't think it would be an uncommon problem. I believe https://bugs.adobe.com/jira/browse/FP-5580 is related.
Thanks!

The problem is based on the player version, not the browser.
The problem showed up in Chrome first because it auto-updates the player; when Firefox users installed the updated player, they started having the problem, too.

Non ASCII String to Hex

Hi,
I am writing an algorithm that first encrypts a String, and then converts that String to Hex String. I can successfully parse Hex Strings into Ascii Strings and vice versa, but when I encrypt I cannot do this anymore. Is there a way to convert a non-ascii character to Hexadecimal and vice versa?
Thanks in advance

Maybe so....
The encrypted value when printed does not format on my screen, and appears as boxes and such.
The values that I am passing in are a very simple encryption....XOR against a key. Here is that algorithm.
public byte[] encrypt(byte[] value) {
          byte[] encrypted = new byte[value.length];
          byte[] adjustedKey = getKey();
          while (adjustedKey.length < encrypted.length) {
               String temp = new String(adjustedKey) + new String(key);
               adjustedKey = temp.getBytes();
          for (int i = 0; i < value.length; i++) {
               encrypted[i] = (byte)(adjustedKey[i] ^ value);
          return encrypted;
Since this is an XOR, I can use the same method to decrypt as well. This is what produces my non-Ascii characters.
Is there a way for me to get the byte value of a 1 digit hex value then? this may help some.

Non-ASCII chars in applets?

hi,
Spent 4 hours to find a way to use non-ASCII chars in applets (buttons, textareas), but didn't make it.
Simply saying
TextFieldObj.setText("\uxxxx");
//or any equivalent obj. Ex. of \uxxxx: \u015F
doesn't work. I even went into Graphics.paint() example, but it too can paint only ASCII chars.
My hunch is that it is smt. about Character.Subset but i still can't figure out how to do it.
Please SOS,
Reshat.

Hi,
I just managed to get Buttons to show Greek characters, so it appears that static buttons are fine.
However, i still face the same problem for TextField's:
TextFields work fine for IE, but in NN they sometimes convert into ASCII and sometimes give ? The same in HotJava.
So there are 2 questions in my head:
1. why can't NN use the fonts used by IE to display Non-ASCII chars?
2. What is the safest font to use for Non-ASCII chars, to cover the widest possible audience.
P.S. Java solves most cross-platform-browser problems, but the font issue still seems to be dependent on a user and his/her browser. It appears Java is not font-independent in non-ASCII context. If so, it would be nice to develop a plug-in to make sure that if the user doesn't have the font, then a Java-standardized Unicode-based font is used. Otherwise, non-ASCII world is still w/o a real Java.)
Thank you for your feedback,
Reshat.

Sql to remove brackets non ascii charaters

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
Hi there,
I have a requirement to remove the any non alphabets like brackets, , quotes and non ascii, languages other than english and everything in lower case. The select is simple
select
lower(customer_name)
from
customer_data
where CONVERT ( customer_name, 'US7ASCII')     = customer_nameHowever, I also want to add maybe the replace function on the customer name, such that the brackets, ", ', / - or any other such character can be removed. I tried unsuccessfully something like
{code}
select replace('cvp, llc','''-/,,".') from dual
I wanted to do something on those lines on my first query so that I only get english alphabets customer names, stripping off all the non english charaters and any non-english characters.
Could you please advise?
Thanks,
Ryan
Edited by: ryansun on Nov 15, 2012 6:09 AM
Edited by: ryansun on Nov 15, 2012 6:10 AM

Hi Chris,
if you want to use TRANSLATE then you have to modify it a bit:
WITH customer_data AS
   SELECT 'cvp, llc' AS customer_name FROM DUAL
SELECT translate(customer_name,'abcdefghijklmnopqurstuvwxyzABCDEFGHIJKLMNOPQURSTUVWXYZ1234567890'
                ,'abcdefghijklmnopqurstuvwxyzABCDEFGHIJKLMNOPQURSTUVWXYZ1234567890') txt
FROM customer_data;
TXT
cvp, llc
WITH customer_data AS
   SELECT 'cvp, llc' AS customer_name FROM DUAL
WITH customer_data AS
   SELECT 'cvp, llc' AS customer_name FROM DUAL
SELECT translate(customer_name,' '
                 ||'[''-/,".]' -- chars to remove here
                ,' ') txt
FROM customer_data;
TXT
cvp llc
{code}
Regards.
Al
Edited by: Alberto Faenza on Nov 15, 2012 3:52 PM
Modified query

Can't get the attachment filename out of a Part (with non ascii characters)

Hello, all and happy new year :)
My issue is with non ascii filename in attachments... Yes i've read the FAQ : http://www.oracle.com/technetwork/java/faq-135477.html#encodefilename
I can't get the filename out of the BodyPart for those kind of attachments
here's my unit test :
     * contains various parts from various mailer encoded in different ways...
     private enum EncodedFileNamePart{
          OUTLOOK("Content-Type: text/plain;\n name=\"=?iso-8859-1?Q?c'estd=E9j=E0no=EBl=E7ac'estcool.txt?=\" \nContent-Transfer-Encoding: 7bit\nContent-Disposition: attachment;\n filename=\"=?iso-8859-1?Q?c'estd=E9j=E0no=EBl=E7ac'estcool.txt?=\" \n\nnoel 2010\n","c'estdéjànoëlçac'estcool.txt"),
          GMAIL("Content-Type: text/plain; charset=US-ASCII; name=\"=?ISO-8859-1?B?ZOlq4G5v62znYWNlc3Rjb29sLnR4dA==?=\"\nContent-Disposition: attachment; filename=\"=?ISO-8859-1?B?ZOlq4G5v62znYWNlc3Rjb29sLnR4dA==?=\"\nContent-Transfer-Encoding: base64\nX-Attachment-Id: f_giityr5r0\n\namluZ2xlIGJlbGxzIQo=\n","déjànoëlçacestcool.txt"),
          THUNDERBIRD("Content-Type: text/plain;\n name=\"=?ISO-8859-1?Q?d=E9j=E0no=EBl=E7acestcool=2Etxt?=\"\nContent-Transfer-Encoding: 7bit\nContent-Disposition: attachment;\n filename*0*=ISO-8859-1''%64%E9%6A%E0%6E%6F%EB%6C%E7%61%63%65%73%74%63%6F;\n filename*1*=%6F%6C%2E%74%78%74\n\njingle bells!\n","déjànoëlçacestcool.txt"),
          EVOLUTION("Content-Disposition: attachment; filename*=ISO-8859-1''d%E9j%E0no%EBl.txt\nContent-Type: text/plain; name*=ISO-8859-1''d%E9j%E0no%EBl.txt; charset=\"UTF-8\" \nContent-Transfer-Encoding: 7bit\n\njingle bells\n","déjànoël.txt"),
          String content=null;
          String target=null;
          private EncodedFileNamePart(String content,String target){
               this.content=content;
               this.target=target;
          public Part get(){
               try{
               ByteArrayInputStream bis = new ByteArrayInputStream(this.content.getBytes());
               Part part = new MimeBodyPart(bis);
               bis.close();
               return part;
               catch(Throwable e){
                    return null;
          public String getTarget(){
               return this.target;
     @Test
     public void testJavamailDecode() throws MessagingException, UnsupportedEncodingException{
          System.setProperty("mail.mime.encodefilename", "true");
          System.setProperty("mail.mime.decodefilename", "true");
          for(EncodedFileNamePart part : EncodedFileNamePart.values())
               assertEquals(part.name(),MimeUtility.decodeText(part.get().getFileName()),part.getTarget());
I take a NullPointerExcepion in the decodeText because getFileName() return null for the EVOLUTION case, and work well with OUTLOOK, THUNDERBIRD and GMAIL.
Evolution's content type is "Content-Disposition: attachment; filename*=ISO-8859-1''d%E9j%E0no%EBl.txt" wich doesn't look like the other (looks like the RFC 2616 or RFC5987 to do it.)
How can i handle this situation except by writting my own decoder?
Thanks for your answers!
Edited by: user13619058 on 4 janv. 2011 07:44

Set the System property "mail.mime.decodeparameters" to "true" to enable the RFC 2231 support.
See the javadocs for the javax.mail.internet package for the list of properties.
Yes, the FAQ entry should contain those details as well.

Unable to play videos with non-ASCII-characters in filename

Hi!
I use a MediaPlayer to display MP4-videos in my application. This works quite well. Unfortunately I have a problem if the filename of the video to be shown contains non-ASCII-charcaters.
I get the following message:
-->file:D:\daten\avi\�� .MPG
Error: Unable to realize com.sun.media.amovie.AMController@4b7651
Failed to realizeThe first line shows the filename I pass to the setMediaLocation()-method of the MediaPlayer-object.
What's wrong? If I rename the file to ABC.mpg it works fine.
Thanks for your help
Thomas

Hi!
I use a MediaPlayer to display MP4-videos in my application. This works quite well. Unfortunately I have a problem if the filename of the video to be shown contains non-ASCII-charcaters.
I get the following message:
-->file:D:\daten\avi\�� .MPG
Error: Unable to realize com.sun.media.amovie.AMController@4b7651
Failed to realizeThe first line shows the filename I pass to the setMediaLocation()-method of the MediaPlayer-object.
What's wrong? If I rename the file to ABC.mpg it works fine.
Thanks for your help
Thomas

Non-ascii charsets in an applet woes

I'm having trouble getting russian, chinese, japanese, and greek charactersets to display correctly in an applet (plugin is jre 1.4.2_02).
1) yes the strings are in i18n'ized properties
2) yes the non-ascii characters are unicode escaped (i.e. \u####) in the properties file
3) yes the computer displaying the applet has its locale and languages set to the correct language
4) yes the computer displaying the applet has the fonts available to display
All the above is done and works great for ascii or extended ascii sets, but it shows junk for the 4 languages listed above.
Anyone run into this problem before?
Please help me out.
Thanks,
Andrew

Ok, I got Russian and Greek to work. I had to switch all my awt controls to swing.
However, I cannot get the Chinese/Japanese characters to display. I double and triple checked the encodings to make sure it was correct (it is).
However, the font.properties.ja (japanese) lists a font called MSMINCHO.TTC and one called MSGOTHIC.TTC. Both are actually in TTF files on my system. I tried correcting the font.properties.ja with TTF; but to no avail.
Also, font.properties.zh (chinese) lists SIMSUN.TTC - which I do NOT have, but this shouldn't be a problem because both font.properties.zh_TW (trad. chinese) and font.properties.zh_CN (simplified chinese) both list fonts thatI DO have (and the only Chinese locales I support are zh/TW and zh/CN. Is this a problem? If so, where can I get simsun.ttc (I search and searched and could not find the file).
Please help,
Andrew

Non-ASCII charatcers converted

Similar Messages

Maybe you are looking for