Ascii characters removal

hii experts,
this is shreya.i have a payload from where i need to remove the special characters searching them through their ascii codes.
Please suggest a solution to my problem.

Hi Shreyasen Gupta,
I think this issue not related to character encoding (ascii or UTF-8). It is related to well formed XML.
Please note:- quote ("), apostrophe ('), ampersand (&), less than (<), greater than (>) are special characters in XML. They should not be present in data.
http://www.w3.org/TR/REC-xml/ The ampersand character (&) and the left angle bracket (<) must not appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section. If they are needed elsewhere, they must be escaped using either numeric character references or the strings " & " and " < " respectively.
Escape characters u201C &quot ; , u2018 &apos ; , & amp ; , < &lt ; , > &gt ;
Not Well formed XML:
<Name>Raghuu2019s</Name> <Name> Raghu & Vamsee </Name>
Well formed XML:
<Name>Raghu's</Name> <Name>Raghu & Vamsee</Name>
It is very difficult (if not impossible) to escape & and <, to make XML 100 % well formed in PI. Escaping should be done at source side, when XML is getting generated.
If you have no control on sender side, then find out in which element of XML these characters are coming and then write a Java mapping to escape the special characters. This solution will not fix the issue to 100% accuracy, but I feel it is the best available solution. Note, Graphical and XML mapping cannot take Not Well formed XML as input, so they cannot be used.
Regards,
Raghu_Vamsee

Similar Messages

Removing Non-Ascii Characters from a String

Hi Everyone,
I would like to remove all NON-ASCII characters from a large string. For example, I am taking text from websites and would like to remove all the strange arabic and asian characters. How can I accomplish this?
Thank you in advance.

I would like to remove all NON-ASCII characters from a large string. I don't know if its a good method but try this:
str="\u6789gj";
output="";
for(char c:str.toCharArray()){
if((c&(char)0xff00)==0){
output=output+c;
System.out.println(output);
all the strange arabic and asian characters.Don't call them so.... I am an Indian Muslim ;-) ....
Thanks!

Removing Non Ascii Characters.

Dear Friends,
In our application, User copying some data from a document and pasting in a field "Comments".
If that data consists anything like bullets,arrows of word document. It is inserting some Non keyboard characters into database like below.
â¢ Analysys
â¢ Do
â¢ Now
â¢ When
â¢ As
â¢ We
donât know how much he love sthe testingâI am not crazyhâ
I AM âUSERâ ï»ï»ï¨
ï¨
ï®
ï¼ Uu
ï¼ Yy
ï¼ tt
Now user asking to remove all those Non-ASCII characters from Comments Column. Please help!

Hi Santosh,
I can remember that I have given you the REGEXP_REPLACE query earlier which you have specified and told you to read some document about it to modify according to your need. It is not very wise thing to depend on others every time.
Re: Removing Junk Characters.
Anyway, REGEXP_REPLACE(str,'[^[a-z,A-Z,0-9,chr(0)-chr(127)[:space:]]]*','') can give you some pointer (not tested).

How to get the correct length...removing appended extra ascii characters

I have a table fs_lg_partgroups, having the columns description,subgroupname.
I have a query like this..
select description,length(description) from fs_lg_partgroups where subgroupname='FORMULA226';
Then i got the output as
formula226 and the length is 42. Even though there are 10 characters only.
I came to know that some ascii characters are appended to description values, so it is showing the length like this.
But i don't know how to remove those ASCII characters from description column for all rows to get the exact length.
Please can u help me..

If you have used char data type then use varchar2 datatype instead of char datatype for description column.
Learn about char and varchar2 datatypes here.
http://download-east.oracle.com/docs/cd/B10501_01/server.920/a96524/c13datyp.htm#7223

Which ASCII Characters are acceptable?

I'm trying to clean up my iPhoto library so it'll sync with my iPhone. One possible cause I've seen mentioned here is the use of non-standard ASCII characters in Keywords, Album names, etc.
Can you wise folks provide a bit more info here?
FIRST QUESTION: I'm trying to figure out exactly which characters need to be eliminated. Are the following characters allowed or not?
. Period
, Comma
- Hyphen
_ Underline
( ) Parentheses
(space)
& Ampersand
123 Numbers
These seem like relatively common characters, so I'm hoping I don't have to remove every one of them. It would be especially difficult to remove every "space" from every title.
SECOND QUESTION: Exactly which fields do I need to clean up?
- Keywords (definitely)
- Albums?
- Film Rolls (I hope not)
- Photo Titles (please say no...)
Please help! Depending on the answers, this could either be a manageable project or a week-long chore.
And by the way... If these characters are verboten, why doesn't iPhoto reject them when I first try to enter the data?

Adam
Are you actually having a problem?
There is no list, but when I post that query it's because some people who post here have had issues that were solved when they removed non-ascii characters. Others have found issues when they removed leading and trailing spaces... so (where _ stands for a space)
_keyword
or keyword keyword_
have caused issues, thought the space between keyword and keyword has not caused issues.
So, here's my (not very helpful, I admit) take on it: non_ascii characters are not a problem, trailing and leading spaces are not a problem, unless, of course, they are.
The real issue seems to be that they lead to a malformed albumdata.xml file, which is the file that other apps read to interact with.
I think the best way to examine the albumdata.xml file is the way that Dave E shows:
http://discussions.apple.com/thread.jspa?messageID=3870956&#3870956
Regards
TD

Convert to ASCII characters

We are extracting information, such as article descriptions to send to some legacy systems. These systems can only accept ASCII characters (space, A-Z, 0-9, a-z, and the standard characters on the US keyboard). These are decimal values 32 through 126 or hex values 20 through 7E.
SAP apparently allows entries of decimal values 127 and greater (such as ½).
We need to either remove or convert such characters to spaces.
Is there a function to do this? Or some simple command?
Thanks in advance for your help.

Thank you both for your suggestions but I couldn't get any of these functions to do what I want.
We have a description for an article that includes "Size 7 ½". The ½ symbol causes a problem with interfaces to some of our legacy systems.
I know I can replace the ½ symbol with a space, but eventually some user will use yet another symbol (such as ¼) and I don't want to try to create a replace for each of these symbols.
I would like to investigate the decimal or hex equivilent of each character and replace those out of range.
Is there any function that would allow this?
Thanks,

Convert smart quotes and other high ascii characters to HTML

I'd like to set up Dreamweaver CS4 Mac to automatically convert smart quotes and other high ASCII characters (m-dashes, accent marks, etc.) pasted from MS Word into HTML code. Dreamweaver 8 used to do this by default, but I can't find a way to set up a similar auto-conversion in CS 4. Is this possible? If not, it really should be a preference option. I code a lot of HTML emails and it is very time consuming to convert every curly quote and dash.
Thanks,
Robert
Digital Arts

I too am having a related problem with Dreamweaver CS5 (running under Windows XP), having just upgraded from CS4 (which works fine for me) this week.
In my case, I like to convert to typographic quotes etc. in my text editor, where I can use macros I've written to speed the conversion process. So my preferred method is to key in typographic letters & symbols by hand (using ALT + ASCII key codes typed in on the numeric keypad) in my text editor, and then I copy and paste my *plain* ASCII text (no formatting other than line feeds & carriage returns) into DW's DESIGN view. DW displays my high-ASCII characters just fine in DESIGN view, and writes the proper HTML code for the character into the source code (which is where I mostly work in DW).
I've been doing it this way for years (first with GoLive, and then with DW CS4) and never encountered any problems until this week, when I upgraded to DW CS5.
But the problem I'm having may be somewhat different than what others have complained of here.
In my case, some high-ASCII (above 128) characters convert to HTML just fine, while others do not.
E.g., en and em dashes in my cut-and-paste text show as such in DESIGN mode, and the right entries
    –
    —
turn up in the source code. Same is true for the ampersand
    &
and the copyright symbol
    ©
and for such foreign letters as the e with acute accent (ALT+0233)
    é
What does NOT display or code correctly are the typographic quotes. E.g., when I paste in (or special paste; it doesn't seem to make any difference which I use for this) text with typographic double quotes (ALT+0147 for open quote mark and ALT+0148 for close quote mark), which should appear in source code as
    “[...]”
DW strips out the ASCII encoding, displaying the inch marks in DESIGN mode, and putting this
    "[...]"
in my source code.
The typographic apostrophe (ALT+0146) is treated differently still. The text I copy & paste into DW should appear as
    [...]’[...]
in the source code, but instead I get the foot mark (both in DESIGN and CODE views):
I've tried adjusting the various DW settings for "encoding"
    MODIFY > PAGE PROPERTIES > TITLE/ENCODING > Encoding:
and for fonts
    EDIT > PREFERENCES > FONTS
but switching from "Unicode (UTF-8)" to "Western European" hasn't solved the problem (probably because in my case many of the higher ASCII characters convert just fine). So I don't think it's the encoding scheme I use that's the problem.
Whatever the problem is, it's caused me enough headaches and time lost troubleshooting that I'm planning to revert to CS4 as soon as I post this.
Deborah

Username with ascii characters

Hello, i'm having and html form and i would like the user in
the username field to type ONLY ascii characters.
For example, in other fields of the form i
would like the user to type his mother language but
as far as the username and password fields are concerned
the characters have to be ascii.
How am i supposed to check when the username is accepted/correct (*consists of ascii characters*)?
and which are the desirable characters a username must have (e.g. *?* is a desirable character in a username , *:* this one?)
Thanks, in advance!

g_p_java wrote:
How am i supposed to check when the username is accepted/correct (*consists of ascii characters*)?ASCII characters are the Unicode characters whose code points are between 0 and 127.
and which are the desirable characters a username must have (e.g. *?* is a desirable character in a username , *:* this one?)I don't understand this. You have already said they must be ASCII. You have other requirements? Fine, go ahead and program them and ask questions if you have problems with that. Personally I don't think that requiring somebody to have a question mark in their user name is a good idea -- but probably you didn't mean it when you suggested that.

Non ascii characters being sent from a parameter in a form

Hi!
I have seen many topics posted on passing non ascii characters through parameters from one servlet to another and converting them into whatever format is necessary.
However, I have not seen anyone answer the following question. I have a jsp page (html) with the character encoding set to utf-8. The user inputs some data in to a text field which is inside a form. The data could be in non ascii characters such as hebrew or arabic. This form is then sent to another jsp where i try to retreive the data from teh text field. No matter what i do, i cannot get the data presented correctly. It is either question marks or other wierd symbols.
I have tried every permetation of encoding of the actual html page, the ecoding of the string from request.getParameter etc but it still is not presented on the new html page correctly.
Can anyone help??
Spencer

Ok, I solved the problem.
I had to put at the top request.setCharacterEncoding("utf-8");
Spencer

Replacing non-ASCII characters with HTML charcter references

Hi All,
In Oracle 10g or greater is there a built-in function that will convert a string with non-ASCII characters like this
a b č 뮼
into an ASCII string with HTML character references like this?
a b & # x 0 1 0 D ; & # x B B B C ;
(note I had to include spaces between each character in the sample code for message to prevent the forum software from converting my text)
I tried using
utl_i18n.escape_reference( val, 'us7ascii' )
but for some reason it returns
a b c & # x B B B C ;
Note how it converted the Western European character "č" to its unaccented counterpart "c", not "& # x 0 1 0 D ;" (is this a bug?).
I also tried a custom solution using regexp_replace and asciistr (which I can't include here because the forum software chokes on it) but it only returns the correct result for values <=4000 characters long. Unfortunately asciistr doesn't appear to accept CLOB values larger than 4000 characters. It returns an error message like
(ORA-22835: Buffer too small for CLOB to CHAR or BLOB to RAW conversion (actual: 30251, maximum: 4000) ).
I'm looking for a solution that works on CLOB data of any size.
Thanks in advance for any insight you can provide.
Joe Fuda

So with that (UTF8) in mind, let's take another look.....
As shown below, I used a AL32UTF8 database.
Note: I did not use a unicode capable tool for querying. So I set console mode code page to 1250 just to have č displayed properly (instead of posing as an è).
Also, as a result of using windows-1250 for client character set, in the val column and in the second select's ncr column (iso8859-1), è (00e8) has been replaced with e through character set conversion going from server back to client.
Running the same code on a database with a db character set such as we8mswin1252, that doesn't define the č (latin small c with caron) character, would yield results with a c in the ncr column.
C:\>chcp 1250
Aktuell teckentabell: 1250
C:\>set nls_lang=.ee8mswin1250
C:\>sqlplus test/test
SQL*Plus: Release 11.1.0.6.0 - Production on Fri May 23 21:25:29 2008
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the OLAP option
SQL> select * from nls_database_parameters where parameter like '%CHARACTERSET';
PARAMETER              VALUE
NLS_CHARACTERSET       AL32UTF8
NLS_NCHAR_CHARACTERSET AL16UTF16
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'us7ascii') NCR from dual;
VAL NCR
č e c e
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'we8iso8859p1') NCR from dual;
VAL NCR
č e &# x10d; e     <- "è"
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'ee8iso8859p2') NCR from dual;
VAL NCR
č e č &# xe8;
SQL> select unistr('\010d \00e8') val, utl_i18n.escape_reference(unistr('\010d \00e8'),'cl8iso8859p5') NCR from dual;
VAL NCR
č e &# x10d; &# xe8;In the US7ASCII case, where it should be possible for all non-ascii characters to be escaped, it seems as if the actual escape step is skipped over.
Hope this helps to understand whether utl_i8n is usable or not in your case.
Message was edited by:
orafad
Fixed replaced character references :)

How can I convert ASCII characters to ISO8859?

Hi All,
I have written a little application that renames a TV episode by scraping a TV listing site for the episode name. It is written in SWT and works great apart from on small problem. When getting the html back from the site, it sometimes contains special ASCII characters that are not in the ISO8859 (Windows filesystem) character set.
For example, this is the line that I have to parse:
<td style='padding-left: 6px;' class='b2'><a href='/Prison_Break/episodes/569183/03x01'>Orientaci��n</a></td>When viewing it in a browser, it is:
<td style="padding-left: 6px;" class="b2"><a href="/Prison_Break/episodes/569183/03x01">Orientaci�n</a></td>Notice that the o in the title has an accent on it. While researching this problem I stumbled across 'HTML Entities to ISO 8859-1 Converter' at http://www.inweb.de/chetan/English/Resources/Java/HTML%202%20ISO.html. This open source project takes in an html entity like & and returns '&'.
So that is not quite what I want, as my BufferedReader is converting the html entity into the ASCII representation already. I need a way of detecting a non ISO8859 character within an ASCII string, and hopefully replacing its natural 'equivalent' (would be o in this case).
Does anyone know how I could do it without having to check for every special char and replacing (not really an option unless someone has done it before!!)
If not that then, perhaps another way to attack the problem?
Any help greatly appreciated ;)
Dave

Hi,
NZ_Dave wrote:
For example, this is the line that I have to parse:
<td style='padding-left: 6px;' class='b2'><a href='/Prison_Break/episodes/569183/03x01'>Orientaci��n</a></td>
This is coded in UTF-8. If you convert the bytes to a String using the UTF-8 encoding, then you will have the correct characters "Orientaci�n" in the string.
Check your parser where it converts the bytes (coming from e.g. an InputStream) to characters. Use UTF-8 as the charset when doing that conversion.

How can I use ASCII Characters on the iPad?

I have an iPad (1) and use iOS 4.3.5. Is there an easier Way of using ASCII Characters than having to use the "copy-Paste" procedure? If not, can we expect to see the feature of entering the ASCII code straight through the standard keyboard soon?

Norisouro wrote:
. Is there an easier Way of using ASCII Characters than having to use the "copy-Paste" procedure?
You need to give some details about what it is you want to do, because ASCII Characters are what are already on the keyboard. It is non-ASCII that you might need to copy/paste.
http://en.wikipedia.org/wiki/ASCII
I think you can be sure that Apple is never going to include a feature in iOS that has you input special characters by typing in numbers like Windows does it. Mac's have always used a different approach.

Given filename or path contains Unicode or double-byte characters.Retry using ASCII characters for filename and path What does this mean? it happen when I publish an OAM

Given file name or path contains Unicode or double-byte characters. Retry using ASCII characters for filename and path
What does this mean? It is happening when I try to publish an OAM for Dreamweaver.
Also: How can I specify the browser in Edge Animate? It is just going wherever. Are there no Preferences for Edge Animate?
BTW. Just call it Edge. Seriously. Do you call it Illustrator Draw? Photoshop Retouching?

No, my file name is mainContent.oam
My project name is mainContent.an
This error happens when I try to import into Dreamweaver. Sorry, I wasn't clear on that earlier.
I thought maybe it was because I had saved my image as a png. So re-saved as a svg, still get the error.
DO I have a setting is Dreamweaver CC that is wrong? Should I try this in Dreamweaver CS6? I might try that next.
Why is this program so difficult? I know Flash. I know After Effects. I can work the timeline part just great. It's always in the export that I have problems.
On a MacPro, 10.7.
Are you an Adobe person or just a nice helper?

Non US-ASCII characters in download file names

I am trying to implement a simple file download in a JSP, and trying to get IE, Firefox and Opera to all display and handle non US-ASCII characters in the suggested download file name. Only concerned with Windows platform for now. Here's the code I am currently using:
String agent = request.getHeader("USER-AGENT");
if (null != agent && -1 != agent.indexOf("MSIE"))
String codedfilename = URLEncoder.encode(cfrfilename, "UTF8");
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + codedfilename);
else if (null != agent && -1 != agent.indexOf("Mozilla"))
String codedfilename = MimeUtility.encodeText(cfrfilename, "UTF8", "B");
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + codedfilename);
else
response.setContentType("application/x-download");
response.setHeader("Content-Disposition","attachment;filename=" + cfrfilename);
}This URL encodes the file name if the browser is IE, MIME encodes it if the browser is Mozilla, and sends plain UTF-8 (the encoding of the JSP) for all other browsers. I get "cfrfilename" from translated properties files, and the string can contain characters from any character set - Chinese, Thai, Korean, etc.
This code works correctly for IE - the file name is displayed correctly in the file Save as dialog, and it is saved correctly on disk, no matter which character set is used.
For Firefox, the file name is displayed correctly in the file Save as dialog, but it is only saved correctly to disk if the file name is in a character set supported by the system locale. This seems to be a known Firefox bug (not fully using the Windows Unicode APIs), so nothing I can do about that.
Nothing seems to work for Opera, however - I cannot get the file name to display correctly in the file Save as dialog, no matter which method I use (I have tried URL encoding and MIME encoding in addition to the plain UTF-8).
Has anybody implemented something similar that works for at least these 3 browsers?

I tested your code today,
                     dialog           save           open
Firefox 1.5          OK                 OK               OK
IE 6.0                OK                 OK                NGdailog: filename show in download popup dialog
save: save to disk from dialog
open: open directly from dailog

Cannot login with password containing non-ascii characters

Hello,
I have web application, form based login. UTF-8 is specified "everywhere".
And it works, except for passwords.
If user register itself with password containing non-ascii characters, it is correctly written in database, but when doing either programmatic login or normal form based login, if fails.
If the password is only ascii, it works.
Username of login could be ascii or non-ascii, it doesn't matter, both works.
I'm using sun java application server 9.1.
jdbc realm.
I'm not using hashing passwords, just clean (now)
I tried configure realm Charset: UTF8 as last chance, but it doesn't work either.
The problem is only with non-ascii characters in password.
Any help very appreciated
Thanks a lot

hi,
I know all that, but that's not the case. My app uses preparedStatements, everything is properly configured, in all pages, utf-8 is going from user to db and back without any problems.
The only problem is with password field. As I am using form based login, with jdbc realm configured (again, nicely working when only ascii characters), I have very little chance to do something bad through the login phase.
I'm not talking about special characters, I'm talking about non-ascii characters, let's say - Chinese, arabish, Russian alphabet etc.
When user registers (my code), the fields are properly written to db. I have checked that, trust me.
But the Sun app server realm seems to have some problems with the password field.
(realm uses jdbc connection to mysql, the url contains all extra parameters to be sure about utf8. there is nothing more what can be configured...)
If I try other alphabet codes in login and ascii in password, it works. But soon, as I use other alphabet code also in password, it doesn't work anymore.
My only idea is, that I could try MD5 to create ascii only characters (I hope it works that way) on the client with javascript and then set Digest to MD5 in realm configuration. But still, it seems very strange. The clear way storage should also function? (now set Digest to 'none')
Is it a bug of Sun App Server?
thanks

Ascii characters removal

Similar Messages

Maybe you are looking for