TextNode containing non-printable characters

Hi,
I have problem with TextNodes that contains non-printable charaters.
i.e: CR and LF
The problem occoures when i'm going to serialize my DOM object to an
XMLString for
storing into our database.
The result of the serialization process is that CR and LF are replaced with
whitespaces.
Does anyone have an 100% safe code example .... ?
TIA.
Borre Nordbakken

Take a look at the constants in cl_abap_char_utilities. A simpler solution would be to ask for a file without such characters...

Similar Messages

Inserting strings of printable and non printable characters

I would very much appreciate some help with the following
To handle an interface with a legacy system I need to create strings containing both printable and non-printabel ascii characters. And with non printable characters I mean in particular those in the range of ASCII 128 to 159.
It seems it is not possible to insert a string containting both printable and not printable characters from the afore mentioned range into a VARCHAR2 table column as the following demonstrates:
insert into test values(chr(156)); -- this inserts the 'œ' symbol.
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
┐                  156            1That the the character mapped is shown as '┐' and not 'œ' is not really issue for my application, what is important is that the ASCII value is shown as 156, which is the ASCII code of the character I inserted.
What is however strange (actually probably not strange but has to do with the lack of understanding of the issue at hand) is that substr returns an empty string...
Now I try to insert a concatenated string, first the "non printable" character then a printable character
insert into test values(chr(156)||chr(65));
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
A                   65            1 A                                     65For some reason the not printable character (chr(156)) is now not inserted or at least does not appear when I selected the data from the table, this effect seems to apply to all characters in the range of ASCII 128 to 159 (tried some but not all) However for instance CHR(13) can be inserted as part of a string as shown above .
For our application I really don't care much what character is shown or not show, what is important is that I can retrieve the ASCII value and that this value matches the one I inserted which for some reason does not seem to work.
This seems to be, at least to some extent a character set issue. I have also tested this on a database with character sets set as follows
NLS_CHARACTERSET
WE8MSWIN1252
NLS_NCHAR_CHARACTERSET
AL16UTF16
With WE8MSWIN1252 the described issue does NOT occur, however unfortunately I must use NLS_CHARACTERSET AL32UTF8 which produces the results as described above!
As said any insights would be much appreciated as I am slowly but surely starting to despair.
For completions sake, character sets are set as follows (changing it is NOT an option):
NLS_CHARACTERSET
AL32UTF8
NLS_NCHAR_CHARACTERSET
AL16UTF16
The test table is created as follows
CREATE TABLE TEST
TEST VARCHAR2(1000 BYTE)
Database Version 11.2.0.3.0
Edited by: helios.taraba on Dec 2, 2012 10:18 AM --Added database version
Edited by: helios.taraba on Dec 2, 2012 10:24 AM Added description of test results using NLS_CHARACTERSET WE8MSWIN1252

Hello Orafad,
Thanks for your reply, at least I understand the effects I'm seeing i.e.
+"For multibyte character sets, n must resolve to one entire code point. Invalid code points are not validated, and the result of specifying invalid code points is indeterminate."+
http://docs.oracle.com/cd/E11882_01/server.112/e26088/functions026.htm
You are absolutely right I could use chr(50579) to get the ligature symbol. However as what we are trying to achieve is to implement a legacy interface to a 20+ years old subsystem we are actually not so much interested in the symbol itself but rather in the ascii value of that symbol (156 as you so rightly point out in the win-1252 characterset), this particular field represents the lenght of the message being sent to the subsystem and can vary from decimal 68 to 164 and is also considered in a checksum calculation which is part of the message.
As changing the nls_characterset of the database is not an option I guess I only have one reasonable avenue to resolve this namely to push the functionality to added the "encoded" length of the message (and the calculation of the checksum) to the java driver which is responsible for sending the message (tcp/ip) to the subsystem. Here we should not have any issues adding a byte with the value 156 (or any other for that matter) to the datastream.
Thankfully all other fields have characters with ascii values below 128 and above 31.
I'm going to leave my question as un-answered for a bit longer in the hopes of someone coming up with a golden bullet, although not getting my hopes up.
Thanks, Helios

Cannot login with password containing non-ascii characters

Hello,
I have web application, form based login. UTF-8 is specified "everywhere".
And it works, except for passwords.
If user register itself with password containing non-ascii characters, it is correctly written in database, but when doing either programmatic login or normal form based login, if fails.
If the password is only ascii, it works.
Username of login could be ascii or non-ascii, it doesn't matter, both works.
I'm using sun java application server 9.1.
jdbc realm.
I'm not using hashing passwords, just clean (now)
I tried configure realm Charset: UTF8 as last chance, but it doesn't work either.
The problem is only with non-ascii characters in password.
Any help very appreciated
Thanks a lot

hi,
I know all that, but that's not the case. My app uses preparedStatements, everything is properly configured, in all pages, utf-8 is going from user to db and back without any problems.
The only problem is with password field. As I am using form based login, with jdbc realm configured (again, nicely working when only ascii characters), I have very little chance to do something bad through the login phase.
I'm not talking about special characters, I'm talking about non-ascii characters, let's say - Chinese, arabish, Russian alphabet etc.
When user registers (my code), the fields are properly written to db. I have checked that, trust me.
But the Sun app server realm seems to have some problems with the password field.
(realm uses jdbc connection to mysql, the url contains all extra parameters to be sure about utf8. there is nothing more what can be configured...)
If I try other alphabet codes in login and ascii in password, it works. But soon, as I use other alphabet code also in password, it doesn't work anymore.
My only idea is, that I could try MD5 to create ascii only characters (I hope it works that way) on the client with javascript and then set Digest to MD5 in realm configuration. But still, it seems very strange. The clear way storage should also function? (now set Digest to 'none')
Is it a bug of Sun App Server?
thanks

Robohelp 9 .properties file inserting non-printable characters @ export

I have a mapped help file that I am generating for integration to an online application. When we export the .properties file from the Project Set-up pod, the mapped files appear to be fine, if viewed in Notepad (see below).
However, when this is viewed in a different text editor, you can see that RoboHelp added additional non-printable characters to the .properties file (see below).
We've tried generating this from different computers, exporting it to different locations, retyping the initial entry, and haven't found a solution to this issue.
Does anyone know if there is a fix available? Are we doing something wrong?
Thanks!!
Kelly

Ask your developers if they think these characters could be what are known as BOM (byte order marks).
That is something can be seen in some files using the default encoding. There it can be changed by changing the encoding in the SSL dialog.
Maybe that explains it and if that is the cause, I don't know how you would prevent it here in Rh. I think you will have to live with your own solution.
See www.grainge.org for RoboHelp and Authoring tips
@petergrainge

Non Printable Characters in varchar or varchar2 filed

How can I know if a filed has non-printable characters.

An example :
TEST@db102 SQL> insert into test values('aaa'||chr(13)||'bbb'||chr(10)||'ccc');
1 row created.
TEST@db102 SQL> select * from test;
A
bbb
ccc
TEST@db102 SQL> select dump(a) from test;
DUMP(A)
Typ=1 Len=11: 97,97,97,13,98,98,98,10,99,99,99
TEST@db102 SQL>

Non printable characters in a text file..

hi,
How to get blank lines and non-printable characters
and remove those characters from the text file being uploaded from application server .
thanks,
Anil.

Take a look at the constants in cl_abap_char_utilities. A simpler solution would be to ask for a file without such characters...

Select xmlelement of clob containing non-printable chars gives ORA-31011

Using sqlplus against Ora9i database. We have some rows where the CLOB column contains non-printable chars because of a conversion from an old system.
I'm trying to extract this data using xmlselect. Most of the time it works great, but for the above rows it dies with error:
ORA-31011: XML parsing failed
If I embed substr(columname,1,25) so that it doesn't get to the non-printable character (which is at about position 200), then the sql works great.
Any ideas on how to massage this data, either on the fly, or I could do a one-time conversion. The problem is that darn CLOB. I can't even figure out how to find all the bad rows. Looking for sql to find the bad rows, or a function that I could put in a package, and then call from the select to convert it on the fly.
thanks,
Rick

Thanks for your help. Anything I do (including your suggestion) to try and parse the xml bombs with an internal error. Without raising the exception.
What I decided to do instead is do a search in the clob using instr(column, CHR(13)) but for also all CHR that are less than 32. I then replace it with a space. So I'm going to fix the data once rather than doing an on the fly fix.
thanks again,
Rick

Does Oracle User Password can contain non-ASCII characters?

Hi Experts,
Can we create a user with password containing non-ASCII characters like "Ro'çá".
I was able to create a database instance by providing the password for sys as "Ro'çá". But now i am not able to login from command prompt using sqlplus or SQLPLUS Application. I am getting below error:
C:\Documents and Settings\xyz>sqlplus system/Ro'çá@test
SQL*Plus: Release 10.2.0.1.0 - Production on Sun Jul 4 12:17:33 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
ERROR:
ORA-12154: TNS:could not resolve the connect identifier specified
Enter user-name:
same in case i enclose the password in ""
C:\Documents and Settings\xyz>sqlplus system/"Ro'çá"@test
SQL*Plus: Release 10.2.0.1.0 - Production on Sun Jul 4 12:17:33 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
ERROR:
ORA-12154: TNS:could not resolve the connect identifier specified
Enter user-name:
same in case i enclose the password in ""
Also using create user i am not able to create user with password in non-ASCII characters but alter user works and changes the password to non-ASCII characters when enclosed in quotes.
I wanted to know whether the password can contain non-ASCII characters or not?
Thanks in advance for your help.

I don't think that the characters used are allowed. See this doc which only mentions three characters to be used .
http://download.oracle.com/docs/cd/E11882_01/server.112/e10575/tdpsg_user_accounts.htm#BEICECGF
I did try the same but it didn't work for me either.
SQL> select * from V$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for 32-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
SQL> grant connect to user1 identified by Ro'çá;
ERROR:
ORA-01756: quoted string not properly terminated
SQL> grant connect to user1 identified by "Ro'çá";
Grant succeeded.
SQL> connect uesr1/"Ro'çá";
ERROR:
ORA-01017: invalid username/password; logon denied
Warning: You are no longer connected to ORACLE.
SQL> connect uesr1/"Ro'çá"
ERROR:
ORA-01017: invalid username/password; logon denied
SQL> connect uesr1/"Ro'çá"
ERROR:
ORA-01017: invalid username/password; logon denied
SQL>HTH
Aman....

Detecting non printables characters in a text file

Hi,
I need to remove some non printable characters like tabs, carriage returns, line feeds,.... and so!
i want to do something like
aString.replaceAll(<the non-printable char>, "");

str = str.replaceAll("\\P{Print}+", "");From http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pattern.html, \p{print} is printable characters and \P is the negative of \p.

Servlet Displaying Quotation Marks as Non-Printable Characters

I have a servlet which is reading an HTML file and displaying it's contents. My problem is that, in the output, quotation marks in the source html (" and ') are being reproduced as non-printable characters (). Furthermore, the same servlet prints the quotation marks fine under the Linux OS and Apache Web Server, but does not under the Windows (2000) OS and IIS Web Server (running j2sdk-1_3_0_02-win). Any suggestions would be appreciated. Code in question is below. "str" is the line from the file. :
     FileReader freader = new FileReader (filePath);
     BufferedReader breader = new BufferedReader(freader);
     String str = null;
     while ((str = breader.readLine()) != null) {
     document = document + str + "\n";
     freader.close();

Technically, you don't need to add the "\n" in there anyway. Newlines mean nothing to an HTML file if all you're doing is displaying that file. The lack of a carriage return, when the HTML is parsed, is completely irrelevant.
Also, when handling large String concatenations, it's always going to be more efficient to use StringBuffer.
StringBuffer sbDocument = new StringBuffer();
while((str = breader.readLine()) != null)
   sb.append(str);
String document = sbDocument.toString()

Need help in removing non printable characters

hi
I am having an issue with non printable characters in webservice. This webservice dishes out xml in B2B communication to my clients programs. Due to data corruption in oracle (dont know who is creating bad data ) I am having non printable characters in the xml file which is generated from database. I am dishing out this to our customers. since the data in updated every day it is imposible to fix the data every time. I need to write a very very effficient method to strip non printable characters from strings from the xml. Can some one Please help on this one. I want to make sure this method is very efficient because this method could be potentially be called lots of times. I am using JDK 1.3.1 and oracle 8i
Any help will be appreciated
Thanks
Ashok Pappu

At some point you existing program is probably converting from String data to the XML bytes through a CharsetEncoder, probably inside a java.io.Writer.
Perhaps your best approach might be to write your own java.nio.charset.CharsetEncoder which deals with the bad characters as you see fit.
You can register a new java.nio.charset.CharSet as a private character set type. Because this should result in simply replacing a standard CharsetEncoder with a non-standard one hopefully the overheads would be low.

Removing non printable characters from an excel file using powershell

Hello,
anyone know how to remove non printable characters from an excel file using powershell?
thanks,
jose.

To add - Excel is a binary file. It cannot be managed via external methods easily. You can write a macro that can do this. Post in the Excel forum and explain what you are seeing and get the MVPs there to show you how to use the macro facility
to edit cells. Outside of cell text "unprintable" characters are a normal part of Excel.
¯\_(ツ)_/¯

Removing non-printable characters

Hi All,
I was suppose to remove all non-printable characters, hence created below function. But in trouble for some rows in table.
function ar1(i_value in varchar2)
return varchar2
as
       pattern varchar2(1000) := '][';
       l_strVal varchar2(1000);
begin
       for i in 32 .. 126 loop
          pattern := pattern || case when chr(i) = '''' then ''''''
                                     when chr(i) in ('[', ']') then null
                                     else chr(i)
                                end;
       end loop;
          l_strVal := regexp_replace(i_value, '([' || pattern || '])|.', '\1', 1, 0, 'n');
return l_strVal;
end;
/Could anyone please help me, is this right way to do it.
Problem is occurring for one row as shown below.
SQL> select num, ar1(author) author1, author from doc where doc_num =37;
   NUM
AUTHOR1
AUTHOR
15098137
OM LESRAVI{|~
OM LES
RAVIIts removing non-printable characters but not sure how these {|~ are included in the resultant.
Would be great if anyone can help me.
Edited by: YasserRACDBA on Nov 9, 2010 3:53 PM

Thanks....but even your method is giving worng result as shown below.
SQL> select regexp_replace(author,'[[:cntrl:]]')
2 from doc where doc_num =15098137;
REGEXP_REPLACE(AUTHOR,'[[:CNTRL:]]')
OM LESRAVI{|~Is there any clue please...how come those {|~ are there??

Non printable characters

Hello
I have seen some posts about this topic, but none helped me.
I need to send to a device, via serial port (rs-232), four codes composed by printable and non-printable characters
For instance, I need to send a string with ASCII character 224, ASCII character 87, ASCII character 10, ASCII character 0 and ASCII character 191, together in the same string
Can someone help me to do this?
Thanks

What part are you having trouble doing?
To create those ASCII characters you can use a string constant or control set to '\' codes or hex display.
Lynn

Excluding all non-printable characters using regexp_replace

Hi All,
I need help in excluding all non-printable charcters using regexp_replace but i was not able to exclude One special character as shown below.
select regexp_replace('¥Tachyon-QX\_4 !? H*'' $(~!#$%^&*()?@#), BA' , '[^[:alnum:] \!\@\#\$\%\^\&\*\_\+\=\:\""\<\>\?\[\]\;\''\,\.\/\\\`\~\?\¥\] [:cntrl:]')
from dual;
REGEXP_REPLACE('%TACHYON-QX\_4!?H*''$(~!#$
%Tachyon-QX\_4 !? H*' $(~!#$%^&*()?@#), BABut problem is having with special character which i am not able to print it here...having char value - 13 and ascii value - 19. I want this value to be excluded as this nonprintable charcater.
Also i need to exclude newlines and print the output in a single line...
Could any one help me in these two requirement..
1. Exclude all non printable characters(Including special character having char value - 13 and ascii value - 19)
2. Exclude all newline characters..
Thanks,
Yasser

How about this?
SQL>select regexp_replace('Tachyon-QX\_4 !? H*'' $(~!#$%^&*()?@#), BA
2 some junk on line 2
3 xyz line 3' , '[^[:print:]]') printable
4 from dual;
PRINTABLE
Tachyon-QX\_4 !? H*' $(~!#$%^&*()?@#), BAsome junk on line 2xyz line 3

TextNode containing non-printable characters

Similar Messages

Maybe you are looking for