Select xmlelement of clob containing non-printable chars gives ORA-31011

Using sqlplus against Ora9i database. We have some rows where the CLOB column contains non-printable chars because of a conversion from an old system.
I'm trying to extract this data using xmlselect. Most of the time it works great, but for the above rows it dies with error:
ORA-31011: XML parsing failed
If I embed substr(columname,1,25) so that it doesn't get to the non-printable character (which is at about position 200), then the sql works great.
Any ideas on how to massage this data, either on the fly, or I could do a one-time conversion. The problem is that darn CLOB. I can't even figure out how to find all the bad rows. Looking for sql to find the bad rows, or a function that I could put in a package, and then call from the select to convert it on the fly.
thanks,
Rick

Thanks for your help. Anything I do (including your suggestion) to try and parse the xml bombs with an internal error. Without raising the exception.
What I decided to do instead is do a search in the clob using instr(column, CHR(13)) but for also all CHR that are less than 32. I then replace it with a space. So I'm going to fix the data once rather than doing an on the fly fix.
thanks again,
Rick

Similar Messages

Map binary data (PDF) to XML not possible due to non-printable chars

Hi XI Gurus,
we have the following issue.
We send a PDF as binary data (as a hex string '25255044462D3' ) along with some other information from ERP within one RFC to XI , doing a message split 1:n for those two kind of messages (1. the PDF, 2. the XML) an send the split messages to file/ftp receiver adapter.
The message split and sending of the messages is working well, but we encounter some problems with non-printable chars (hex code below 0x20) in the pdf binary. The PDF data and the dynamic file name is mapped into an XML. But the non printable characters are converted into '#' when mapped into the XML target field.
(Due to the 1:n multi mapping, we cannot put the filename into dynamic configuration, so we have to map the pdf and the filename into an XML and extract the content with variable substitution in receiver file adapter....)
My question is: how can binary data with non-printable chars be sent through XI and can be mapped into an XML without beeing replaced by '#' ?
Any help will be greatly appreciated.
Thanks and regards
Holger

Maybe I didn´t explained it clearly enough.
We do not have the issue that the RFC puts the '#' into the string. We got from the RFC a hex string containing the pdf as visible hex values like:
As you can see we have the pdf as hex string. During message mapping XI replaces some non-printable chars like '0x04' or '0x19' with '#'.
My question is: how can we avoid those char replacement ?
BTW: I grabbed a pdf with sender file adapter, routed through XI without any mapping and send it with ftp in binary mode. But the pdf contains more chars as the origional file and the pdf content is not visible when opening with a pdf reader like Acrobat. I guess the file adpter has problems with carriage return and linefeed chars. Some CR and LF are replaced inserted somehow.
Best regards,
Holger

How to load file thru reader which contains non-english char in file name

Hi ,
I want to know how to load file in english machine thru reader which contains non-english chars in file names (eg. 置顶.pdf)
as LoadFile gives error while passing unicode converted file name.
Regards,
Arvind

You don't mention what version of Reader? And you are using the AcroPDF.dll, yes?
Sent from my iPad

TextNode containing non-printable characters

Hi,
I have problem with TextNodes that contains non-printable charaters.
i.e: CR and LF
The problem occoures when i'm going to serialize my DOM object to an
XMLString for
storing into our database.
The result of the serialization process is that CR and LF are replaced with
whitespaces.
Does anyone have an 100% safe code example .... ?
TIA.
Borre Nordbakken

Take a look at the constants in cl_abap_char_utilities. A simpler solution would be to ask for a file without such characters...

Can Java be started in a directory that contains non ascii char

I installed a product developed using Java in a folder whose name contains non-ascii chars, such as Japanese chars or german chars.
This will cause error said: unable initialise java virtual machine, error code -1
Some one said Java doesn't like being started in a directory that contains non ascii characters. There appears to be no way of passing it unicode parameters.
Is there anyone once hit the similiar issue or know the root cause of such problem?
Thanks

Yes you can use your web start application console. To enter data which is required for your application it is better idea to use java application which runs in console mode althou you may try to run console of windows and then read data from its input stream.

Search for description containing non-English chars -- ?

Hello!
I've implemented a search class, which allows customers to search folders/documents by name, owner, description, etc.
And here's the problem: if description contains non-English (Russian, in my case) characters, search does not work! Everything (AS infrastructure, CM SDK DB, etc) was installed using UTF-8 Unicode charset. When I debug the code, I see that when I build AttributeQualification and later compose a comples SearchQualification, value in these is correct, but when I call getSQL(), I see string like this:
... ( nls_upper(ALIASDOCUMENT.DESCRIPTION) LIKE nls_upper('????') ) ...
So it seems as if SQL converted passed UNICODE value into ANSI string, and since server's system language is English, my Russian letters were lost -- ?
Can anybody shed some light here? Is there a way to search for UNICODE descriptions (and content, for that matter)?
Thanks,
Sasha.

Hi Sasha,
I want you to try the following code. It should output the file description and query to a text file. Use internet explorer / or notepad to open this file and ****specify that the file encoding is UTF8.*****
thanks,
matt.
java -classpath ...blah blah.. RussianSearch parameterfile=c\cmsdkparameters.txt
cmsdkparameters.txt contains:
Username = system
Password = oracle9i
SchemaPassword = cmsdk
Domain = ifs://ifspm-sun2.us.oracle.com:1521:mjs92.us.oracle.com:cmsdk903
ServiceConfiguration = SmallServiceConfiguration
Service = TestService
import oracle.ifs.beans.LibraryService;
import oracle.ifs.beans.LibrarySession;
import oracle.ifs.beans.ClassObject;
import oracle.ifs.beans.Document;
import oracle.ifs.beans.DocumentDefinition;
import oracle.ifs.beans.Folder;
import oracle.ifs.beans.FolderDefinition;
import oracle.ifs.beans.LibraryObject;
import oracle.ifs.beans.PublicObject;
import oracle.ifs.beans.Search;
import oracle.ifs.beans.SearchResultObject;
import oracle.ifs.common.IfsException;
import oracle.ifs.common.AttributeValue;
import oracle.ifs.common.CleartextCredential;
import oracle.ifs.common.Credential;
import oracle.ifs.common.ParameterTable;
import oracle.ifs.search.AttributeQualification;
import oracle.ifs.search.AttributeSearchSpecification;
import oracle.ifs.search.SearchClassSpecification;
import oracle.ifs.search.SearchSortSpecification;
import java.io.FileOutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.util.Hashtable;
import java.util.Vector;
* Copyright (c) 2003 Oracle Corporation. All rights reserved.
* Matt Shannon.
* Description:
* Test searching in Russian Language
* View output file in notepad or IE - make sure to specify character
* set of document to be UTF8 when opening.
public class RussianSearch implements Runnable
// set to 'false' to prevent the class from freeing objects that it creates
public static final boolean performCleanup = true;
protected ParameterTable m_parametertable;
private Vector m_ObjectsRequiringCleanup;
public RussianSearch(String[] args)
    // parameter file is retrieved through command line argument parameterfile=
    m_parametertable = new ParameterTable(args, "parameterfile");
public static void main(String[] args)
    new Thread(new RussianSearch(args)).start();
   *   This is where you write your test program.
public void run()
    LibraryService service = startService();
    LibrarySession session = establishSession(service);
    if (session == null)
      return;
    try
      DocumentDefinition ddef = new DocumentDefinition(session);
      ddef.setAttribute(PublicObject.NAME_ATTRIBUTE,
        AttributeValue.newAttributeValue("blah.txt"));
      ddef.setAttribute(PublicObject.DESCRIPTION_ATTRIBUTE,
        AttributeValue.newAttributeValue("Я скучаю по родине"));
      ddef.setEmptyContent();
      Document newdoc = (Document) session.createPublicObject(ddef);
      addObjectRequiringCleanup(newdoc);
      /* Construct AttributeSearchSpecification.
       * Attribute based conditions are allowed, context conditions are not!
      AttributeSearchSpecification attrSrchSpec =
        new AttributeSearchSpecification();
      /* Construct SearchClassSpecification.
       * This represents the FROM and SELECT clauses of the query.
      SearchClassSpecification srchClsSpec = new SearchClassSpecification();
      srchClsSpec.addSearchClass(Document.CLASS_NAME);      // from clause
      srchClsSpec.addResultClass(Document.CLASS_NAME);      // select clause
      /* Construct SearchSortSpecification.
       * This represents the ORDER BY clause of the query.
      SearchSortSpecification srchSortSpec = new SearchSortSpecification();
      // upper case ascending sort on Name
      srchSortSpec.add(Document.CLASS_NAME, PublicObject.NAME_ATTRIBUTE,
        SearchSortSpecification.ASCENDING, "nls_upper");
      /* AttributeQualification is a WHERE clause component representing an
       * attribute condition.
      // scalar AttributeQualification - name like '%.html'
      AttributeQualification aq = new AttributeQualification();
      aq.setAttribute(Document.CLASS_NAME, PublicObject.DESCRIPTION_ATTRIBUTE);
      aq.setOperatorType(AttributeQualification.LIKE);
      aq.setValue("%родине");
      // set SELECT & FROM clauses
      attrSrchSpec.setSearchClassSpecification(srchClsSpec);
      // set ORDER BY clause
      attrSrchSpec.setSearchSortSpecification(srchSortSpec);
       // set WHERE clause
      attrSrchSpec.setSearchQualification(aq);
      /* Construct Search, supply SearchSpecification */
      Search s = new Search(session,attrSrchSpec);
      System.out.println("File encoding system property: "+System.getProperty("file.encoding"));
      boolean append = false;
      FileOutputStream fos = new FileOutputStream("c:/test.txt",append);
      OutputStreamWriter osw = new OutputStreamWriter(fos);
      System.out.println("Default character encoding: "+osw.getEncoding());
      osw = new OutputStreamWriter(fos,"UTF8");
      System.out.println("New character encoding: "+osw.getEncoding());
      PrintWriter out = new PrintWriter(osw,true);
      out.println(s.getSQL());
      SearchResultObject obj = null;
      // Open Search!
      s.open();
      try
         * A SearchResultObject encapsulates a row of a search result. It
         * contains 1 or more LibraryObjects (depending on number of result
         * classes specified).
        while ( (obj = s.next()) != null )
          Document d = (Document)(obj.getLibraryObject(Document.CLASS_NAME));
          out.println(d.getName() + " " + d.getDescription());
      catch (Throwable e)
        if ((e instanceof IfsException) &&
          (((IfsException)e).containsErrorCode(22000)))
        else
          System.out.println("Unexpected exception occurred in selector cursor");
          System.out.println((e instanceof IfsException)
            ? ((IfsException)e).toLocalizedString()
            : e.toString());
      finally
        out.close();
        if (performCleanup)
          cleanup();
        s.close();
        s.dispose();
    catch (Throwable e)
      System.out.println("Fatal exception occurred in run():");
      System.out.println((e instanceof IfsException)
        ? ((IfsException)e).toLocalizedString()
        : e.toString());
    finally
      disconnectSession(session);
public LibraryService startService()
    String schemapassword = m_parametertable.getString("SchemaPassword");
    String domain = m_parametertable.getString("Domain");
    String servicename = m_parametertable.getString("Service",domain);
    String serviceconfiguration =
      m_parametertable.getString(
        "ServiceConfiguration","SmallServiceConfiguration"
    LibraryService service = null;
    try
      if (servicename != null &&
        LibraryService.isServiceStarted(servicename))
        // The service name was specified, and is already running.
        // So just use it.
        System.out.println("Service already running: "+servicename);
        service = LibraryService.findService(servicename);
        System.out.println("Existing service retrieved");
      else
        service = LibraryService.startService(
          servicename, schemapassword, serviceconfiguration, domain);
        System.out.println("Service started: '"+servicename+
          "' (version: "+service.getVersionString()+")");
    catch (Throwable e)
      System.out.println("Unable to start service:");
      System.out.println((e instanceof IfsException)
        ? ((IfsException)e).toLocalizedString()
        : e.toString());
    return service;
public LibrarySession establishSession(LibraryService service)
    String username = m_parametertable.getString("Username");
    String password = m_parametertable.getString("Password");
    return establishSession(service, username, password);
public LibrarySession establishSession
    LibraryService service,
    String username,
    String password
    LibrarySession session = null;
    try
      CleartextCredential cred = new CleartextCredential(username,
        password);
      session = establishSession(service, cred);
    catch (Throwable e)
      System.out.println("Unable to create credential:");
      System.out.println((e instanceof IfsException)
        ? ((IfsException)e).toLocalizedString()
        : e.toString());
    return session;
public LibrarySession establishSession
    LibraryService service,
    Credential cred
    LibrarySession session = null;
    if (service != null)
      try
        String username = cred.getName();
        session = service.connect(cred, null);
        System.out.println("Session established for " + username);
      catch (Throwable e)
        System.out.println("Unable to create session:");
        System.out.println((e instanceof IfsException)
          ? ((IfsException)e).toLocalizedString()
          : e.toString());
    return session;
public void disconnectSession(LibrarySession session)
    System.out.println("Disconnecting session");
    try
      session.disconnect();
    catch (Throwable e)
      System.out.println("Error disconnecting session:");
      System.out.println((e instanceof IfsException)
        ? ((IfsException)e).toLocalizedString()
        : e.toString());
public void addObjectRequiringCleanup(LibraryObject lo)
    Vector v = getObjectsRequiringCleanupVector();
    v.addElement(lo);
private Vector getObjectsRequiringCleanupVector()
    if (m_ObjectsRequiringCleanup == null)
      m_ObjectsRequiringCleanup = new Vector();
    return m_ObjectsRequiringCleanup;
   * Frees objects that were marked as requiring clean up
public void cleanup()
    Vector v = getObjectsRequiringCleanupVector();
    System.out.println("Cleanup - delete objects created during the session");
    int count = (v == null) ? 0 : v.size();
    System.out.println("# of objects to free: "+count);
    // Free the objects in reverse order from which they were added
    for (int i = count - 1; i >= 0; i--)
      LibraryObject lo = (LibraryObject)v.elementAt(i);
      try
        discardObject(lo);
      catch (Exception e)
        System.out.println("Unable to discard an object during cleanup - continuing...");
public void discardObject(LibraryObject lo) throws IfsException
    if (lo != null)
      try
        System.out.println("Attempting to free: "+getDisplayName(lo));
        LibrarySession session = lo.getSession();
        if (lo instanceof Folder)
          System.out.println("Attempting to free Folder with Deep Option!");
          // free Folder using "Deep" option to free
          // all items in the folder, and all of their items, etc.
          Folder folder = (Folder)lo;
          FolderDefinition def = new FolderDefinition(session);
          def.setFolderDepthOption(
            Folder.SYSTEMOPTIONVALUE_FOLDER_DEPTH_DEEPEST);
          folder.free(def); // removes object from the repository, with options
        else
          // just a regular free
          lo.free();
      catch (Exception e)
        System.out.println("Unable to free an object during cleanup - continuing");
        System.out.println((e instanceof IfsException)
          ? ((IfsException)e).toLocalizedString()
          : e.toString());
public String getDisplayName(LibraryObject lo)
    throws IfsException
    String displayName;
    if (lo != null)
      displayName = lo.getClassObject().getName()
        + " '" + lo.getName() + "'";
    else
      displayName = "<null object>";
    return displayName;
}

Flash player will not play flvs from folder with name containing non-Roman Chars

We're trying to play flash as part of a program that can be installed to locations including non-Roman chars (European, Arabic, Chinese, etc.). When the installation path includes any non Roman chars, video will not play. The non-roman chars are garbled when Flash tries to load the flvs. Correct non-Roman chars are: ńśćłł. When loading video it gets translated to: Ĺ„Ĺ›Ä‡Ĺ‚Ĺ‚
We've found this happens whether we are trying to access the video from the xulrunner app or by taking the activities out and running them within an html frame.
Audio on the other hand is no problem
The path information for the xulrunner app from Process monitor is:
Video not found from the video player.
10:41:25.3337898    xulrunner.exe    2316    CreateFile    C:\Pro\Ĺ„Ĺ›Ä‡Ĺ‚Ĺ‚\Flash dll Test\.shared\content\assets\video\VI-d9e37af511faa\Engplus1.flv    PATH NOT FOUND    Desired Access: Generic Read, Disposition: Open, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, AllocationSize: n/a
10:41:25.3339452    xulrunner.exe    2316    CreateFile    C:\Pro\Ĺ„Ĺ›Ä‡Ĺ‚Ĺ‚\Flash dll Test\.shared\content\assets\video\VI-d9e37af511faa\Engplus1.flv    PATH NOT FOUND    Desired Access: Generic Read, Disposition: Open, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, AllocationSize: n/a
Audio found.
10:41:59.7659648    xulrunner.exe    2316    ReadFile    C:\Pro\ńśćłł\Flash dll Test\.shared\content\assets\audio\AU-fa6ed5d31f577\LG0CD1Track06.mp3    SUCCESS    Offset: 0, Length: 4,096, Priority: Normal
10:41:59.7660482    xulrunner.exe    2316    ReadFile    C:\Pro\ńśćłł\Flash dll Test\.shared\content\assets\audio\AU-fa6ed5d31f577\LG0CD1Track06.mp3    SUCCESS    Offset: 4,096, Length: 4,096
Video not found from a flash interactive
10:43:50.0804572    xulrunner.exe    2316    CreateFile    C:\Pro\Ĺ„Ĺ›Ä‡Ĺ‚Ĺ‚\Flash dll Test\.shared\content\assets\flash_interactive\content\FL-123456780l0l\assets\video\video. flv    PATH NOT FOUND    Desired Access: Generic Read, Disposition: Open, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, AllocationSize: n/a
10:43:50.0806413    xulrunner.exe    2316    CreateFile    C:\Pro\Ĺ„Ĺ›Ä‡Ĺ‚Ĺ‚\Flash dll Test\.shared\content\assets\flash_interactive\content\FL-123456780l0l\assets\video\video. flv    PATH NOT FOUND    Desired Access: Generic Read, Disposition: Open, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, AllocationSize: n/a
Audio found from a flash interactive
10:43:50.1928413    xulrunner.exe    2316    ReadFile    C:\Pro\ńśćłł\Flash dll Test\.shared\content\assets\flash_interactive\content\FL-123456780l0l\assets\audio\journa list.mp3    SUCCESS    Offset: 0, Length: 4,096, Priority: Normal
10:43:50.1929148    xulrunner.exe    2316    ReadFile    C:\Pro\ńśćłł\Flash dll Test\.shared\content\assets\flash_interactive\content\FL-123456780l0l\assets\audio\journa list.mp3    SUCCESS    Offset: 0, Length: 25,206, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal

Not exactly sure what you are doing, but it is obvious that some part of the program you are running does not support multibyte characters like Unicode.
If you are developing that application, make sure it is Unicode enabled. If it is Flash (or Flash Player) that doesn't understand it, then you may not be able to do anything about it.
What I would try if I were to encounter the situation, I would try to use the DOS (8.3) path names, like C:\Documents and Settings would normally be C:\DOCUME~1 or C:\Documents and Settings\All Users would be C:\DOCUME~1\ALLUSE~1 etc.
Use the DIR /X command to find out what the DOS folder names are.

Regex with strings that contain non-latin chars

I am having difficulty with a regex when testing for words that contain non-latin characters (specifcally Japanese, I haven't tested other scripts).
My code:
keyword = StringUtil.trim(keyword);
//if(keywords.indexOf(keyword) == -1)
regex = new RegExp("\\b"+keyword+"\\s*;","i");
if(!regex.test(keywords))
{Alert.show('"'+keywords+'" does not contain "'+keyword+'"'); keywords += keyword + "; ";}
Where keyword is
日本国
and keywords is
Chion-in; 知恩院; Lily Pond; Bridge; 納骨堂; Nōkotsu-dō; Asia; Japan; 日本国; Nihon-koku; Kansai region; 関西地方; Kansai-chihō; Kyoto Prefecture; 京都府; Kyōto-fu; Kyoto; Higashiyama-ku; 東山区; Places;
When the function is run, it will alert that keywords does not contain keyword, even though it does:
"Chion-in; 知恩院; Lily Pond; Bridge; 納骨堂; Nōkotsu-dō; Asia; Japan; 日本国; Nihon-koku; Kansai region; 関西地方; Kansai-chihō; Kyoto Prefecture; 京都府; Kyōto-fu; Kyoto; Higashiyama-ku; 東山区; Places; " does not contain "日本国"
Previously I was using indexOf, which doesn't have this problem, but I can't use that since it doesn't match the whole word.
Is this a problem with my regex, is there a modifier I need to add to enable unicode support or something?
Thanks
Dave

ogre11 wrote:
> I need to use refind to deal with strings containing
accented characters like
> ?itt? l?su, but it doesn't seem to find them. Also when
using it with cyrillic
> characters , it won't find individual characters, but if
I test for [\w] it'll
> work.
works fine for me using unicode data:
<cfprocessingdirective pageencoding="utf-8">
<cfscript>
t="TÃ¡ mÃ© in ann gloine a ithe;
NÃ chuireann sÃ© isteach nÃ³ amach
orm";
s="Ã¡";
writeoutput("search:=#t#<br>for:=#s#<br>found
at:=#reFind(s,t,1,false)#");
</cfscript>
what's the encoding for your data?

UploadedFile and filenames with non-ascii chars

Hi
I'm using an UploadedFile object in my web app, and all works fine. However, when I try to upload a file, with a filename containing non-ascii chars (e.g. Spanish), I see that the getBytes method returns an empty byte array, the filename is not stored correctly (the non-ascii chars are lost, replaced by another representation), and that the content-type is application/octet-stream instead of image/png as supposed to be.
If I rename that same file to have only ascii chars - everything is back to normal.
How can I upload files with non-ascii chars in their name?

Hi, back! Spent a few hours experimenting and found
that everything is working great (including the creation
of international non-ASCII foldernames) when I used
utf-8 encoding in the sieve filters rules for the
the match strings and the folder names... at least
so far so good... for your ref and sorry for bothering.

Detecting non printables characters in a text file

Hi,
I need to remove some non printable characters like tabs, carriage returns, line feeds,.... and so!
i want to do something like
aString.replaceAll(<the non-printable char>, "");

str = str.replaceAll("\\P{Print}+", "");From http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pattern.html, \p{print} is printable characters and \P is the negative of \p.

Inserting strings of printable and non printable characters

I would very much appreciate some help with the following
To handle an interface with a legacy system I need to create strings containing both printable and non-printabel ascii characters. And with non printable characters I mean in particular those in the range of ASCII 128 to 159.
It seems it is not possible to insert a string containting both printable and not printable characters from the afore mentioned range into a VARCHAR2 table column as the following demonstrates:
insert into test values(chr(156)); -- this inserts the 'œ' symbol.
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
┐                  156            1That the the character mapped is shown as '┐' and not 'œ' is not really issue for my application, what is important is that the ASCII value is shown as 156, which is the ASCII code of the character I inserted.
What is however strange (actually probably not strange but has to do with the lack of understanding of the issue at hand) is that substr returns an empty string...
Now I try to insert a concatenated string, first the "non printable" character then a printable character
insert into test values(chr(156)||chr(65));
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
A                   65            1 A                                     65For some reason the not printable character (chr(156)) is now not inserted or at least does not appear when I selected the data from the table, this effect seems to apply to all characters in the range of ASCII 128 to 159 (tried some but not all) However for instance CHR(13) can be inserted as part of a string as shown above .
For our application I really don't care much what character is shown or not show, what is important is that I can retrieve the ASCII value and that this value matches the one I inserted which for some reason does not seem to work.
This seems to be, at least to some extent a character set issue. I have also tested this on a database with character sets set as follows
NLS_CHARACTERSET
WE8MSWIN1252
NLS_NCHAR_CHARACTERSET
AL16UTF16
With WE8MSWIN1252 the described issue does NOT occur, however unfortunately I must use NLS_CHARACTERSET AL32UTF8 which produces the results as described above!
As said any insights would be much appreciated as I am slowly but surely starting to despair.
For completions sake, character sets are set as follows (changing it is NOT an option):
NLS_CHARACTERSET
AL32UTF8
NLS_NCHAR_CHARACTERSET
AL16UTF16
The test table is created as follows
CREATE TABLE TEST
TEST VARCHAR2(1000 BYTE)
Database Version 11.2.0.3.0
Edited by: helios.taraba on Dec 2, 2012 10:18 AM --Added database version
Edited by: helios.taraba on Dec 2, 2012 10:24 AM Added description of test results using NLS_CHARACTERSET WE8MSWIN1252

Hello Orafad,
Thanks for your reply, at least I understand the effects I'm seeing i.e.
+"For multibyte character sets, n must resolve to one entire code point. Invalid code points are not validated, and the result of specifying invalid code points is indeterminate."+
http://docs.oracle.com/cd/E11882_01/server.112/e26088/functions026.htm
You are absolutely right I could use chr(50579) to get the ligature symbol. However as what we are trying to achieve is to implement a legacy interface to a 20+ years old subsystem we are actually not so much interested in the symbol itself but rather in the ascii value of that symbol (156 as you so rightly point out in the win-1252 characterset), this particular field represents the lenght of the message being sent to the subsystem and can vary from decimal 68 to 164 and is also considered in a checksum calculation which is part of the message.
As changing the nls_characterset of the database is not an option I guess I only have one reasonable avenue to resolve this namely to push the functionality to added the "encoded" length of the message (and the calculation of the checksum) to the java driver which is responsible for sending the message (tcp/ip) to the subsystem. Here we should not have any issues adding a byte with the value 156 (or any other for that matter) to the datastream.
Thankfully all other fields have characters with ascii values below 128 and above 31.
I'm going to leave my question as un-answered for a bit longer in the hopes of someone coming up with a golden bullet, although not getting my hopes up.
Thanks, Helios

Excluding all non-printable characters using regexp_replace

Hi All,
I need help in excluding all non-printable charcters using regexp_replace but i was not able to exclude One special character as shown below.
select regexp_replace('¥Tachyon-QX\_4 !? H*'' $(~!#$%^&*()?@#), BA' , '[^[:alnum:] \!\@\#\$\%\^\&\*\_\+\=\:\""\<\>\?\[\]\;\''\,\.\/\\\`\~\?\¥\] [:cntrl:]')
from dual;
REGEXP_REPLACE('%TACHYON-QX\_4!?H*''$(~!#$
%Tachyon-QX\_4 !? H*' $(~!#$%^&*()?@#), BABut problem is having with special character which i am not able to print it here...having char value - 13 and ascii value - 19. I want this value to be excluded as this nonprintable charcater.
Also i need to exclude newlines and print the output in a single line...
Could any one help me in these two requirement..
1. Exclude all non printable characters(Including special character having char value - 13 and ascii value - 19)
2. Exclude all newline characters..
Thanks,
Yasser

How about this?
SQL>select regexp_replace('Tachyon-QX\_4 !? H*'' $(~!#$%^&*()?@#), BA
2 some junk on line 2
3 xyz line 3' , '[^[:print:]]') printable
4 from dual;
PRINTABLE
Tachyon-QX\_4 !? H*' $(~!#$%^&*()?@#), BAsome junk on line 2xyz line 3

Error converting data type varchar to numeric - maybe non printable character?

Hello all. I'm having some trouble converting a column in my table from varchar(50) to decimal(20,4). I've also tried to convert it to int and had no luck. In trying to find the records with non numeric data, I ran this query...
SELECT *
FROM [table]
WHERE [column] not like '%[^0-9]%'
My result set was 32 rows where the field in question had zeros. This doesn't make sense to me. If it was a real zero, it would have been excluded from the query.
So, I flipped it around and ran the following query, and got the same results.
SELECT *
FROM [table]
WHERE [column] = '0'
If it's really a string 0, what would keep it from converting to a numeric type? I'm wondering if there are non printable characters in there because this table was populated via SSIS package using a flat file.
Thanks for any help you can provide!
WeeLass

I have discovered one of the values that will not convert. It is '-1.2317 ' and I tried RTRIM and those trailing spaces stay there.
It has a length of 8, and ASCII value of 45, and converting to varbinary gives 0x2D312E3233313709.
The 45 and 2D both point to a negative sign. This one does indeed have a negative sign, but it's in the front, not trailing. Also, another value that was failing was not negative, but did have the trailing spaces.
Any ideas on how to figure out what those spaces actually are? Hopefully, if I know that, then I can know how to get rid of them.
WeeLass
It may be a hard space
RTRIM trims only soft spaces
try this and see if works
REPLACE(ColumnName,CHAR(160),'')
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page

The InfoCube contains non-cumulative values

Hi,
While creating multicube for inventory on two cubes its askin the below "The InfoCube contains non-cumulative values. A validity table is created for these non-cumulative values, in which the time interval is stored, for which the non-cumulative values are valid.
The validity table automatically contains the "most detailed" of the selected time characteristics (if such a one does not exist, it must be defined by the user, meaning transfered into the InfoCube)", what is that and how to solve this,
Please through some light on this its urgent.
Chandan

Hi,
Your multi cube is probably based on the infociune 0IC_C03 containing non cumulative key figures taht's why you get this message.
You generaly don't have to maintain validity area unless you are in a special configuration (for exemple loading data from two source systems).
UThe following link should give more information about validity area with non cumulative :
[http://help.sap.com/saphelp_nw04/helpdata/en/02/9a6a1c244411d5b2e30050da4c74dc/frameset.htm|http://help.sap.com/saphelp_nw04/helpdata/en/02/9a6a1c244411d5b2e30050da4c74dc/frameset.htm]
Hope this helps.
Cyril

Non Printable Characters in varchar or varchar2 filed

How can I know if a filed has non-printable characters.

An example :
TEST@db102 SQL> insert into test values('aaa'||chr(13)||'bbb'||chr(10)||'ccc');
1 row created.
TEST@db102 SQL> select * from test;
A
bbb
ccc
TEST@db102 SQL> select dump(a) from test;
DUMP(A)
Typ=1 Len=11: 97,97,97,13,98,98,98,10,99,99,99
TEST@db102 SQL>

Select xmlelement of clob containing non-printable chars gives ORA-31011

Similar Messages

Maybe you are looking for