SQL Loader doesn't handle special characters

Hi All:
My DB characterset and NLS characterset is UTF8.
When i tried to insert records with SQL *Loader having special characters, it doesn't get stored correctly, so when i try to get it out thru' SQLPlus Worksheet, it displays garbage characters.
Do we have to set something in the control file?
Pls. help.

You need to create the data file in UTF8 format. For eg. On Win2K, "Save As" in Notepad as UTF8. Or use the uniconv utility to convert the data file to utf8.
Here is an example of a control file
'sqlldr32.cnt'
load data
infile 'sqlldr32.dat'
APPEND into table nls_demo
fields terminated by ',' optionally enclosed by '"'
(myno char(10),
myname char(40),
mydesc char(1000))
null

Similar Messages

Time-dependent Vendor Master & Handling Special Characters

Hi,
I need to extract time-dependent Vendor Master.
1. The data source for 0VENDOR does not have fields to hold the valid date range.
2. Does the Master data in R/3 for Vendors will hold the valid date range?
3. The text for 0VENDOR provides time-dependent, but how to map the valid from and valid to fields?
Handling Special Characters:
We are trying to extract data from Legacy system via DB Connect. The item text field consists of special characters. Of course in BW customization we can specify all the special characters to consider. But the special character we observed is 'square' symbol i.e. 'new line character' in Oracle. We are updating this to an ODS object. When looked at error log, observed that green light for the number of records transferred and updated, but finally when it load into ODS object and activates popping up the error message saying 'could not recognize special character'.
Please help me getting the 2 issues resolved.
Thanks in advance.
Regards,
Sudhakar.

Hi Everyone,
Thanks for inputs on Special characters issue...
Finally resolved with below piece of code in the start routine:
DATA: FLAG,
 OFF TYPE I,
 LEN TYPE I VALUE 1,
 ALLOWED_CHAR(95) VALUE
'0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ`~!@#$%^&*()-_=+ ' &
'abcdefghijklmnopqrstuvwxyz:;<>,.?/|\{}[]"'''.
CONSTANTS: C_CHAR VALUE '-'.
LOOP AT DATA_PACKAGE WHERE NOT /BIC/ZI_DESC IS INITIAL .
 DO.
 IF DATA_PACKAGE-/BIC/ZI_DESC CN ALLOWED_CHAR.
 REPLACE SECTION OFFSET SY-FDPOS LENGTH LEN OF
 DATA_PACKAGE-/BIC/ZI_DESC WITH C_CHAR.
 FLAG = SPACE.
 ELSE.
 FLAG = 'X'.
 ENDIF.
 IF FLAG = 'X'.
 EXIT.
 ENDIF.
 ENDDO.
 MODIFY DATA_PACKAGE.
ENDLOOP.
if abort is not equal zero, the update process will be canceled
ABORT = 0.
I have seen the link sent by 'Eugene Khusainov' today. Thought putting my piece of code that may help others...
Regards,
Sudhakar.

Handling special characters in XML

Hi,
I am using Oracle 10g 'XMLType' datatype to store XML files. Before storing I parse the XML document using Java Xerces Parser. If it parses successfuly, then I perform some business rule execution based on XML file which was parsed. So till this stage there is no problems. But when XML file contains some special characters like copy-paste of some description from MS-Word document into XML tags, then Xerces parser will parse such characters with out any exceptions, but while inserting XML document, Oracle database just throws exception saying unable to handle special characters.. So how to avoid such exceptions or silent such exceptions with any specific settings respect to XMLType datatype in 10g DB.
Please advice!
Arvind Patil - IN

Monica--
In XI 2.0, we've noticed a number of issues processing special characters, primarily caused by the version of JCO that we're running. It sounds like SAP has spent some time in the past few months focusing on these errors, so make sure you're on the most recent patchlevels of all your middleware components, including any of the middleware libraries that BC uses. In XI, we had to update the 3 files that make up the RFC library and JCO library. SDM couldn't update the libraries for us -- we had to manually move the files to the right place.
Escaped XML characters like "&" """ """ were fixed as of JCO 2.0.10 (the current patchlevel on AIX/UNIX), the special character "'" is fixed in the next release, JCO 2.0.11, due out in a few weeks (hotfixes are available). I don't know the equivalent versions on other platforms. By default, XI 2.0 appears to have shipped with JCO 2.0.5. I would expect many XI 3.0 users to also be affected.
This may or may not apply to BC, because I don't know what BC uses to talk to SAP under the covers.
--Dan King
Capgemini

To Handle Special Characters(Guideu0099 ) in MATMAS IDOC fields

Need to handle special characters like Guide, as an attached superscript in MATMAS02/05 IDOC field . The field name is TDLINE in E1MTXLM segment.
As a trial run when these special characters are pasted in the TDLINE field, it throws an error that "the input field contains prohibited characters"
Please let me know if there is any workaround for this.

hi
good
go through these links, i hope these ll help you to solve your problem.
http://help.sap.com/printdocu/core/Print46c/en/data/pdf/CAGTFADMLO/CAGTFADMLO.pdf
http://www.erphome.net/wdb/upload/forum14_f_2908.doc
thanks
mrutyun^

Handle special characters in the attribute name

Hi,
I am generating different view element in WD application dynamically. How to handle special characters other than '-/ABCDEFGHIJKLMNOPQRSTUVWXYZ_0123456789' for the attribute name dynamically?
Thank you, in advance.
Trupti

Going with the obvious response - don't use them?
if you're using dynamic code, there is no reason (other than debug support) to give your created elements any meaningful name.
Just generate a GUID for each new element and use that.
If you need to be able to later search for and update the element a simple lookup table of GUID to reference string should work reasonably well.
Cheers,
Chris

How to handle special characters in SQL statements

How do you handle special charactes in a SQL statement? Here is an example:
update table
set notefield = 'This is Waldo's note'
where keyfield = 7;
Because the database connectivity vi accepts a string datatype wire, the ' in Waldo's note is seen as the end of string and an error is generated.
Is there a way to tell labview that the ' is part of the string and not the string delimiter?
Waldo

If two single quotes don't work, try backslash single quote, like \'
The backslash is often used as an escape character, meaning to treat the next character literally.
- tbob
Inventor of the WORM Global

How to handle special characters in Stage web view with Load URL method?

Hi,
I have Latin special characters in my text. My HTML content contains the text as well as the image. If I use stage webview LoadString, images will not be displayed. But my latin characters are displayed correctly.
If I use LoadURL method, the images are displayed correctly but I have problem in displaying the latin special characters. How to solve this issue?

Don't pass data values in URLs. Whilst it is possible to escape values to make them URL- and APEX parameter-safe, it is much better practice to avoid the problem altogether.
Create a surrogate primary key on the table that is an immutable discrete numeric identifier and only use this as a URL parameter. In target pages/regions, use this PK value to retrieve the required data from the table.

How to handle special characters such as apostrophe, ampersant

Hello All,
Tools:
JSP, JSDK 2
Oracle 8
Issue:
If the user supplies a String value inside a text box which has special characters such as apostrophe, while we try to compare that string with some value in the database, the sql error comes up as the sql string will be broken.
Example:
input string--> ABC D'OLL
sql --> "select * from verndor where vendor_name ='ABC D'OLL' "
Present solution:
I am reading each input string & look for such special characters. if so, replaces that character with escape character + that character.
This solution works.
Problem:
If there any more efficient way to handle such character??
At java level or Database level
Thanks In advance :)

Use PreparedStatements

[Solved] TTY doesn't display special characters correctly

Hello everyone.
I want to set my system language to german. I followed this tutorial: https://wiki.archlinux.de/title/Arch_Li … .BCber_HAL
Everything works good except that my TTY can't display special characters. Example:
http://i.imgur.com/ZW6kbOS.png
The is " not showing correctly at the bottom. However, the german umlaut ä works (as you can see).
I have already searched the arch forum. The only thing i have found is setting DAEMON_LOCALE="yes" in rc.conf. However, these post are already quite old and don't solve the my problem (as öäü are working). Additionally, Arch has stopped using rc.conf.
So, what causes this problem and how can i solve it?
Here are my configs:
/etc/locale.conf
LANG=de_DE.UTF-8
LC=
locale -a
C
de_DE.utf8
en_US.utf8
POSIC
locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=
/etc/vconsole.conf
KEYMAP=de-latin1-nodeadkeys
Note that i am running Arch as guest in a VirtualBox.
-- mod edit: read the Forum Etiquette and only post thumbnails http://wiki.archlinux.org/index.php/For … s_and_Code [jwr] --
Last edited by Oznerol256 (2013-11-01 09:49:08)

I'm pretty sure you have to specify font and font map in vconsole.conf
like so:
FONT=Lat2-Terminus16
FONT_MAP=8859-2
If that doesn't help it could be this.
If so try to load settings manually, I can't test German setup, but for Polish I could do this to get national characters:
loadkeys pl
setfont Lat2-Terminus16 -m 8859-2
that requires to be root/sudo
note that -m 8859-2 part is quite important
edit:
actually got all German symbols to be displayed correctly, though I do not know if they are assigned for right keys for obvious reasons
loadkeys de-latin1-nodeadkeys
setfont Lat2-Terminus16 -m 8859-2
read read topic in then link to hopefuly fix why settings are not loaded on boot
Last edited by nutsh (2013-10-31 02:47:48)

How to Handle Special Characters in PI7.1

Hi Team,
I need to handle some special characters like <,>,& etc.. from WS Adapter to CRM in PI 7.1.
http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420 [original link is broken]
By using the above blog i had implemented the Java Code as
public void execute(InputStream in, OutputStream out){
try{
int read_data;
while((read_data = in.read()) != -1){
if (read_data == '&'){
out.write("&".getBytes());
}else if (read_data == '>'){
out.write(">".getBytes());
}else if (read_data == '<'){
out.write("<".getBytes());
}else if (read_data == '/'){
out.write("&frasl;".getBytes());
}else if (read_data == '\''){
out.write("'".getBytes());
else { out.write(read_data);
out.flush();
} catch (Exception e){}
I had added this class file in Operational Mapping.
It is working if we have only one IF condition for & in while
Any suggestion
Thanks
Sriram

Hi Ramesh,
Thanks for your inputs, I have tried your code but it is not working. The error message stays the same.
Dear Stephane,
To describe the error more, the requirement is the payload coming from source through WS Adapter consists of the special characters <, > , & which are basic sematics of XML syntax. I need PI to process this payload with special characters and successfully transfer it to target CRM system. I have created the Java class with code (ref: Blog 9420) as stated earlier and added this class to my existing Operation Mapping. I am expecting the java mapping to replace the special characters in payload like < with ">" . So as the case with the other characters >,&,'
After activaton of my operation mapping, I triggered the test message with Soap UI client and I could able to get a successful mapping only When I put the logic for &ampersand symbol only. However when I am trying to add the logic for > or < or ' the mapping is failing . I am using UTF-8 encoding across the source and PI enviroments.
Sample SOAP message :
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:abcabca.com">
 <soapenv:Header/>
 <soapenv:Body>
 <urn:MT_ABCDEFG_Req>
 <activity>
 <id/>
 <type>ZEMA</type>
 <actionType>C</actionType>
 <firewall>10000003</firewall>
 <subject>small > < attachment test</subject>
 <location/>
 <startDate>2010-07-08T10:53:31.000Z</startDate>
 <endDate>2010-07-08T10:53:31.000Z</endDate>
 <mainClient>1000319</mainClient>
 <mainContact>1000003</mainContact>
 <isConfidential>false</isConfidential>
 <summary/>
 <fullText>test body - small.txt</fullText>
 <owner>1000021</owner>
 <from>ABCDEDF</from>
 <sendTo>emailaddress</sendTo>
 <copyTo/>
 <keywords/>
 <referenceId/>
 <createdBy>1000021</createdBy>
 <additionalContacts/>
 <additionalClients/>
 <additionalParticipants/>
 <status>A0008</status>
 <attachments>
 <fileUrl>20100708110053-XXXXXXXXX</fileUrl>
 <fileName>small.txt</fileName>
 </attachments>
 <attachments>
 <fileUrl>20100708110053-XXXXXXXXX</fileUrl>
 <fileName>EMail 2010-07-08.pdf</fileName>
 </attachments>
 </activity>
 </urn:MT_ABCDEFG_Req>
 </soapenv:Body>
</soapenv:Envelope>
Output on the SOAP UI client for the above request:

<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
 <SOAP:Body>
 <SOAP:Fault>
 <faultcode>SOAP:Server</faultcode>
 <faultstring>Server Error</faultstring>
 <detail>
 <s:SystemError xmlns:s="http://sap.com/xi/WebService/xi2.0">
 <context>XIAdapter</context>
 <code>ADAPTER.JAVA_EXCEPTION</code>
 <text>com.sap.engine.interfaces.messaging.api.exception.MessagingException: com.sap.engine.interfaces.messaging.api.exception.MessagingException: XIServer:NO_MAPPINGPROGRAM_FOUND:
 at com.sap.aii.adapter.soap.ejb.XISOAPAdapterBean.process(XISOAPAdapterBean.java:1160)
 at sun.reflect.GeneratedMethodAccessor342.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at com.sap.engine.services.ejb3.runtime.impl.RequestInvocationContext.proceedFinal(
What do you think where I am doing the wrong?
Sriram

How to handle special characters in NWDI

Dear All
I am trying to update the Description from JSP form. Through JCO we are calling the RFC of ABAP. We are passing these description from Java to RFC of ABAP and this RFC update the text in Database.
We have noticed that if there is some special character in description like as : or , complete description is not getting updated in to the SAP database.
Can anyone help me how to handle these special characters in Java. There may be N number of special characters. We want to generalize it. We want to replace these characters by s and z respectively.
For example : We want to update this description.
APPLERR H'4301 H'FA03 H'254C na Zagreb TC4 rijeen je cleaning procedurom, te je i kroz CSR odgovoreno da trap korekcija N01TCADG-0052 u bloku UPDC vie nije potrebna, te se moe izbaciti (AP143).
Uspjeno su zavrene HR17/CN-A3 FOA-e na tranzitnom nivou, te slijedi roll-out u dva termina 12/13.04 i 19/20.04. ETK je na sastanku isporučio SW, te ALEX i mini PLEX za sve objekte.
AP147. Poslati finalnu dokumentaciju za uvođenje paketa (implementacijsku instrukciju i sve popratne datoteke).
WHile updated text is as follows :
APPLERR H'4301 H'FA03 H'254C na Zagreb TC4 rije
N01TCADG-0052 u bloku UPDC vi
Uspje
sastanku isporu
AP147. Poslati finalnu dokumentaciju za uvo
Regards
Bhavishya

Hi Bhavishya,
Apparently your SAP database isn't configured to support Unicode. That would be the first solution to your problem, but I can imagine it's a bit drastic to convert your DB.
A second solution would be to encode the input description to ASCII before storing it in the database. When reading from the database, decode again to Unicode. This way, no information is lost. A suitable encoding would be Base64. e.g.
String description = "unday ebra";
String descriptionBase64 = new sun.misc.BASE64Encoder().encode(
description.getBytes("UTF-8")); // ""
// store descriptionBase64 in the DB
// later, when reading descriptionBase64 from the DB
String description2 = new String(
new sun.misc.BASE64Decoder().decodeBuffer(descriptionBase64), "UTF-8");
Instead of using Sun's implementation, a better alternative is to use the open source implementation
org.apache.commons.codec.binary.Base64 from Commons Codec .
The 3rd approach is indeed to normalize the description by replacing all special characters with their ASCII equivalent. A rather easy solution is as follows:
String description = "unday ebra";
String descriptionNormalized = sun.text.Normalizer.normalize(
description, sun.text.Normalizer.DECOMP, 0).replaceAll(
"[^p{ASCII}]", "");
sun.text.Normalizer decomposes the string, e.g. "éàî" becomes "e´a`i^", after which non-ASCII characters are being removed using a regular expression.Again, note that it's usually a bad idea to use sun.* packages, see note about sun.* packages. The above code only works with J2SE 1.4 and J2SE 5.0, but it breaks in J2SE 6.0 (where
java.text.Normalizer became part of the public API ;-). A good open source implementation can be found here: ICU4J (com.ibm.icu.text.Normalizer).
Kind regards,
/Sigiswald

Error while handling special characters ( and &) in Sender File adapter

Dear All,
Scenario: Third Party System --> (File adapter) SAP PI 7.1 (SP05) --> (Proxy) ECC
We are receiving CSV file and no mapping is required in PI as all the processing is being done by third party tool. But we are receiving special characters like ',",>,< and & in the csv file.
When we are using File Type as TEXT and File Encoding as ISO-8859-1 than we are getting error. We changed File Type to BINARY and than we are able to handle ',",> but having issue in handling < and &.
Any idea how we can handle it.
Please note trhere is no Mapping in SAP PI so cannot uise Java mapping.
Thanks a lot
Chanakya

Hey,
You can add ABAP Script in Receiver Proxy for eliminating or replacing with NULL or empty. Since you dont have mapping for formatting these kind of junks you can follow the above approach.
Else, you can remove the junks at OS level while running a small script and make it available to PI.
Or else, you need to create mapping for sender and recevier structure though it is same, then you add a very simple UDF for fine tune.
Use this UDF
Here a is the input value.
String b = "";
b = a.replaceAll(",", "0");
b = a.replaceAll("@", "0");
Like all the special characters you can take and replace with zero.
return b;
Thanx
Pothana

Firefox doesn't reconvert special characters in the file names when download a file with any special characters in the file name

<blockquote>Locking duplicate thread. 
Please continue here: [/questions/815207]</blockquote> 
if i try to download a file with any special characters in file name (e.g. File_Name.pdf), it doesn't reconvert them from the "sanitize url" process and download the file an incorrect name (e.g. File%5FName.pdf).
This is really annoying.
Thank you for your patient

Start Firefox in [[Safe Mode]] to check if one of the extensions is causing the problem (switch to the DEFAULT theme: Firefox (Tools) > Add-ons > Appearance/Themes).
* Don't make any changes on the Safe mode start window.
* https://support.mozilla.com/kb/Safe+Mode
* [[Troubleshooting extensions and themes]]

Oracle SQL query for getting specific special characters from a table

Hi all,
This is my table
Table Name- Table1
S.no Name
1 aaaaaaaa
2 a1234sgjghb
3 a@3$%jkhkjn
4 abcd-dfghjik
5 bbvxzckvbzxcv&^%#
6 ashgweqfg/gfjwgefj////
7 sdsaf$([]:'
8 <-fdsjgbdfsg
9 dfgfdgfd"uodf
10 aaaa bbbbz#$
11 cccc dddd-/mnm
The output has to be
S.no Name
3 a@3$%jkhkjn
5 bbvxzckvbzxcv&^%#
7 sdsaf$([]:'
8 <-fdsjgbdfsg
10 aaaa bbbbz#$
It has to return "Name" column which is having special characters,whereas some special chars like -, / ," and space are acceptable.
The Oracle query has to print columns having special characters excluding -,/," and space
Can anyone help me to get a SQL query for the above.
Thanks in advance.

You can achieve it in multiple ways. Here are few.
SQL> with t
2 as
3 (
4 select 1 id, 'aaaaaaaa' name from dual union all
5 select 2 id, 'a1234sgjghb' name from dual union all
6 select 3 id, 'a@3$%jkhkjn' name from dual union all
7 select 4 id, 'abcd-dfghjik' name from dual union all
8 select 5 id, 'bbvxzckvbzxcv&^%#' name from dual union all
9 select 6 id, 'ashgweqfg/gfjwgefj////' name from dual union all
10 select 7 id, 'sdsaf$([]:''' name from dual union all
11 select 8 id, '<-fdsjgbdfsg' name from dual union all
12 select 9 id, 'dfgfdgfd"uodf' name from dual union all
13 select 10 id, 'aaaa bbbbz#$' name from dual union all
14 select 11 id, 'cccc dddd-/mnm' name from dual
15 )
16 select *
17 from t
18 where regexp_like(translate(name,'a-/" ','a'), '[^[:alnum:]]');
 ID NAME
 3 a@3$%jkhkjn
 5 bbvxzckvbzxcv&^%#
 7 sdsaf$([]:'
 8 <-fdsjgbdfsg
 10 aaaa bbbbz#$
SQL> with t
2 as
3 (
4 select 1 id, 'aaaaaaaa' name from dual union all
5 select 2 id, 'a1234sgjghb' name from dual union all
6 select 3 id, 'a@3$%jkhkjn' name from dual union all
7 select 4 id, 'abcd-dfghjik' name from dual union all
8 select 5 id, 'bbvxzckvbzxcv&^%#' name from dual union all
9 select 6 id, 'ashgweqfg/gfjwgefj////' name from dual union all
10 select 7 id, 'sdsaf$([]:''' name from dual union all
11 select 8 id, '<-fdsjgbdfsg' name from dual union all
12 select 9 id, 'dfgfdgfd"uodf' name from dual union all
13 select 10 id, 'aaaa bbbbz#$' name from dual union all
14 select 11 id, 'cccc dddd-/mnm' name from dual
15 )
16 select *
17 from t
18 where translate
19 (
20 lower(translate(name,'a-/" ','a'))
21 , '.0123456789abcdefghijklmnopqrstuvwxyz'
22 , '.'
23 ) is not null;
 ID NAME
 3 a@3$%jkhkjn
 5 bbvxzckvbzxcv&^%#
 7 sdsaf$([]:'
 8 <-fdsjgbdfsg
 10 aaaa bbbbz#$
SQL>

Query to Handle special characters in the conditions

Hi all,
We have a table for colour codes and there related information but the colour codes have special characters in them like
AL&.MPD
CH(SB00
ECA&BC1
TD..0023
0O'DON4
i need to check if these exist in the table or not but when i query like
select * from art.tb_color_code where color_code in ('AL&.MPD','CH(SB00','ECA&BC1','TD..0023','0O'DON4');
i get prompts for entering the variable values there are 2500 such codes
how do i negate the special meanings of these characters
Regards
Maverick

Hi,
If a string literal contains an apostrophe, use two consecutive apostrophes.
For example:
'0O''DON4'is a seven-character string. The third character is an apostrophe.
As others have said,
SET DEFINE OFFwill disable the special meaning of & in SQL*Plus.

SQL Loader doesn't handle special characters

Similar Messages

Maybe you are looking for