Convert ANSI to UTF-8
hi guys,
recently i import xml file to other system, but the source xml's format is ANSI and maybe contain english,simplified Chinese,Chinese Traditional and so on. so i need to develop a custome receive pipeline for this. and i did it as below
reference,it still can not convert it correctly.
http://blogs.msdn.com/b/apacbiztalk/archive/2013/02/01/how-to-convert-messages-from-ansi-to-utf8-encoding-in-a-pipeline-component.aspx
who can tell me the reason?
Hi,
I doubt its just encoding conversion as you are dealing with NON-ANSI characters (Chinese,Chinese Traditional and so on). I would suggest first you can format the xml than you can go ahead with encoding conversion .
Thanks
Abhishek
Similar Messages
-
What is the best way to import an ANSI as UTF-8 Text file using the import wizard
I'm using SQL Server 2008 and am trying to import a flat txt file that is ANSI as UTF-8 and keep getting error after error on different fields. I'm wondering if there is a series of settings that will give me the best chance of importing this
file. I've inserted a a sample that you can look at.
The row delimiters are SOH and CR LF
Column delimiters are Pipe
This is one record
01|KSB|266916|5/1/2008 0:00:00|9.69|03|OUT|KSB|3|13429|5/1/2008 0:00:00|9.69|5/1/2008 0:00:00|14.56|C|January 28, 200810:47 am Called MARCI from EHRLICH, JERRY S M.D. marci will callme back if she needs any boostrix. She was very busy.Copied: 01/28/08
10.80Copied: 02/28/08 12.53Copied: 03/05/08 10.67 Completion memo:PREBOOKED FLU ALREADY WITH ANOTHER SUPPLIER. NO NEED FOR ANYTHING TODAY.Copied: 04/04/08 9.44 Completion memo:placed order 4/1/08Copied:
05/01/08 14.56 Completion memo:call marci tomorrow|N|N|||||||||N|0
Anytime you see 01| it's the next record
01|KSB|266923|5/1/2008 0:00:00|9.70|03|OUT|KSB|3|6784|5/1/2008 0:00:00|9.70|5/1/2008 0:00:00|12.74|C|July 25, 200704:29 pm Called MARK from INFECTIOUS DISEASE spokt to mark nn. thought he prebooked with us. No order but I sent him FFCopied: 07/25/07
16.50Copied: 10/25/07 15.58Copied: 11/19/07 15.57Copied: 12/04/07 16.08Copied: 01/04/08 9.48Copied: 01/30/08 15.23Copied: 02/25/08 16.53Copied: 03/11/08 10.98Copied: 04/04/08 9.45
Completion memo:placed order 4/1/04Completed: 05/01/08 12.74 Completion memo:placed order today for meds|N|N|||||||||N|0
01|KSB|266935|4/4/2008 0:00:00|9.75|03|OUT|KSB|3|77377|4/4/2008 0:00:00|9.75|4/4/2008 0:00:00|9.60|C|(History Copy)FDA NEWS FLASH from Seacoast MedicalFeds Prescribe "new recipe" for Flu Shot
Copied: 03/05/08 11.20Copied: 04/04/08 9.50 Completion memo:flu rep called todayCompleted: 04/04/08 9.60 |N|N|||||||||N|0
01|KSB|267768|5/30/2008 0:00:00|15.01|03|OUT|KSB|3|6401|5/30/2008 0:00:00|15.01|5/30/2008 0:00:00|10.24|C|July 23, 200710:40 am Called LIZ from MAIN STREET FAMILY HEALTH spoke with Liz and she said to fax her ff menactra and adacel pricingCopied:
09/27/07 14.88Copied: 11/06/07 11.23 Completion memo:left msg for Liz to call me if she needs anything today.Copied: 11/27/07 10.75 Completion memo:No need for anything today.Copied: 01/28/08 15.30 Completion
memo:LEFT MSG FOR LIZ TO CALL ME IF SHE NEEDS ANYTHIN TODAY.Copied: 02/29/08 12.66Copied: 04/07/08 14.76 Completion memo:spoke to liz says she orders from mckesson but I can fax her pricing.Copied: 05/30/08 10.24
|N|N|||||||||N|0
01|KSB|267880|5/7/2008 0:00:00|15.93|03|OUT|KSB|3|13649|5/7/2008 0:00:00|15.93|5/12/2008 0:00:00|12.73|C|July 11, 200704:17 pm Called INGA from ASSOCIATES IN FAMILY PHYSICIAN left msg on vm about Mnactra and AdacelCopied: 07/11/07 16.30Copied:
08/06/07 14.89Copied: 10/25/07 14.37Copied: 11/27/07 16.37 Completion memo:left msg for denise to call me if she needs anything today.Copied: 01/04/08 11.47 Completion memo:left msg for Denise to call me if she needs
anything today.Copied: 02/05/08 11.66 Completion memo:LEFT MSG FOR D HUNT ABOUT FLU AND OTHER MEDS.Copied: 03/10/08 11.33Copied: 04/07/08 15.68 Completion memo:lef msg on vm to see if she needs anything. faxing her
pricingCopied: 05/12/08 12.73 Completion memo:LEFT MSG FOR DEBBIE TO CALL ME IF SHE NEEDS ANYTHING TODAY.|N|N|||||||||N|0
01|KSB|267942|5/7/2008 0:00:00|16.41|03|OUT|KSB|3|16945|5/7/2008 0:00:00|16.41|5/7/2008 0:00:00|14.81|C|January 8, 200811:10 am Called MARY from INTERNAL MEDICINE PHYSICIANSCopied: 01/09/08 12.44Copied: 03/31/08 10.84 Completion memo:Sent
email that nail clippers are in.Copied: 04/07/08 16.16 Completion memo:PLACED 3 ORDERS ON 4/7/08Copied: 05/07/08 14.81 Completion memo:ORDER PLACED 5/1/08|N|N|||||||||N|0
01|KSB|267949|4/21/2008 0:00:00|16.44|02|OUT|KSB|3|11700|4/21/2008 0:00:00|16.44|4/22/2008 0:00:00|9.79|C|March 4, 200802:35 pm Called DEBBIE/Gail from BURKERT,THOMAS S. ,MD SPOKE TO DEBBIE AND SHE SAID TO FAX HER PREBOOK FORMCopied: 03/04/08 14.60Copied:
04/07/08 16.19Completed: 04/22/08 9.79 Completion memo:PREBOOKED FLU ON 4/8/08|N|N|||||||||N|0
01|KSB|269137|5/9/2008 0:00:00|15.29|03|OUT|KSB|3|16489|5/9/2008 0:00:00|15.29|5/13/2008 0:00:00|10.51|C|July 17, 200709:47 am Called MICHAEL from MEDICAL ASSOCIATES DANBURY HOSCopied: 10/26/07 14.64Copied: 11/28/07 15.16 Completion memo:LEFT
MSG FOR MICHAEL TO CALL ME IF SHE NEEDS ANYTHING.Copied: 0101|KSB|270740|4/14/2008 0:00:00|16.42|03|OUT|KSB|3|2224|4/11/2008 0:00:00|16.42|4/14/2008 0:00:00|14.07|C|April 11, 200804:10 pm Called BECKI from AHC - WOMEN'S HEALTH CARE BECKI IS OFF TODAY.
CAAL BACK 4/14/08Copied: 04/14/08 14.07 Completion memo:ORDER PLACED 4/9/08|N|N|||||||||N|0
anything today|N|N|||||||||N|0
01|KSB|270816|5/14/2008 0:00:00|9.26|03|OUT|KSB|3|12320|5/14/2008 0:00:00|9.26|5/14/2008 0:00:00|10.78|C|August 7, 200710:35 am Called DR. from ALIG, HOWARD M., MD INC SPOKE WITH KATHY FOR A 3RD TIME. SAYS SHE OPEN THE BOX AND WAS MISSING ONE VIAL OF
DEPO-TESTOSTERONE. BOB AND DIANE CHECK IT OUT AND ALL DEPO QUANTATIES SENT OUT THAT DAY ALL MATCHED UP, AND THEY SHOULD OF GOT 3 VIALS. KATHY SAID SHE WOULDNT LIE AND COULDN'T USE THEM FOR HERSELF. TOLD HER I DO CFA AS PER DCF. SHE SAID OK.Copied:
08/07/07 10.66Copied: 10/26/07 9.79 Completion memo:left msg for kathy to call me if she needs anythingCopied: 11/27/07 10.38 LEFT MSG TO SEE IF SHE NEEDS ANYTHING.Copied: 01/10/08 8.76 TOLD KATHY TO CALL ME IF
SHE NEEDS ANYTHING TODAY.Copied: 03/10/08 15.69 Completion memo:SPOKE WITH RECPT AND SAID TO FAX HER UPDATED PRICING.Copied: 04/14/08 9.01 Completion memo:spoke with kathy's recpt and she said to fax over pricing.Copied:
05/14/08 10.78 Completion memo:KATHY NOT IN. CALL BACK TOMORROW|N|N|||||||||N|0
01|KSB|270822|5/14/2008 0:00:00|9.27|03|OUT|KSB|3|2213|5/14/2008 0:00:00|9.27|5/14/2008 0:00:00|11.49|C|March 6, 200809:52 am Called KELLY from MICHELS, DALE E MD CALL KELL TOMORROW AT 402 488-7400Copied: 03/06/08 9.88Copied: 03/10/08
15.70Copied: 04/14/08 9.02 Completion memo:placed order 3/26/08Copied: 05/14/08 11.49 Completion memo:placed order 5/14/08|N|N|||||||||N|0
01|KSB|270830|4/14/2008 0:00:00|9.30|03|OUT|KSB|3|6250|4/14/2008 0:00:00|9.30|4/14/2008 0:00:00|10.75|C|September 19, 200709:54 am Called TINY from WANG, SHIUSH C. MD SPOKE TO TINY. NO NEED FOR ANYTHING TODAYCopied: 09/19/07 9.91Copied:
11/05/07 9.65 NO NEED FOR ANYTHING TODAYCopied: 11/05/07 9.66Copied: 12/12/07 15.58Copied: 01/04/08 14.12Copied: 01/14/08 14.76 Completion memo:SPOKE TO TINY. NO NEED FOR ANYTHING TODAY.Copied:
02/20/08 11.18 Completion memo:LEFT MSG FOR TINY TO CALL ME IF SHE NEEDS ANYTHING.Copied: 03/27/08 10.19 Completion memo:PLACED ORDER 3/12/08Copied: 04/14/08 9.05 Completion memo:placed order 3/28/08Copied:
04/14/08 10.75 |N|N|||||||||N|0
01|KSB|270839|5/5/2008 0:00:00|9.39|03|OUT|KSB|3|6987|4/24/2008 0:00:00|9.39|5/5/2008 0:00:00|9.59|C|October 10, 200708:55 am Called Karen from FAMILY MEDICINE OF GREENHILL SPOKE TO LYNETTE AND SHE SAID FAX HER THE PREBOOK FORM FOR FLUVIRIN.Copied:
10/10/07 8.94Copied: 11/09/07 11.37Copied: 12/18/07 14.89Copied: 01/18/08 10.45 SPOKE TO LYNETTE. NO NEED FOR ANYTHING TODAYCopied: 02/22/08 11.17Copied: 03/28/08 10.32 Completion memo:CALL BACK 4/12/08Copied:
04/14/08 9.14 Completion memo:spoke with rep 4/3/08Copied: 05/05/08 9.59 |N|N|||||||||N|0
01|KSB|270841|4/23/2008 0:00:00|9.40|03|OUT|KSB|3|2224|4/23/2008 0:00:00|9.40|4/23/2008 0:00:00|11.26|C|September 14, 200702:18 pm Called BECKI from AHC - WOMEN'S HEALTH CARE Spoke to Sue. No need for anything todayCopied: 09/14/07 14.31Copied:
11/05/07 10.59Copied: 11/05/07 11.23Copied: 12/12/07 15.82Copied: 01/11/08 16.24 Completion memo:CALL BACK 1/14/08Copied: 01/14/08 15.27 Completion memo:left msg for Sue to call me if she needs anythingCopied:
02/20/08 14.43 Completion memo:spoke to becky. no need for anythingCopied: 03/27/08 11.89Copied: 03/31/08 01|KSB|270876|5/14/2008 0:00:00|9.87|03|OUT|KSB|3|81523|5/14/2008 0:00:00|9.87|5/14/2008 0:00:00|11.74|C|March 26, 200802:26
pm Called Kathy from FRONTIER HEALTHCARE RECPT GAVE CATHY THE MSG YESTRDAY ABOUT THE CORRECT EMAIL ADDRESS(dcf). SHE IS WITH A PATIENT AND WILL CALL BACK.Copied: 03/26/08 14.46Copied: 04/14/08 9.62 Completion memo:placed order
4/08/0/8Copied: 05/14/08 11.74 Completion memo:PLACED ORDER 5/6/08|N|N|||||||||N|0
memo:PLACED ORDER 4/21/08|N|N|||||||||N|0
01|KSB|270887|5/5/2008 0:00:00|9.89|03|OUT|KSB|3|6526|5/5/2008 0:00:00|9.89|5/6/2008 0:00:00|10.46|C|July 13, 200709:49 am Called ANU from SIRIKONDA, PURNACHANER MD ANU ON VACY UNTIL 7/26. FAXED JUY/AUG SPECIALCopied: 07/13/07 9.82Copied:
08/07/07 14.84 Completion memo:SPOKE TO ANU AND SHE SAID SEND HER INFO ON FLU AND OTHER VACCINESCopied: 10/29/07 11.08 Completion memo:No need for anything today.Copied: 12/04/07 10.69 Completion memo:CALL BACK
AFTER 3PCopied: 12/12/07 14.34 Completion memo:ANU SAID NO NEED FOR ANYTHING TODAY.Copied: 01/11/08 15.78 Completion memo:NO NEED. SEND PRICING ON MED SUPPLIESCopied: 01/14/08 15.20Copied: 02/21/08
9.45Copied: 03/26/08 14.52 CALL BACK NEXT MONTHCopied: 04/14/08 9.64 Completion memo:placed order 4/4/08Copied: 05/06/08 10.46 |N|N|||||||||N|0
01|KSB|270912|4/14/2008 0:00:00|10.27|03|OUT|KSB|3|2229|4/14/2008 0:00:00|10.27|4/14/2008 0:00:00|10.03|C|October 12, 200711:25 am Called BETTY from HEALTH CARE PROFESSIONALS they will call back if they need any flu vaccineCopied: 10/12/07 11.42Copied:
11/12/07 9.37 Completion memo:Spoke to Betty. No need for anything today.Copied: 12/04/07 16.10Copied: 01/04/08 14.22Copied: 02/14/08 11.28Copied: 03/24/08 10.00 Completion memo:placed ordered 3/14/08Copied:
04/14/08 10.02 Completion memo:betty said no need today. Fax scaleCopied: 04/14/08 10.03 Completion memo:placed order for scale|N|N|||||||||N|0
01|KSB|270914|5/14/2008 0:00:00|10.28|03|OUT|KSB|3|2229|5/14/2008 0:00:00|10.28|5/14/2008 0:00:00|11.80|C|October 12, 200711:25 am Called BETTY from HEALTH CARE PROFESSIONALS they will call back if they need any flu vaccineCopied: 10/12/07 11.42Copied:
11/12/07 9.37 Completion memo:Spoke to Betty. No need for anything today.Copied: 12/04/07 16.10Copied: 01/04/08 14.22Copied: 02/14/08 11.28Copied: 03/24/08 10.00 Completion memo:placed ordered 3/14/08Copied:
04/14/08 10.02 Completion memo:betty said no need today. Fax scaleCopied: 04/14/08 10.03 Completion memo:placed order for scaleCopied: 05/14/08 11.80 Completion memo:PLACED ORDER 5/08/0/8|N|N|||||||||N|0
01|KSB|270944|4/14/2008 0:00:00|10.59|03|OUT|KSB|3|1387|4/14/2008 0:00:00|10.59|4/14/2008 0:00:00|10.45|C|December 3, 200710:54 am Called PAT from NEBRASKA SPINE CENTER LLP SPOKE TO PAT. NO NEED FOR ANYTHING TODAY.Copied: 12/03/07 10.90Copied:
01/04/08 13.74Copied: 01/17/08 10.68Copied: 02/13/08 10.86Copied: 03/14/08 10.69 Completion memo:SPOKE WITHE BRENDA 3/5/08Copied: 04/14/08 10.34 Completion memo:LEFT MSG FOR PAT TO CALL ME IF SHE NEEDS ANYTHING.
FAXED SCALE INFO.Copied: 04/14/08 10.45 Completion memo:spoke to Pat no need today. Fax her scale pricing.|N|N|||||||||N|0
01|KSB|270960|5/14/2008 0:00:00|10.71|03|OUT|KSB|3|1387|5/14/2008 0:00:00|10.71|5/14/2008 0:00:00|11.81|C|December 3, 200710:54 am Called PAT from NEBRASKA SPINE CENTER LLP SPOKE TO PAT. NO NEED FOR ANYTHING TODAY.Copied: 12/03/07 10.90Copied:
01/04/08 13.74Copied: 01/17/08 10.68Copied: 02/13/08 10.86Copied: 03/14/08 10.69 Completion memo:SPOKE WITHE BRENDA 3/5/08Copied: 04/14/08 10.34 Completion memo:LEFT MSG FOR PAT TO CALL ME IF SHE NEEDS ANYTHING.
FAXED SCALE INFO.Copied: 04/14/08 10.45 Completion memo:spoke to Pat no need today. Fax her scale pricing.Copied: 05/14/08 11.81 Completion memo:PLACED ORDER 5/9/08|N|N|||||||||N|0
01|KSB|271011|5/14/2008 0:00:00|10.98|03|OUT|KSB|3|12692|5/14/2008 0:00:00|10.98|5/14/2008 0:00:00|9.74|C|September 19, 200709:43 am Called JANET from FAMILY PRACTICE G01|KSB|271065|5/14/2008 0:00:00|11.33|03|OUT|KSB|3|82059|5/14/2008 0:00:00|11.33|5/14/2008
0:00:00|12.00|C|December 4, 200702:29 pm Called Karen from HENRIETTA JOHNSON MEDICAL CTR LEFT MSG FOR KAREN TO CALL ME IF SHE NEEDS ANYTHING.Copied: 12/04/07 14.49Copied: 01/04/08 14.20Copied: 01/11/08 15.60 Completion memo:number
is disconnectedCopied: 03/13/08 11.68 Completion memo:placed orderCopied: 04/14/08 11.08 Completion memo:faxed over pricing on scale and April special.Copied: 05/14/08 12.00 Completion memo:FLU REP CALLED
5/14/08|N|N|||||||||N|0
mo:PLACED ORDER 3/12/08Copied: 04/14/08 9.05 Completion memo:placed order 3/28/08Copied: 04/14/08 10.75 placed order 4/08/08Copied: 05/08/08 9.47 Completion memo:PLACED ORDER 5/2/08|N|N|||||||||N|0
01|KSB|271245|5/14/2008 0:00:00|14.27|03|OUT|KSB|3|82241|5/14/2008 0:00:00|14.27|5/14/2008 0:00:00|16.15|C|December 5, 200708:49 am Called TERESA from COVENANT CLINIC MEDICAL ASSOC CALL BACK 12/7Copied: 12/05/07 8.82Copied: 01/18/08
9.19 Completion memo:LEFT MSG FOR TERESA TO CALL ME IF SHE NEEDS ANYTHING TODA7Y.Copied: 02/04/08 15.79 CB 2/5/08Copied: 02/28/08 14.28 Completion memo:SPOKE WITH MARC AND HE IS FAXING OVER DEA LICENSESCopied:
03/11/08 8.83 Completion memo:PLACING ORDER ONLINE.Copied: 04/14/08 14.02 Completion memo:NO NEED FOR ANYTHING RIGHT NOW.Copied: 05/14/08 16.15 Completion memo:Spoke with Marcus Sowinski, who took over Marc
Simpsons place. Said he will go on the website and check our pricing. I will call hin in 2 weeks to see if he needs anything, or if he needs help.|N|N|||||||||N|0
01|KSB|271293|4/18/2008 0:00:00|14.71|03|OUT|KSB|3|11658|4/18/2008 0:00:00|14.71|4/21/2008 0:00:00|9.73|C|September 14, 200710:32 am Called CECILIA from MEDHEALTH-SGP LEFT MSG FOR CECILIA TO CALL ME IF SHE NEEDS ANYTHING.Copied: 09/14/07 10.54Copied:
11/02/07 11.28Copied: 12/03/07 15.09 Completion memo:LEFT MSG FOR CECILIA TO CALL ME IF SHE NEEDS ANYTHING.Copied: 01/04/08 10.75Copied: 01/30/08 15.29Copied: 03/03/08 13.83 Completion memo:Sent email to
see if they need anything.Copied: 03/28/08 14.28 Completion memo:emailed formulariesCopied: 04/14/08 14.46Copied: 04/21/08 9.73 Completion memo:left msg for Cecilia to call me if she needs anything today.|N|N|||||||||N|0
01|KSB|271352|5/14/2008 0:00:00|15.55|03|OUT|KSB|3|11520|5/14/2008 0:00:00|15.55|5/15/2008 0:00:00|10.29|C|October 18, 200709:35 am Called Amy from ATRIUM OB/GYN LEFT MSG FOR AMY TO SEE IF THEY NEED ANYMORE FLU VACCINE.Copied: 10/18/07 9.60Copied:
11/26/07 14.17 Completion memo:LEFT MSG FOR AMY TO CALL ME IF SHE NEEDS ANYMORE FLU VACCINE.Copied: 01/09/08 14.73 Completion memo:left msg for Amy to call me if she needs anything today.Copied: 02/15/08 9.10
Completion memo:LEFT MSG FOR AMY TO CALL ME IF SHE NEEDS TO PREBOOK FLU VACCINE.Copied: 03/14/08 10.89 Completion memo:LEFT MSG ABOUT M/S.Copied: 04/14/08 15.30 Completion memo:spoke to amy. No need for anything today
but fax her scale info.Copied: 05/15/08 10.29 Completion memo:left msg for any to call me if she needs anything today.|N|N|||||||||N|0
01|KSB|271550|4/16/2008 0:00:00|10.49|03|OUT|KSB|3|15520|4/16/2008 0:00:00|10.49|4/18/2008 0:00:00|10.26|C|March 5, 200810:01 am Called Marty from AICHELE & FREY FAMILY PRACTICE LEFT MSG ON MARTI'S VM TO SEE IF SHE WANTS TO PREBOOK FLU. FAXED FF.Copied:
03/10/08 10.48 Completion memo:cbCopied: 04/15/08 10.24 Completion memo:marty off today. call back tomorrow.Copied: 04/18/08 10.26 Completion memo:SPOKE WITH MARTY. NO NEED FOR ANYTHING TODAY.|N|N|||||||||N|0
Thanks!For this I think you may be better off creating SSIS package from Business Intelligence Development Studio rather than using Export Import wizard. You need to convert the code page of columns from 1252 to 65001 for saving it to UTF 8 file. In SSIS you
can use a derived column task for that.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs -
Hi,
We are recieving files as ANSI for UTF-8 and when we get this file from MQ JMS adapter we get the message: com.sap.aii.utilxi.misc.api.BaseRuntimeException:Invalid byte 2 of 2-byte UTF-8.
If we manually convert file to UTF-8 it works ok.
We are using CCSID 1143 adn we tried others to like 1208, 1140.
Any help appriciated.
Tonywas a server problem, not SAP
-
LPX-00200: could not convert from encoding UTF-8 to UCS2
Dear Gurus,
I am facing this error while I am parsing an XML File.
My Database settings are
NLS_LANGUAGE -- AMERICAN
NLS_CHARACTERSET - AL32UTF8
NLS_NCHAR_CHARACTERSET - AL16UTF16
Database version is 10gr2. (10.2.0.3.0 )
My XML File is having version encoding given below
<?xml version="1.0" encoding="ISO-8859-1" ?>
If I convert encoding to UTF-8 Then the XML Parsing is doing fine, but the special characters are like Ö ß converted to some other Characters like öß, which I don't want.
Appreciate any help.
Regards
Madhu KI have been across the this problem. I solve the problem that come to original ml file. '-' character cause this problem:
<?xml version="1.0" ?>
- <ROWSET>
- <ROW>
<DEPTNO>10</DEPTNO>
<DNAME>ACCOUNTING</DNAME>
<LOC>NEW YORK</LOC>
</ROW>
- <ROW> -
XML: LPX-00200: could not convert from encoding UTF-8 to UCS2
Hi,
Greetings!
I have special character(s) in a column and that character is chr(189) and because of that when i use the xml functions in my query it returns below error.
ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00200: could not convert from encoding UTF-8 to UCS2
Error at line 1
ORA-06512: at "SYS.XMLTYPE", line 0
ORA-06512: at line 1
I am using sys_xmlagg and getting above error when i encounter the data as below:
"Dixon¿s Chicago".
Note: When ever It encounters the bold character string it fails ... Any help !!!
And one more thing when i create another record with same data by copy and pasting it works fine and when i did a dump on that column data its different. see the below result of dump.
Naveen.
SQL> desc temp_xml;
Name Null? Type
TNO NUMBER(4)
NAME VARCHAR2(255)
SQL> select name,length(name),dump(replace(name,chr(189),'')) data_dmp from temp_xml;
NAME LENGTH(NAME) DATA_DMP
¿s Chicago 10 Typ=1 Len=12: 239,191,189,115,32,67,104,105,99,97,103,111
¿s Chicago 10 Typ=1 Len=11: 194,191,115,32,67,104,105,99,97,103,111
SQL>
if you observe the above 2 rows the fist row shows length as 12 and second shows as 11. actually 2nd rows works fine but first gives error. I am not able to see where that hidden character is and not able to remove that character.
Message was edited by:
naveenhksHi,
I have a similar problem:
ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00200: could not convert from encoding UTF-8 to ISO-8859-1I'm executing the following Select when encountering this error:
SELECT /*+ INDEX(resource_view XDBHI_IDX) */
extract(resource_view.res, '/Resource/Contents/*').getClobVal() AS Dokument
FROM resource_view
WHERE resource_view.any_path LIKE '%PATH_TO_FILE%';I have 5 XML-documents and this error occurs at two ('A' and 'B') of them. When I transfer the same 5 documents from another PC the error occurs at document 'C' and not at 'A' and 'B'.
Any clue or hint which could explain this behaviour? What NLS parameters can I check in order to help you understand the situation?
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for IBM/AIX RISC System/6000: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production -
LPX-00200: could not convert from encoding UTF-8 to UCS2 during
When doing an query with SQL/XML on a database with the following characterics
NLS_LANGUAGE -- AMERICAN
NLS_CHARACTERSET - UTF8
NLS_NCHAR_CHARACTERSET - AL16UTF16
Database version is 9.2.0.5.0
I get the following error on a text with special characters
ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00200: could not convert from encoding UTF-8 to UCS2
I used the tip from another developer to use the convert function :
XMLForest(convert(train.programma, 'UTF8', 'WE8ISO8859P1') AS "programma"
Now at least I don't get the error but in the resulting all special characters (and there are a lot of them!) are garbled
So for example (hope this comes across during the post)
original : coördinatoren
in xml : coördinatoren
Any suggestions here? I'm not really allowed to change any database-parameters or doing an upgrade
Maybe using something else in the convert function?Here's the complete query (with the convertfunction in it). Hope this helps
select XMLElement("lmsImport", XMLAttributes('LmsImportSchema.xsd' as "xsi:noNamespaceSchemaLocation",
'http://www.w3.org/2001/XMLSchema-instance' as "xmlns:xsi"),
XMLElement("trainingen",(
Select XMLAGG(XMLElement("training",
XMLElement("id", train.id),
XMLElement("code", train.code),
XMLElement("leverancier",
XMLElement("leverancier",
XMLATTRIBUTES(NVL(train.leverancier, 'vendr000000000001021') as "leverancierId"))),
XMLElement("status", (CASE when TRUNC(NVL(train.disc_from, sysdate)) >= TRUNC(sysdate) then
'Actief'
else
'Inactief'
END )),
XMLElement("naam", train.naam),
XMLForest(
XMLForest(Decode(train.afronding, 'Bewijs van Deelname', 'Instituutsdiploma', train.afronding) AS "afronding") as "afrondingen"),
XMLForest(
XMLForest(train.lesmethode AS "lesmethode") as "lesmethodes"),
XMLForest(
XMLForest(convert(train.programma, 'UTF8', 'WE8ISO8859P1') AS "programma") as "memos"),
XMLForest(train.lesduur AS "lesduur")).extract('/*'))
FROM (select cours.id id, cours.course_no code, cours.custom2 leverancier, cours.disc_from disc_from, cours.title naam, 'Training' lesmethode,
cours.desc1||cours.desc2||cours.desc3||cours.desc4 programma, ROUND(cours.num_days) lesduur, ddcus.str_value afronding
from tpt_courses cours
inner join fgt_domain domin
on cours.split = domin.id
left join (fgt_dd_custom ddcus
inner join fgt_dd_domain_to_attr ddoat
on ddoat.id = ddcus.attr_id
on ddcus.owner_id = cours.id
where (domin.name = 'Content' or domin.name like 'CT%')
and ddoat.attr_id = 'ddatr900000000000009'
union
select prdct.id id, prdct.part_no code, prdct.vendor_id, prdct.disc_from disc_from, prdct.name naam,
decode(prdct.equip_cat_id, 'eqcat000000000000005', 'WBT', 'eqcat000000000001006', 'Book', 'eqcat000000000001019', 'Training', 'eqcat000000000001037', 'Training') lesmethode,
prdct.desc1||prdct.desc2||prdct.desc3||prdct.desc4, null, custom7
from tpt_product_catalog prdct, fgt_domain domin
where prdct.split = domin.id
and (domin.name = 'Content' or domin.name like 'CT%')) train )).extract('/*')).extract('/*') from dual; -
Help on - convert ansi sql to oracle sql
hi gurus,
i'm try'g to convert ansi sql to oracle sql.
but i'm getting an error. can u please let me know, if i can convert it or is it better to use ansi sql!
original code in ansi format::
select distinct bfc.NBR_SEQ, bfc.IDN_ENTITY, bfc.CDE_TYPE_ENTITY, n.CDE_STATUS,
gec.idn_group as idn_parent_id, bfc.IDN_FUNC_BUSS,
case when bfc.CDE_TYPE_ENTITY = 'P' OR bfc.CDE_TYPE_ENTITY = 'A' then
PKG_FW_NVGTR.GET_NAM_PAGE(bfc.IDN_ENTITY)
else
PKG_FW_NVGTR.GET_NAM_GROUP(bfc.IDN_ENTITY)
end as ENTITY_NAM,
p.Category, p.Display_Txt, p.show_in_nav, p.page_uri
from
T_BUSS_FUNC_ENTITY_CREF bfc
inner join t_buss_func bf on bfc.idn_func_buss = bf.idn_func_buss
inner join t_entity_prog_cref epc on bfc.idn_entity = epc.idn_entity and bfc.cde_type_entity = epc.cde_type_entity
left outer join t_page p on bfc.idn_entity = p.idn_page
left outer join t_group_entity_cref gec on bfc.idn_entity = gec.idn_entity and bfc.cde_type_entity = gec.cde_type_entity
left outer join t_nvgtr n on bfc.idn_entity = n.idn_entity and bfc.cde_type_entity = n.cde_type_entity
where
BF.nam_func_buss = 'CP' AND--P_NAM_FUNC_BUSS and
epc.cde_program in
( select p.cde_program from t_program p where p.nam_program in (
SELECT * FROM TABLE (CAST(PKG_FW_NVGTR.FN_GET_ARRAY_FROM_COMMA_LIST('LC', ',') AS TYP_ARRAY))))
order by
bfc.NBR_SEQ;tried to convert into oracle
SELECT DISTINCT bfc.NBR_SEQ,
bfc.IDN_ENTITY,
bfc.CDE_TYPE_ENTITY,
n.CDE_STATUS,
gec.idn_group as idn_parent_id,
bfc.IDN_FUNC_BUSS,
case when bfc.CDE_TYPE_ENTITY = 'P' OR bfc.CDE_TYPE_ENTITY = 'A' then
PKG_FW_NVGTR.GET_NAM_PAGE(bfc.IDN_ENTITY)
else
PKG_FW_NVGTR.GET_NAM_GROUP(bfc.IDN_ENTITY)
end as ENTITY_NAM,
p.Category,
p.Display_Txt,
p.show_in_nav,
p.page_uri
FROM T_BUSS_FUNC_ENTITY_CREF bfc,
T_BUSS_FUNC bf,
T_ENTITY_PROG_CREF epc,
T_PAGE p,
T_GROUP_ENTITY_CREF gec,
T_NVGTR n
WHERE bfc.IDN_FUNC_BUSS = bf.IDN_FUNC_BUSS
AND bfc.IDN_ENTITY = epc.IDN_ENTITY
AND bfc.CDE_TYPE_ENTITY = epc.CDE_TYPE_ENTITY
AND bfc.IDN_ENTITY(+) = p.IDN_PAGE
AND bfc.IDN_ENTITY(+) = gec.IDN_ENTITY
AND bfc.CDE_TYPE_ENTITY(+) = gec.CDE_TYPE_ENTITY
AND bfc.IDN_ENTITY(+) = n.IDN_ENTITY
AND bfc.CDE_TYPE_ENTITY(+) = n.CDE_TYPE_ENTITY
AND BF.nam_func_buss = 'CP' AND--P_NAM_FUNC_BUSS and
epc.cde_program in
( select p.cde_program from t_program p where p.nam_program in (
SELECT * FROM TABLE (CAST(PKG_FW_NVGTR.FN_GET_ARRAY_FROM_COMMA_LIST('LC', ',') AS TYP_ARRAY))))
order by
bfc.NBR_SEQ;error is
ORA-01417: a table may be outer joined to at most one other tableso how can i convert it?
thanksuser642856 wrote:
explain plan for this select statement
ID PID Operation Name Rows Bytes Cost CPU Cost IO Cost Temp space IN-OUT PQ Dist PStart PStop
0 SELECT STATEMENT 10M 1183M 570694 33G 567917
1 0 SORT UNIQUE 10M 1183M 287940 17G 286502 2607M
2 1 HASH JOIN 10M 1216M 179 1G 84
3 2 COLLECTION ITERATOR PICKLER FETCH PKG_FW_NVGTR.FN_GET_ARRAY_FROM_COMMA_LIST
4 2 HASH JOIN OUTER 2693 299K 58 34M 55
5 4 HASH JOIN OUTER 37 3959 11 24M 9
6 5 HASH JOIN OUTER 37 3663 10 18M 8
7 6 HASH JOIN 37 1147 7 12M 6
8 7 HASH JOIN 59 1180 5 6060823 4
9 8 NESTED LOOPS 2 24 3 15843 3
10 9 TABLE ACCESS BY INDEX ROWID T_BUSS_FUNC 1 6 1 8341 1
11 10 INDEX UNIQUE SCAN UK_NAM_FUNC_BUSS 1 0 1050 0
12 9 TABLE ACCESS FULL T_PROGRAM 2 12 2 7501 2
13 8 INDEX FULL SCAN PK_T_ENTITY_PROG_CREF 59 472 1 18921 1
14 7 TABLE ACCESS FULL T_BUSS_FUNC_ENTITY_CREF 37 407 2 14891 2
15 6 TABLE ACCESS FULL T_PAGE 26 1768 2 14141 2
16 5 INDEX FULL SCAN UK_UNIQUE_GROUP_DEPENDENT 26 208 1 12321 1
17 4 TABLE ACCESS FULL T_NVGTR 6986 47K 46 3179613 46 as you can see.. cpu cost is in 33G..
is there any better way?
thanksCan you run this in a session please:
ALTER SESSION set optimizer_dynamic_sampling = 4;Then run an explain plan again.
And you have a function in there - PKG_FW_NVGTR.FN_GET_ARRAY_FROM_COMMA_LIST - which isn't helping things. -
Regarding Converting file from UTF-8 encoding to ANSI
Hi Experts!
I have a problem that my given file is in encoding UTF-8 and i have to convert it into ANSI first so that i can use my bdc session to upload the file.
Kindly if you can help me out with that.If there is any fm available to do that.
thanks in advance
jawadhi Jawad,
place the matter u want to convert in the c program given in the following link it will solve your problem
http://www.codeguru.com/forum/archive/index.php/t-288665.html
Reward if useful -
Convert data from ANSI to UTF-8 format
Hello,
We have data in ANSI format (binary) which we need to convert into UTF-8 format. We are able to do it using FM's GUI_UPLOAD and GUI_DOWNLOAD but as the amount of data is huge we are planning to get the conversion done using background job. But, we are not able to get it yet. Can anybody suggest ways to resolve this? Can class CL_ABAP_CONV_X2X_CE be used for the same?
Thanks...
With Regards,
Mukul KulkarniHello,
If you have the data in Binary mode, you can use the addition IN BINARY MODE while uploading.
For downloading in UTF-8, you can use the addition TEXT MODE ENCODING UTF-8.
BR,
Suhas -
Conversion of XML file from ANSI to UTF-8 encoding in SAP 4.6C
Hi All,
Im working on SAP 4.6C version.I have generated a XML file from my custom report.It is downloading in ANSI format.But i need to download this into UTF-8 format.So can anyone please let me know how to do this?
Is this possible in 4.6C version?
Thanks in Advance,
Aruna A NHello
It is possible in 4.6.
Try this code:
REPORT Z_TEST_XML_DOWN .
data:
lp_ixml type ref to if_ixml,
lp_xdoc type ref to if_ixml_document,
lp_sfac type ref to if_ixml_stream_factory,
lp_ostr type ref to if_ixml_ostream,
lp_rend type ref to if_ixml_renderer,
lp_enco type ref to if_ixml_encoding.
data:
lp_root type ref to if_ixml_element,
lp_coll type ref to if_ixml_element,
lp_elem type ref to if_ixml_element.
class cl_ixml definition load.
data:
udat like lfa1,
s type string.
select single * from lfa1 into udat where lifnr = '0000000001'. " <- set here real number
*** create xml
lp_ixml = cl_ixml=>create( ).
lp_xdoc = lp_ixml->create_document( ).
lp_root = lp_xdoc->create_simple_element( name = 'Node'
parent = lp_xdoc ).
s = udat-land1.
call method lp_root->set_attribute( name = 'country_name'
value = s ).
s = udat-name1.
call method lp_root->set_attribute( name = 'vendor_name'
value = s ).
s = udat-ort01.
call method lp_root->set_attribute( name = 'city_name'
value = s ).
*** render xml
types: begin of xml_tab_line,
line(256) type x,
end of xml_tab_line.
types: xtab type table of xml_tab_line.
data: t_xml type xtab,
size type i,
rc type i.
lp_sfac = lp_ixml->create_stream_factory( ).
lp_ostr = lp_sfac->create_ostream_itable( table = t_xml ).
lp_enco = lp_ixml->create_encoding( character_set = 'utf-8'
byte_order = if_ixml_encoding=>co_none ).
call method lp_ostr->set_encoding( encoding = lp_enco ).
lp_rend = lp_ixml->create_renderer( ostream = lp_ostr
document = lp_xdoc ).
rc = lp_rend->render( ).
*** export to file
size = lp_ostr->get_num_written_raw( ).
call function 'WS_DOWNLOAD'
exporting
bin_filesize = size
filename = 'c:\sapxml_test.xml'
filetype = 'BIN'
tables
data_tab = t_xml
exceptions
others = 1.
It is just simple example. -
Problem writing file ANSI to UTF-8. OPEN DATASET FOR OUTPUT IN TEXT MODE
Hello everybody,
My report needs to create a file in ANSI so I am using this sentence: * "OPEN DATASET p_dir_a FOR OUTPUT IN TEXT MODE ENCODING DEFAULT."* almost all charters in the file are well generating excepting for this two: ( Á ) & ( Í ) because SAP assigns the same code to both characters ( Ã? ) = 'xC3' which is not correct, the right code should be ( Á ) = 'xC1' ( Í ) = 'xCD' respectively
from this string = ( ÁÉÍÓÚáéíóúàèìòùÀÈÌÒÙ ) only those two characters are wrong converted.
Also I tray to catch a conversion error with this sentence:
TRY.
TRANSFER <field> TO p_dir_a NO END OF LINE.
CATCH cx_sy_conversion_codepage.
MESSAGE 'error' type 'I'.
ENDTRY.
but never enters in it.
Does anyone know where I can configure SAP to SET this value correctly?
Thanks for your help.was a server problem, not SAP
-
Problem in converting value in UTF-8
Hello experts,
I have to store my textfields value in UTF-8 format. Whenever I inserted the qury write the following code for converting into UTF-8 fromat. But its store in like. e.g. बीड format for "बीड" value. I want to insert as like I am typing in my textfield.
code for converting to UTF-8
String name=request.getParameter("txtname");
byte arra[]=name.getBytes("ISO8859-1");
String nameM=new String(arra ,"UTF-8");Please help me out...go through all the links in this url, may be it will helps you
http://java.sun.com/javase/technologies/core/basic/intl/faq.jsp#core-encoding -
Convert String from UTF-8 to IsoLatin1
Hi everyone !
I'm trying to convert a String from utf-8 to IsoLatin1, but i got somt problems.... I'm using
actually this code, but it won't work...
I'm getting a utf-8 html String with some data and i will write it down in latin1 to a text file
String newString = new String(oldString.getBytes("UTF-8"), "ISO-8859-1");If i'm now writing this newString to a TextFile it contains cryptic signs like
& # 1 3 ; or & # 1 3 7 ; or & # 1 2 8 ;(i separated this chars)
can anyone tell me where is my fault and how can i solve this problem ?
Thanks a lot
Edited by: Sephiknight on Feb 23, 2008 2:41 AMYes its XML, i got a web editor where i can add input (utf-8) and i want to write it down in my class to a xml file (isoLatin1).
My code looks likte this
public static void setEditFragment(String content, String xPath) throws Exception {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document document = builder.parse("3001300.xml");
XPath xpath = XPathFactory.newInstance().newXPath();
Node node = (Node)xpath.evaluate(xPath, document, XPathConstants.NODE);
Charset charset = Charset.forName("ISO-8859-1");
CharsetEncoder encoder = charset.newEncoder();
ByteBuffer buf = encoder.encode(CharBuffer.wrap(content));
node.setTextContent(buf.toString());
// Use a XSLT transformer for writing the new XML file
Transformer transformer = TransformerFactory.newInstance().newTransformer();
DOMSource source = new DOMSource( document );
FileOutputStream os = new FileOutputStream("tmp.xml");
StreamResult result = new StreamResult( os );
transformer.transform( source, result );
}The example from http://www.exampledepot.com/egs/java.nio.charset/ConvertChar.html looks great, but if I add my own input string i get a exception that looks like this
java.nio.charset.UnmappableCharacterException: Input length = 1
at java.nio.charset.CoderResult.throwException(Unknown Source)
at java.nio.charset.CharsetEncoder.encode(Unknown Source)
at HagerAbs.setEditFragment(HagerAbs.java:91)
at HagerAbs.main(HagerAbs.java:108)When i write my input to the xml file it doesnt look like xml at all, it looks more like
<synthese>& # 13;
& # 13;
& lt;br/& gt;& # 13;
& lt;img class="thumb" src="http: ......{code}
(i seperated the char so you can see)
and this is not what i expected... how can i write it down correctly ?
Edited by: Sephiknight on Feb 23, 2008 3:26 AM -
SSIS: Change Code page 1252 ANSI to UTF-8 65001
Hi All;
I have created SSIS Package to import data from OLE DB to a flat file. The code page in the flat file connection manager was set to default
1252 ANSI - Latin I. Now I have been asked to output the flat file in
UTF-8 format. I have changed the code page to 65001 UTF-8 but the package will not run. I got the following message:
the code page on input column COLUMN_NAME (184)is 1252 and is required
to be 65001
I have searched around and i have read that i need to change the column data type from
VARCHAR to NVARCHAR!! Is this the solution?
Is there is any other option? Thanks.Hello Guys,
i am having the same problem but none of the proposed solutions work for me. the thing is that my source DB is an SQL server DB table that contains Arabic fields the code page of the Source is 1256.
my destination DB is an SQL Compact Edition (with nvarchar fields) the destination page is 1252
if i use SSIS to import Data to the compact DB the arabic fields in the source Arabic destination appear rubbish
if i set AlwaysUseDefaultCodePage to true, an error occurs (cannot be processed
because more than one code page (1252 and 1256) are specified for it )
Please Help!! -
Converting Unicode to UTF-8 character set through Oracle forms(10g)
Hi,
I am working on oracle forms (10g) where i need to load files containing unicode character set (multilingual characters) to database.
but while loading the file , junk characters are getting inserted into the database tables.
while reading the file through forms , i am using utl_file.fopen_nchar,utl_file.get_line_nchar functions to read the unicode characters ...
the application server , and database server characterset are set to american utf8 characteset.
In fact , when i change the text file characterset to utf8 through an editor(notepad ++,etc) , in that case , data is getting inserted into database properly,(at least working for english characters) , but not with unicode ...
Any guidance in this regard are highly appreciated
Thank you in advance
Sanuhi
please check out the following link.
http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm
sarah
Maybe you are looking for
-
for a few weeks i have not been able to access the itunes store from the itunes on my laptop. every time i click on it it tells me it cannot access the store...not sure why. and says this: ITUNES HAS STOPPED WORKING a problem caused the program to st
-
Problem in creating a nested table
Hi i am working on Oracle 10g and cleint is sqlplus. Now while creating a nested table following error occured. This is the script for your reference.: CREATE OR REPLACE TYPE sec_pwd_hist_table AS TABLE OF sec_pwd_history_type index by binary_integer
-
The new update for OS x is not installing. Anyone know why?
I am trying to install the new update. No success. Anyone know why?
-
Installation of Solaris 9 and 8 Fail on E450
Hello All! We have 4 SCSI disk drives--3 of which are no longer visible by the E450. (They seem dead before their time.) Also, there are two appended A1000 RAID arrays. Depending on the presence and arrangement of the SCSI disks, one of two things ha
-
No places when syncing photos from iphoto 09 to iphone
ok, i spent days tagging faces and places in every single photo i have in iphoto since upgrading to 09. now when i sync my iphone and fill it with photos, the photos doesn't have any "places" info in my iphone. in my iphone places only appear the pic