UTF8 encoding Issue in the conversion program

Hi all,
Our requirement is to get data from a file stored in application server and update the SAP database.
The flat file has some special characters such as u2122, ® etc.
In the conversion program, we are using the encoding format as UTF-8. This means it will only handle UTF-8 file ( byte order mark (BOM) 8).
After running the conversion program, we came to know that the special character u2122 is getting replaced wtih #.
And when we modified the format of flat file to UTF-8 instead of ANSI or UTF-16 or 32.
We were able to get the trademark symbol in correct look in addition with other special characters.
Since this involves a manual of processing modifying the flat file to UTF-8 format and then running the conversion program for getting all the special characters in good look, we are thinking to make this process "automatic" in the conversion program itself i.e. whatever the format of flat file may be, we would convert that file to UTF with BOM 8 format and then go ahead with the next updating processes.
So, can anybody let me know if we can make this an automatic process? Can we convert each non UTF-8 record read from flat file to UTF-8 record in the program itself by any means and then update that record in the database? Is there any FM for this? Or can we convert the flat file to UTF-8 file programmatically irrespective of the original file formats?
Any inputs on this would be greatly appreciated.
Thanks,
Shamim

Sham,
Try with the class CL_BCS_CONVERT and the method STRING_TO_SOLIX.
Regards,
Kiran Bobbala

Similar Messages

Issue with the ABAP program to find BI lookups and code Patterns

Hello dears,
I'm trying to use the ABAP program LOOKUP_FINDER:
http://wiki.sdn.sap.com/wiki/display/BI/ABAPprogramtofindBIlookupsandcodePatterns
But I have the following issue:
Runtime Errors
RAISE_EXCEPTION
Short text
    Exception condition "NO_FIELDCATALOG_AVAILABLE" raised.
Error analysis
    A RAISE statement in the program "CL_GUI_ALV_GRID===============CP" raised the
     exception
    condition "NO_FIELDCATALOG_AVAILABLE".
    Since the exception was not intercepted by a superior
    program, processing was terminated.
Source Code Extract
Line SourceCde
    1 method set_sort_criteria.
    2
    3 *... (1) Trace?
    4   if not mr_trace is initial.
    5     call method mr_trace->add_trace_item
    6       exporting
    7         i_trace_item = 'SET_SORT_CRITERIA'
    8         ir_variant   = m_cl_variant
    9         it_data      = mt_data
   10         it_info      = mt_info.
   11   endif.
   12
   13   if m_cl_variant->mt_fieldcatalog is initial.
>>>>>     raise no_fieldcatalog_available.
   15   endif.
   16
   17   m_cl_variant->mt_sort = it_sort.
   18
   19   call function 'LVC_SORT_COMPLETE'
   20        exporting
   21             it_fieldcat = m_cl_variant->mt_fieldcatalog
   22        changing
   23             ct_sort     = m_cl_variant->mt_sort.
   24
   25 endmethod.
This issue is located in the FM 'REUSE_ALV_GRID_DISPLAY' called at the end of the program.... maybe because the catalog is empty?
For your information, I called the program 'ZLOOKUP_FINDER'.
Can you help me to fix this issue?
Regards,
Vince.

Hi Vince,
If a table is empty field catalog usually return no error it just display your field catalog with headers .
I think the issue is with building of field catalog .
May be some settings are missing there .
Please check FM REUSE_ALV_FIELDCATALOG_MERGE where field catalog is getting generated.
Regards,
Jaya Tiwari

BW Upgrade "Execute the conversion programs for the product master "

Hi BW Gurus,
We are in the process of upgrading BW 3.5 to BW 7.0. We did not get any conversion error in BW Dev but in BWQ we got error in one of ABAP program. The error message in log is :-
"Value '######' of characteristic 0TIME is not a number with 000006 spaces"
and job pertaining to the program is canceled in SM37.
My question is, are we suppose to run any conversion program before or after the upgrade?
Warm Regards,
Anil

Hi Chandu,
I am also looking for the same if u have fond a bapi for any other program please forward the information.
Thanks & Regards,
Basheer

Why can't you simply put a solution to the exclamation point issue into the iTunes program and save a lot of grief for your customers??

Why can't the great computer iTunes gods simply add a program that solves the great Exclamation Point Bugaboo, making iTunes capable of finding all files and eliminating this massive annoyance and the misery it causes?? Why do we, the customer, have to do the boring drone work and find the **** files when the computer should be doing it??

This happens if the file is no longer where iTunes expects to find it. Possible causes are that you or some third party tool has moved, renamed or deleted the file, or that the drive it lives on has had a change of drive letter. It is also possible that iTunes has changed from expecting the files to be in the pre-iTunes 9 layout to post-iTunes 9 layout, or vice-versa, and so is looking in slightly the wrong place.
Select a track with an exclamation mark, use Ctrl-I to get info, then cancel when asked to try to locate the track. Look on the summary tab for the location that iTunes thinks the file should be. Now take a look around your hard drive(s). Hopefully you can locate the track in question. If a section of your library has simply been moved, or a drive letter has changed, it should be possible to reverse the actions.
Alternatively, as long as you can find a location holding the missing files, then you should be able to use my FindTracks script to reconnect them to iTunes .
tt2

Slimer qt fonts & wierd encoding issues after the upgrade to qt 4.5

After the upgrade to qt 4.5, all qt apps have really slim fonts http://i42.tinypic.com/2ln8x10.png
here on an older screenshot http://i42.tinypic.com/2chmbnn.png.
Also there are some glitches with the encoding http://i43.tinypic.com/16ke3o6.png, after i press something it fixes itself http://i44.tinypic.com/2ds302d.png
Last edited by Maki (2009-03-15 11:26:55)

About the slim fonts... this is actually a fix with qt4.5. Old qt didnt respect some system font settings so now that it does, tweak ~/.fonts.conf how you want it.
And about glitches... there will be some minor problems untill kdemod rebuilds against the new qt, you can rebuild workspace yourself if you dont want to wait for them.

Why am I being charged for the Edge program

I recently went into a Verizon Wireless store to get some questions answered about my account and upgrade eligibility. Both my husbands and my phone were having issues and we needed to upgrade or find some other options. We began to speak with an employee at the store and were told we could upgrade to the Edge program so we could receive new phones and not wait another 5 months to our upgrade date. My phone would make random calls, text people out of nowhere, and my husbands would randomly shut down or end calls, as well as his screen was cracked. The employee stated that we could upgrade to the edge program and all we needed to do was send in an iPhone (the phone brand we were using). He said we would either need to get the screen replaced on my husbands phone as well as take my phone to apple to get it diagnosed and get that problem fixed. It sounded like an excellent deal and we asked if we could just buy a good used phone from a used website and send those phones in instead of going through the trouble of getting our current phones fixed to send in. The employee stated that as long as the screens weren't cracked and they were an iPhone 4 or higher then we would be fine. So we bought two used iPhone 4's that were in excellent condition and had no mechanical issues. We sold the phones we had to offset the cost to purchase the two used ones. We got signed up for our new edge program and sent in the two used iPhones. After a few months we received our Verizon bill and it was over $900! We called customer service and they began to tell us that our phones were broke and didn't pass the mechanical performance test so we were charged a fee. So we not only had to pay for those two used phones to send in and pay for our new phones, we are now being charged a $750 fee for these phone being so called broke? We were told that because we broke the edge contract there is no way to get a refund for this fee and have to pay it! This is the most RIDICULOUS thing I have ever heard of and is about to cause a huge appeal to this situation. I will be getting an attorney and am looking for any others who have had issues with the Edge program or similar problems. I will be filing suit against Verizon Wireless for this issue and if we get enough people with similar problems with the Edge program, then we will file a Class Action Lawsuit against Verizon Wireless to get the money back that they have stolen from us, the customers. We will also file for Verizon Wireless to pay for personal loss and harm these fee's have caused. Please reply to this thread and if you have the same problems then we will add you onto the Class Action Lawsuit so you can get back what you have lost or are losing. Thank you. Verizon Wireless Customer Support

Sounds like you Early Edged before finishing your contract. So Verizon told you you could turn in just any iPhone for the Edge program? I always thought you had to turn in the phone that you had not finished paying for yet, the one under contract.

Love Acrobat XI, but when I installed it became the default program to open all my photos. I'd like them to open in theirr native (JPEG, etc. format) How do I disable the PDF default conversion ? Cant' find anything in Preferences that works.

Love Acrobat XI, but when I installed it became the default program to open all my photos. I'd like them to open in theirr native (JPEG, etc. format) How do I disable the PDF default conversion ? Cant' find anything in Preferences that works.

Sounds like a file association issue in Windows. Right click on the files of interest and select Open With>Choose Default Program and select the program you want used (be sure the "always use" box is checked. The Acrobat installation should not have done this change, but this is the process to make the change on each file type.

Media Encoder issue - The application was unable to start correctly (0xc0000022)

I've just installed Adobe Media Encoder CC and upon opening the application I receive this error:
How should I fix this?

This is a Dynamic Link error.
DL is technology linking DVA applications together.
Do you by any chance have mixed versions of DVA application? Like Premiere Pro CC 2014 and AME CC (2013)?
That could possibly cause such issue. You should use matching releases as much as possible.
So if you update one video product (Premiere Pro, AE, Prelude, AME, Audition) you should update all the other ones.
If that's not the case, open up task manager, Force End Dynamic Link task. Then try to launch AME.
If still no help, reboot the system and then try to launch AME.
If still have this issue, hold the shift key then launch AME (This will reset AME preferences).
--Nariman

Conversion program for the open sales orders

Hi ,
How to proceed for writing a conversion program for open sales order ??
Please share if 've any document or idea.
LSMW object: 0090, Method: 0000
Business Object : BUS2032, Method : CREATEFROMDAT2,
Message Type: SALESORDER_CREATEFROMDAT2
Basic Type: SALESORDER_CREATEFROMDAT202
Regards,
Jyoti Shankar

Dear Saso,
I think, instead using Report: VA05/ VA05N; its better to use Report: VL10C
In this report, you will get all the Sales Orders along with Materials - relevant for open delivery, along with Open Qty. and various other options can be selected from "Change Layout - (Ctrlt+F8)".
Best Regards,
Amit
Note: Don't maintain any Field-Entry (while executing the Report: VL10N; in order to list all Sales Orders. But, if you want to list Sales Orders based upon some criterias, maintain the Field-entries, as reqd.

I updated to iTunes 10.5 on Vista. As with iTunes 10.0, The DEP will not allow iTunes to open. WHEN IS APPLE GOING TO ADDRESS THIS ISSUE? I can't update my telephone or iPod. I can't even open the iTunes program!!!

I am sick of dealing with this issue. I wish not that I had never even started using any Apple products, beginning with iTunes. WHEN IS APPLE GOING TO ADDRESS THIS ISSUE AND MAKE ITUNES SOMEHOW COMPATIBLE WITH DEP IN WINDOWS VISTA?
That being said, Is there any way to resolve this? What I did in the past is completely wipe ALL Apple files from my computer, and install an older version of iTunes. It was at least functional. It seems that any version of iTunes 10.0 or more recent is not compatible with the Data Execution Prevention in Windows. You can not remove iTunes from DEP, yet DEP prevents iTunes from running.
The down side is that I can't get the latest updates for my iPhone or iPod. Now, I have a new phone for my husband that needs syncing on my iTunes, and I can't do that because iTunes won't even open.
Can somebody help me out with this, please?

OK, lets do a complete overhaul to make sure it's working ok.
First to check your Windows system file for any corruption:
Go to command prompt (START/ALL PROGRAMS/ACCESSORIES right mouse click "command prompt" and choose "Run as Administrator"
type in
sfc /scannow
let Windows fix any system files that need to be repaired.
Restart the computer.
Start iTunes and see if it is ok.
If it doesn't, will have to go through the whole process of uninstalling and installing acccording to the following steps:
1. Go to Microsoft website to fix install and Unistall problems. Click "Run now" from Fix it to remove all iTunes & related installer files:
http://support.microsoft.com/mats/Program_Install_and_Uninstall
Be aware that Windows Installer CleanUp Utility will not remove the actual program from your computer. However, it will remove the installation files so that you can start the installation, upgrade, or uninstall over.
2. You should remove all instances of iTunes and the rest of the components listed below:
it may be necessary to remove all traces of iTunes, QuickTime, and related software components from your computer before reinstalling iTunes.
Use the Control Panel to uninstall iTunes and related software components in the following order:
iTunes
QuickTime
Apple Software Update
Apple Mobile Device Support
Bonjour
Apple Application Support (iTunes 9 or later)
(Referenceinstructions from Apple article listed here: http://support.apple.com/kb/HT1923 to remove all components)
3. Reboot your computer. Next, download iTunes from here:http://www.apple.com/itunes/download/ and install from scratch

Itunes was working fine. Tries to install latest upgrade and get error message about an invalid character in the path "Program Files (x86)". PC, Win7, nothing else appears to be having same issue.

Itunes was working fine. Tried to install latest upgrade and get error message about an invalid character in the path "Program Files (x86)". PC, Win7, nothing else appears to be having same issue. Program still works, simply cannor upgrade.

Thanks b noir,
I tried this solution without success. After FixIt ran and didn't find a problem, either in looking for issues with "software upgrade" or "iTunes" it kindly offered to help me uninstall iTunes. I had thought of this as a possibilty but it seems to me that if you do that you lose a lot of "non-native-to-Apple" information you might have entered. I did this once and recovery was painfull. Is there a way to uninstall iTunes without losing all of that sort of thing? Any help would be appreciated.

I have downloaded the converter program and the conversion from PDF to Word is terrible. Is there a better solution or setting?

I am trying to convert a PDF to Word without re-typing. I downloaded the converter program onto my MAC but the conversion was terrible. Unusable. Is there a better way. I also have a PC so I could try that as well if would make any difference.

"Scanned" Scanned anything starts you with an image / picture of the page content that was on the source paper.
There is no "renderable" text. OCR can provide an output of text that can be exported. Without that all that is exported is the image.
As to OCR, Acrobat's ClearScan lends itself to repurpose via the export process.
Regardless, anything sourced from a scanner output is the "pig's ear" and that'll not yield the "silk purse".
Export is dictated by the input (GIGO). Export output identifies input quality.
As a C Student stated a Word file (being well-built helps) that sourced the PDF will yield workable export back to Word.
PDF's sourced for FrameMaker export rather nicely.
For optimal export always start with a well-formed Tagged PDF (ISO 14289-1, PDF/UA-1 compliant).
Two core design considerations of tagged PDF are (1) support Accessible PDF and (2) support export (repurpose) of PDF content.
Be well...

Node doesn't belong to the current document and Invalid UTF8 encoding errors

Hi
I'm creating a XSQL page that calls a PL/SQL procedure:
<?xml version="1.0" encoding="ISO-8859-1"?>
<page xmlns:xsql="urn:oracle-xsql" connection="sgvrt">
<xsql:include-owa>xml_indicadores_epm;</xsql:include-owa>
</page>
When I run the page I get:
Node doesn't belong to the current document.
if I use "<include-xml>" I get the following error:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<page>
<xsql-error action="xsql:include-xml">
<message>Invalid UTF8 encoding.</message>
</xsql-error>
</page>
The XML page generated is well formed and it sends "test/xml; charset="ISO-8859-1" myme type. I test it in IE and it runs fine. I wrote a simple procedure that generates a simple one tag XML file and I still get the same errors.

Thanks for the reply
We've tried using the encoding attribute (encoding="ISO-8859-1") but it generates the same errors. We're using XSQL that comes with 9iAS 1.0.2.2.2a. We also tried with XSQL that comes with Oracle 9.2.0. It seems that the parser is not detecting if the XML document has a different encoding than UTF8.

Node doesn't belong to the current document and Invalid UTF8 encoding.

Hi
I'm creating a XSQL page that calls a PL/SQL procedure:
<?xml version="1.0" encoding="ISO-8859-1"?>
<page xmlns:xsql="urn:oracle-xsql" connection="sgvrt">
<xsql:include-owa>xml_indicadores_epm;</xsql:include-owa>
</page>
When I run the page I get:
Node doesn't belong to the current document.
if I use "<include-xml>" I get the following error:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<page>
<xsql-error action="xsql:include-xml">
<message>Invalid UTF8 encoding.</message>
</xsql-error>
</page>
The XML page generated is well formed and it sends "test/xml; charset="ISO-8859-1" myme type. I test it in IE and it runs fine. I wrote a simple procedure that generates a simple one tag XML file and I still get the same errors.
The page generated by the PL/SQL procedure is:
<indicadores_gestion_epm>
<fecha>Junio 2001</fecha>
<indicador><descripcion>Costo de Generacisn de Potencial por Perforacisn (Conv.)</descripcion><plan_valor>N/D</plan_valor><real_valor>48,21</real_valor><unidad_medida>M$/Bl</unidad_medida></indicador>
<indicador><descripcion>Costo de Generacisn de Potencial por Perforacisn (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>0</real_valor><unidad_medida>M$/Bl</unidad_medida></indicador>
<indicador><descripcion>Costo de Exploracion Aqo Actual (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>22,48</real_valor><unidad_medida>$/BPE</unidad_medida></indicador>
<indicador><descripcion>Costo de Exploracion Promedio (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>19,98</real_valor><unidad_medida>$/BD</unidad_medida></indicador>
<indicador><descripcion>Incorporacion de Reservas Probadas y Probables de Crudo (GD)</descripcion><plan_valor>647</plan_valor><real_valor>2</real_valor><unidad_medida>MMBLs</unidad_medida></indicador>
<indicador><descripcion>Incorporacion de Reservas Probadas y Probables Gas (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>4</real_valor><unidad_medida>MMMPC</unidad_medida></indicador>
<indicador><descripcion>Contribuciones a las Reservas Probadas de Gas (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>5</real_valor><unidad_medida>MMMPC</unidad_medida></indicador>
<indicador><descripcion>Produccion Disponible Promedio de Crudo (GD)</descripcion><plan_valor>362,8</plan_valor><real_valor>1401,66</real_valor><unidad_medida>MBD</unidad_medida></indicador>
<indicador><descripcion>Capacidad de Produccisn de Crudo</descripcion><plan_valor>N/D</plan_valor><real_valor>0</real_valor><unidad_medida>MBD</unidad_medida></indicador>
<indicador><descripcion>Contribuciones a las Reservas Probadas de Crudo (GD)</descripcion><plan_valor>1750</plan_valor><real_valor>1750</real_valor><unidad_medida>MMBLs</unidad_medida></indicador>
<indicador><descripcion>Costo Unitario de Operacisn (Conv.)</descripcion><plan_valor>0</plan_valor><real_valor>,01</real_valor><unidad_medida>$/BL</unidad_medida></indicador>
<indicador><descripcion>Costo Unitario de Operacisn (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>102,53</real_valor><unidad_medida>$/BL</unidad_medida></indicador>
<indicador><descripcion>Disponibilidad de Gas (Acumulado)</descripcion><plan_valor>N/D</plan_valor><real_valor>0</real_valor><unidad_medida>MMPCD</unidad_medida></indicador>
<indicador><descripcion>Produccion de Crudo Entregada a RSC Promedio</descripcion><plan_valor>N/D</plan_valor><real_valor>0</real_valor><unidad_medida>MBD</unidad_medida></indicador>
<indicador><descripcion>Costo de Generacion de Potenc. RA/RC (Conv.)</descripcion><plan_valor>N/D</plan_valor><real_valor>3,15</real_valor><unidad_medida>M$/BD</unidad_medida></indicador>
<indicador><descripcion>Costo de Generacion de Potenc. RA/RC (GD)</descripcion><plan_valor>N/D</plan_valor><real_valor>0</real_valor><unidad_medida>M$/BD</unidad_medida></indicador>
</indicadores_gestion_epm>

Thanks for the reply
We've tried using the encoding attribute (encoding="ISO-8859-1") but it generates the same errors. We're using XSQL that comes with 9iAS 1.0.2.2.2a. We also tried with XSQL that comes with Oracle 9.2.0. It seems that the parser is not detecting if the XML document has a different encoding than UTF8.

SPOOL the file in UTF8 Encoding

Hi
My Database supports the storage of chineese characters. I need to generate a file from my db which is used for further Processing. The file generated from the database using spool command should support chineese characters. How to generate a flat file with UTF8 Encoding using Spool command. Please do help me out.
Here i am giving the database settings.
ARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-FXYYYY
NLS_DATE_LANGUAGE AMERICAN
NLS_CHARACTERSET AL32UTF8
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE

Theoretically, nothing special is required to spool UTF-8 characters to a spool file. You just turn spooling on, select the data you need, and it will go into your spool file.
<br>
Here is some sample code showing that it works.
<br>
SQL> !cat create_spool_file.sql
set heading off
set feedback off
set sqlprompt ""
set echo off
spool spool_file.sql
select
'select asciistr('' ' || unistr( '\BBBB') || ''') as result from dual;'
from dual;
spool off
set heading on
set feedback on
set sqlprompt "SQL> "
SQL> start create_spool_file.sql
select asciistr(' ë®»') as result from dual;
SQL> start spool_file.sql
RESULT
\BBBB
1 row selected.Practically however, there are a number of challenges when working with UTF-8 characters. In the example above you can see that, even though the "select asciistr(' ë®»') as ..." command looks like it has garbage characters in it, the character actually made it into the spool file o.k. The problem has more to do with viewing the UTF-8 characters than with spooling them into a file. You can find more info. in this posting cannot display unicode Hex value 25A0 in SQL Plus / 10g .
<br>
If you give a small example of what you're trying to accomplish and where you're having difficulty, perhaps we can help more.

UTF8 encoding Issue in the conversion program

Similar Messages

Maybe you are looking for