Special characters in UTF-8 UNIX file

We have a program which downloads data from certain info-types in to the UNIX file, Fields are written to the specific position in the UNIX file. Some of the fields contains "Special Characters" in them.
When we download the file in UTF-8 mode (encoding default) then the file display special characters correctly but the all the characters in that record gets shifted to left.
When we download the file in ANSI mode then the file display doesn't special characters correctly but the all the characters in that record do not get shifted from their place.
How can i find the special character in the field and accordingly i will shift the field right so that in the final UNIX file field won't shift their positions.

Hi Ramnivas.
Have you tried to read the characters with class: CL_ABAP_CHAR_UTILITIES (transaction SE24, label Attributes)
For example you can see # in ABAP but in the file will be a new line or a carriage return, you can detect that an adapt it with:
CL_ABAP_CHAR_UTILITIES=>NEWLINE or CL_ABAP_CHAR_UTILITIES=>CR_LF
Other hand, if you are using OPEN DATASET to download the file, look at the options of encoding
See F1 of OPEN DATASET and then encoding..
OPEN DATASET - encoding
Syntax
... ENCODING { DEFAULT
| {UTF-8 [SKIPPING|WITH BYTE-ORDER MARK]}
| NON-UNICODE } ... .
Hope it helps
Regards.
Alfonso.

Similar Messages

Special characters issue in output XML - file adapter - SOA 10.1.3.4

Hi,
I use a DB adapter and File adapter to retreive data from database and create output XML file.
For the database record which have special characters (for example ' , <, >), it will just output the same character in XML file, which cause other system to reject this XML file because of those characters.
Anyone have this issue ? How can i resolve that ?
Thanks

Try converting the characters to < and >. This should work. Make sure the stand-alone & character is converted to & amp; (written with space as HTML will convert it back to &).
-AR

Special characters from textfield to properties file and back

Hi,
The character "o with two dots above it" is being read from a file into a String then written to a proerties file.
It is stored in the proerties file as \uFFFD
When I read this character from the proerties file, and place it in a JTextField, it is displayed as a box character.
Is the character \uFFFD a box ?, or is the corruption happening in the JTextField ?
So my question is,
Is the Properties object useless or is the JTextfield useless ?
Cheers

\uFFFD is a replacement character; it is the substitute for characters that could not be converted to Unicode. This means that the problem is most likely in the code where you read the original file, in effect, that the character encoding you are using to read is different from the encoding of the text file.
Can you post the code where you read the file? It should look something like this if the original file is encoded in UTF-8:
http://javaalmanac.com/egs/java.io/ReadFromUTF8.html

Replacing Special Characters when writing to a file

I am using the following to replace Muñoz (the ñ with null) . It should output as Muoz.
This should be simple but it is not working. 209 and 241 represent the upper and lower case ñ.
This statement is in a CFFILE. I am generating the file but the ñ is still there.
replace(LGL_NM,chr(209)&chr(241),'3','all')
I also tried the following without success:
<cfset x = ReplaceNoCase(#Object_query.LGL_NM#,"#chr(209)#", " ", "all")>
<cfset LGL_NM = ReplaceNoCase(x,"#chr(241)#", " ", "all")>
Please Help.
Thanks!
Bill

I mean on the page itself, in the meta tag. Not sure if it's what's causing the difference though.
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

Special characters conditions in receiver determination

hi there,
i have the following problem:
There are conditions in receiver determination to find the proper xsl mapping for different segments in IDocs.
That works fine.
But if there are special characters like &, the interface mappings can't be found, because for the conditions the xml is already corrupt.
So, i've tried to replace all occurrences of & with & in a java mapping that is called before the xsl mappings.
It doesn't work, because in that case conditions are processed before the java and xsl mappings.
Do you have any idea how to replace special characters before conditions in receiver determination are processed?
thx in advance
Stefan

Handling the Special Characters in XI
https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/9420 [original link is broken] [original link is broken] [original link is broken]
Find out which character encoding the receiver system expects and put the same to the receiver channel.
By default the file adapter would create a file with UTF-8, which might not be useable for the receiver.
check this thread.
Handling Special Characters
check the document :
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79fb42
Depends upon encoding methods handling will differ,
you can use ISO-8859-1 or ISO-8859-2 instead of UTF-8 for some special characters.
check this blog:
Unicode File Handling in ABAP
*Suppress Special Character *
Suppress Special Character

FTP ADAPTER: How to handle Special characters

Hi All,
We are running into issues with special characters while fpt'ing the file using FTP adapter.
ex: japanese characters '東邦チタニウム株式会社' would look as '???????????' in the transferred file.
The following steps are involved in the process.
1. Reads the data from a database.
2. Constructs an XML object.
3. Transforms the data into ftp msg.- In this step the spl. characters are fine
4. FTP's the msg.
5. The file on FTP server has issues.
Tried to change the file type to binary, UTF-8 etc.. but nothing worked.
Any help/pointers will be highly appreciated.
Thanks in advance.

Hi Bhavishya,
Apparently your SAP database isn't configured to support Unicode. That would be the first solution to your problem, but I can imagine it's a bit drastic to convert your DB.
A second solution would be to encode the input description to ASCII before storing it in the database. When reading from the database, decode again to Unicode. This way, no information is lost. A suitable encoding would be Base64. e.g.
String description = "unday ebra";
String descriptionBase64 = new sun.misc.BASE64Encoder().encode(
description.getBytes("UTF-8")); // ""
// store descriptionBase64 in the DB
// later, when reading descriptionBase64 from the DB
String description2 = new String(
new sun.misc.BASE64Decoder().decodeBuffer(descriptionBase64), "UTF-8");
Instead of using Sun's implementation, a better alternative is to use the open source implementation
org.apache.commons.codec.binary.Base64 from Commons Codec .
The 3rd approach is indeed to normalize the description by replacing all special characters with their ASCII equivalent. A rather easy solution is as follows:
String description = "unday ebra";
String descriptionNormalized = sun.text.Normalizer.normalize(
description, sun.text.Normalizer.DECOMP, 0).replaceAll(
"[^p{ASCII}]", "");
sun.text.Normalizer decomposes the string, e.g. "éàî" becomes "e´a`i^", after which non-ASCII characters are being removed using a regular expression.Again, note that it's usually a bad idea to use sun.* packages, see note about sun.* packages. The above code only works with J2SE 1.4 and J2SE 5.0, but it breaks in J2SE 6.0 (where
java.text.Normalizer became part of the public API ;-). A good open source implementation can be found here: ICU4J (com.ibm.icu.text.Normalizer).
Kind regards,
/Sigiswald

How can I display special characters of NonEuropean languages (French, Italian, Spanish, German and Portuguese.) using import string mechanism

I would like to translate the User Interface of my application to French, Italian, Spanish, German and Portuguese.
When I put special characters in the import string file they showed up as ? (question mark)
The import strings file includes the following parameters for each string: font, text, size and style. (but no field for script)
In order to use a unicode font such as Arial I need to select a french script. But this option is not supported bu LabView (As far as I saw)
A) Is there a font which is directly German/ French etc and not regula font + script parameter?
B) Are there another required step
s for special characters support? (When I put speciual characters in the import string file the showed up as ? question mark)

This was discussed last week in this group- read the previous messages.
Look for the thread "Foreign Languages in Labview"
And recite the mantra
ActiveX is good
ActiveX is holy
All Hail Bill
Those who claim otherwise are heretics and not to be trusted
Actually, in this case the non-ActiveX suggestions may be good.
talia wrote in message
news:[email protected]..
> How can I display special characters of NonEuropean languages (French,
> Italian, Spanish, German and Portuguese.) using import string
> mechanism

Replace special characters in FDF

Hello,
I have searched everywhere and cannot find the answer to this simple problem. I can manipulate javascript in an HTML environment by using free scripts, but it's not so easy in a PDF form. I am using Adobe Acrobat 9 Pro and I have a form that when filled out and submitted, generates and emails an FDF file. I am using a server-side script called FormGenie to do this, and the script hates parentheses and some other special characters, they break the FDF file. All I want to do is set up a document level Javascript that will check for and replace all instances of these special characters with something allowed like a hyphen- or just a space.
I have tried:
function clearText() {
     document.formname.fieldname.value=filterNum(document.formname.fieldname.value)
     function filterNum(str) {
          re = /\$|#|~|\%|\*|\^|$|$|\+|\=|\[|\]|\[|\}|\{|\;|\"|\<|\>|\?|\||\\|\!|\$/g;
          // remove special characters like "$" and "," etc...
          return str.replace(re, "-");
and
function clearText2(str) {
stringName = stringName.replace(/\$|#|~|\%|\*|\^|$|$|\+|\=|\[|\]|\[|\}|\{|\;|\"|\<|\>|\?|\||\\|\!|\$ /g,'-');
I know these aren't right, but I have to be at least barking up the right tree.
Can someone help?
Thank you

OK, here is my first stab at putting together this script from searching the API reference.
To loop through the text fields and get their values, I have:
for ( var i = 0; i < this.numFields; i++) {
var fname = this.getNthFieldName(i);
if ( fname.type = "text" ) f.value;   [Not sure if "f.value" is correct or legal here]
If this is even getting close, where I get stuck is how to get the original script to take those values and replace any unwanted characters found.
if (!event.willCommit) {
    event.change = event.change.replace(/[\$\#\~\%\*\^\+\=\[\]\{\}\;\"\<\>\?\|\\\!]/g, "-");
I just need to put this together correctly and I don't know how.
I have the form submission part down, that was already done, I just need to add the custom "Run a javascript" item above it so that is executed first.
Need a little guidance,
Thanks

Problem with special character in Unix file

Hi All,
Need a help here. We have unicoded our system recently.
It's regarding a special character (umlaut) that comes through a 3rd party system to an Unix file. This gets displayed in AL11 file as # instead of ö.
As our program picks the file from Unix, it also has the # but we want ö.
We could have done some fixes in our code to fix if AL11 file at least had ö and we got # in our program.
But it's the other way round. So, how can we get rid of this issue? Please suggest.
Regards,
Sanj.

How Al11 is reading file ? Look for
OPEN DATASET "yourfilename" IN TEXT MODE ENCODING DEFAULT FOR INPUT
IGNORING CONVERSION ERRORS.
If the above code is there .. you might need to play with different 'OPEN DATASET " options .
Also look for Note 1174468 - Non-7bit-ASCII characters used in ABAP Workbench
Note 1227961 - Names of text fields with non-7-bit ASCII characters
Good Luck !
^Saquib

Mac file name restrictions/special characters

What are the Mac OS file name restrictions? What are the special characters to avoid?
Is this a legitimate file name? Are the colons allowed?
/Users/Joe/Desktop/MyDocument 3:2:06.xls
Windows XP Pro

Thanks, Sagesse. I'm new to the Mac OS. Is it tied to Unix in such a way that the colons are allowed?
A user tried to provide a file with a name similar to that in my original post (e.g. "name 3:4:07.rtf") using a web form, and I'm at a loss as to how they could have saved it with the colons. Thank you for your assistance.

ESB or BPEL file adapter and special characters

Hi,
We have a scenario where we import rows from .csv file through an ESB project into a database. We use the file adapter for this. There appears to be a problem with special characters (like é). Both in the ESB control (with variable tracking) and in the database, they appear as upside down questionmarks (¿). I've tried doing the same with a BPEL project (file adapter as client PL) and in the BPEL console, I also see strange characters instead of the expected special characters (diamond shaped characters, like ♦ to be precise).
I can't find anything about character sets of character set conversions in the documentation. What am I missing?
Regards,
Arjan

see
http://download-west.oracle.com/docs/cd/B31017_01/inte
grate.1013/b28994/nfb.htm#CIAEFBHHI've looked into the properties mentioned. They are set when you go through the wizard. Everything is set to UTF-8, which should provide me with all special characters I need.
BPEL does the exact same thing, so I'm starting to believe that the problem really is with the file adapter.
Regards,
Arjan

Special characters being read from the unicode file for greek language

Hi All,
I have a report which would upload a unicode file and then update the vendor master data accordingly.
The file contains greek characters too.
when the file is being read in the code, some special characters are being added up to the vendor number which is the first entry. The special characters are not included in the file , but are added up only to the first vendor number and not any other vendor numbers.
The logic that is being used is as follows :
TRY.
      IF unicode IS INITIAL.
        IF codepage IS INITIAL.
*-->      For backward compatibility where this FM might be called from
*         other dependent objects (FMs/dynamic subroutines)
*         which donot have access to user's input w.r.t Unicode parameters
          OPEN DATASET filename FOR INPUT
               IN LEGACY TEXT MODE
               MESSAGE msg
               REPLACEMENT CHARACTER repl_char
               IGNORING CONVERSION ERRORS
               FILTER filter.
        ELSE.
*--> System in non-unicode and Unicode Environment (Phases I and II)
          OPEN DATASET filename FOR INPUT
               IN LEGACY TEXT MODE CODE PAGE codepage MESSAGE msg
               REPLACEMENT CHARACTER repl_char
               IGNORING CONVERSION ERRORS
               FILTER filter.
        ENDIF.
      ELSE.
*--> Extract File in Unicode format - Phase III
        OPEN DATASET filename FOR INPUT IN TEXT MODE ENCODING UTF-8
        MESSAGE msg
        FILTER filter.
      ENDIF.
      IF sy-subrc NE 0.
        MESSAGE e001(zuni) WITH filename sy-subrc
         RAISING file_open_error.
      ENDIF.
the unicode parameters used are : codepage = 8000.
An early reply is most appreciated.
Regards,
Manu.

Please check SAP notes for Eastern European Characters in Unicode system. and may be below code helps you
DATA:
    ltp_bom                  TYPE sychar01,
    ltp_encoding             TYPE sychar01,
    ltp_codepage             TYPE cpcodepage.
Processing --------------------------------------------------------- *
TRY.
      CALL METHOD cl_abap_file_utilities=>check_utf8
        EXPORTING
          file_name         = itp_filename
          max_kb            = -1
          all_if_7bit_ascii = abap_true
        IMPORTING
          bom               = ltp_bom
          encoding          = ltp_encoding.
    CATCH cx_sy_file_open .
      MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
              WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4
              RAISING file_open_error.
    CATCH cx_sy_file_authority .
      MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
              WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4
              RAISING file_authority_error.
    CATCH cx_sy_file_io .
      MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
              WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4
              RAISING file_io_error.
ENDTRY.
CASE ltp_encoding.
    WHEN cl_abap_file_utilities=>encoding_utf8
      OR cl_abap_file_utilities=>encoding_7bit_ascii.
      CASE ltp_bom.
        WHEN cl_abap_file_utilities=>no_bom.
          OPEN DATASET itp_filename FOR INPUT IN TEXT MODE
            ENCODING UTF-8.
        WHEN cl_abap_file_utilities=>bom_utf8.
          OPEN DATASET itp_filename FOR INPUT IN TEXT MODE
            ENCODING UTF-8
              SKIPPING BYTE-ORDER MARK.
        WHEN cl_abap_file_utilities=>bom_utf16_be.
          ltp_codepage = '4102'.
          OPEN DATASET itp_filename FOR INPUT IN LEGACY BINARY MODE
            BIG ENDIAN CODE PAGE ltp_codepage.
        WHEN cl_abap_file_utilities=>bom_utf16_le.
          ltp_codepage = '4103'.
          OPEN DATASET itp_filename FOR INPUT IN LEGACY BINARY MODE
            LITTLE ENDIAN CODE PAGE ltp_codepage.
        WHEN OTHERS.
          OPEN DATASET itp_filename FOR INPUT IN TEXT MODE
            ENCODING UTF-8.
      ENDCASE.
    WHEN OTHERS.
      OPEN DATASET itp_filename FOR INPUT IN LEGACY TEXT MODE.
ENDCASE.
Edited by: Nilesh Shete on May 7, 2010 5:29 PM

Special characters in Unix

I have pdf document which contains special characters in it.
The character is rendering correctly in Window nt but in Solarix/Unix it is being rendered as ?. How do I correct the problem on Unix and not create a problem on nt?

What character set are you using? Perhaps the document is using Windows-specific character encoding. Try using Unicode/UTF-8 instead, if you can. Or maybe your Linux fonts just don't contain the character you're trying to use.
I've never had problems with viewing a PDF on Linux before though.

Special characters in file ^@

hi,
i used method tostring to convert a double into string and have written in a file,
while i try to open the file in shell(unix), it shows some special characters as follows,
^@0^@.^@0^@4^@3^@7^@4^@0^@7^@
^@0^@.^@0^@3^@1^@3^@1^@9^@2^@
^@0^@.^@0^@1^@6^@5^@0^@2^@9^@
^@0^@.^@0^@0^@3^@1^@8^@5^@5^@1^@
in between each numbers, when i open the same file in textedit it shows as follows,
0.0655756
0.0976344
also i try to do global substitution using unix commands but shows the message pattern not found.
here's my code what should do to get rid of this trouble,
try {
            FileOutputStream fwrite = new FileOutputStream("gridfile.grid");
            DataOutputStream dwrite = new DataOutputStream(fwrite);
            for(int z=0;z<15;z++){
                for(int y=0;y<15;y++){
                    for(int x= 0;x<15;x++){
                        doubgrid = Double.toString(filegrid[x][y][z]);
                        dwrite.writeChars(doubgrid);
                        dwrite.writeChars("\n");
                        //System.out.println("raech");
            fwrite.close();

i didnt finish what i want to write so reposted it
again,
sorry for that,That's ok, it's just that two threads means people can spend time answering the question that's already been answered in the other one. The better thing to do in the future would be just make a new post in the original thread.
I see from your code that you're using the default encoding for the output stream. Since you're writing text, a better way would be to use a BufferedWriter wrapped around a Writer:
new BufferedWriter(new OutputStreamWriter(new FileWriter("filename"), "specify the charset here") );That lets you use a specific encoding instead of relying on the default.

Problem storing special characters (\, \n etc.) in the Properties file

Hello,
I am trying to store a string that contains special characters like \, \n, which are normally escaped using a \ character. If I try to save the Properties object into a file (using the store() method call), it doesn't store these escape characters as they are. The following is what I am trying to do:
Properties p = new Properties();
String samp = "This string will have a \\ and a newline \ncharacter";
p.setProperty( "sample", samp );
p.store( new FileOutputStream( "/path/to/my/file" ) );I am trying to achieve the following result:
sample=This string will have a \ and a newline
characterBut, instead, I get this:
sample=This string will have a \\ and a newline \\\ncharacterCan somebody help me with this problem?
Thanks,

Thanks for your replies and efforts to help me out. I
guess there is no solution to my unique problem.
Basically I need this because one of my values (for a
key) is a multi-line value. It will probably load
fine when I use the \ character to delimit multiple
lines, but when I want to write back the same value
as multiple lines (using \ and \n characters), I am
unable to do that. I don't follow, or else if you're saying what I think you are, I disagree.
Let's say you have a String in your program that contains some \n characters, so if you print that string, say with println, on a Unix terminal, you'll see it on separate lines.
Now you write that string out to a properties file with store().
Forget how it looks in the props file. Consider that opaque, a black box.
Now you use load() to get it back from the properties file. It will be re-created in its original form, so if you print it out on a unix terminal, you'll again see multiple lines. The fact that it's all on one line in the .properties file doesn't affect that.
Now, if you want it to appear on multiple lines in the props file, then we're back to my previous post--you're breaking the format that load expects.
At least, that's my understanding of how it works. I might be mistaken, but I think I've tested this very thing in the not-too-distant past.

Special characters in UTF-8 UNIX file

Similar Messages

Maybe you are looking for