Japanese Characters in Tables

I have a table which stores name,address and country of emplyees.
I see that address for most of the employees from Japan are shown as SII¿¿5F
It looks like its a combination english and japanese characters where japanese characteres are displayed as ¿
Is there any way to read these records?

Thanks.
Changing my question a little.
Can i view already stored data(japanese characters) with following NLS settings?
SQL> SELECT * FROM v$nls_parameters;
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RR
NLS_DATE_LANGUAGE AMERICAN
NLS_CHARACTERSET UTF8
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE
Note: Db had same settings when data was stored
Thanks,
Coolguy

Similar Messages

Converting garbled characters for JAPANESE characters in a custom table

Hi all,
I have a custom table that store Japanese characters.
After my company has upgraded to ECC6.0, this data in the custom table has become garbled and its alot of it garbled.
Is there any SAP tool that can I use to make the correction on those garbled Japanese characters?
Thanks,
William Wilstroth

Hi Nils,
I really really really had a field day reading and testing around UC... To my dissappointment, I do not have the authorization to use SUMG and SCP too as well as a few of the TCODES...
I finally told my higher level technical mgnt. that this table might need some changes...
Has this problem of mine got anything to do with MDMP since its no longer supported in ECC6 and I found one coding that search for MDMP in RSVTPROT...
My colleagues suggest that the data be corrected from table DBTABLOG... which i find, in my opinion, is not the right way...
Thanks,
William

While loading through External Tables, Japanese characters wrong load

Hi all,
I am loading a text file through External Tables. While loading, japanese characters are loading as junk characters. In text file, the characters are showing correctly.
My spool file
SET ECHO OFF
SET VERIFY OFF
SET Heading OFF
SET LINESIZE 600
SET NEWPAGE NONE
SET PAGESIZE 100
SET feed off
set trimspool on
spool c:\SYS_LOC_LOGIC.txt
select CAR_MODEL_CD||',' || MAKER_CODE||',' || CAR_MODEL_NAME_CD||',' || TYPE_SPECIFY_NO||',' ||
     CATEGORY_CLASS_NO||',' || SPECIFICATION||',' || DOOR_NUMBER||',' || RECOGNITION_TYPE||',' ||
     TO_CHAR(SALES_START,'YYYY-MM-DD') ||',' || TO_CHAR(SALES_END,'YYYY-MM-DD') ||',' || LOGIC||',' || LOGIC_DESCRIPTION
from Table where rownum < 100;
spool off
My External table load script
CREATE TABLE SYS_LOC_LOGIC
     CAR_MODEL_CD                         NUMBER               ,
     MAKER_CODE                              NUMBER,
     CAR_MODEL_NAME_CD                    NUMBER,
     TYPE_SPECIFY_NO                         NUMBER               ,
     CATEGORY_CLASS_NO                    NUMBER               ,
     SPECIFICATION                         VARCHAR2(300),
     DOOR_NUMBER                              NUMBER,
     RECOGNITION_TYPE                    VARCHAR2(30),
     SALES_START                          DATE ,
     SALES_END                               DATE ,
     LOGIC                                   NUMBER,
     LOGIC_DESCRIPTION                    VARCHAR2(100)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER
DEFAULT DIRECTORY XMLTEST1
ACCESS PARAMETERS
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL
                    CAR_MODEL_CD,MAKER_CODE,CAR_MODEL_NAME_CD,TYPE_SPECIFY_NO,
                    CATEGORY_CLASS_NO,SPECIFICATION,DOOR_NUMBER,RECOGNITION_TYPE,
                    SALES_START date 'yyyy-mm-dd', SALES_END      date 'yyyy-mm-dd',
                    LOGIC, LOGIC_DESCRIPTION
LOCATION ('SYS_LOC_LOGIC.txt')
--location ('products.csv')
REJECT LIMIT UNLIMITED;
How to solve this.
Thanks in advance,
Pal

Just so I'm clear, user1 connects to the database server and runs the spool to generate a flat file from the database. User2 then uses that flat file to load that data back in to the same database? If the data isn't going anywhere, I assume there is a good reason to jump through all these unload and reload hoops rather than just moving the data from one table to another...
What is the NLS_LANG set in the client's environment when the spool is generated? Note that the NLS_CHARACTERSET is a database setting, not a client setting.
What character set is the text file? Are you certain that the text file is UTF-8 encoded? And not encoded using the operating system's local code page (assuming the operating system is capable of displaying Japanese text)
There is a CHARACTERSET parameter for the external table definition, but that should default to the character set of the database.
Justin

How Oracle tables can be used to display Chinese/Japanese characters

If anyone knows how to display Chinese/Japanese characters from Oracle tables please reply this email. Thanks.
Regards,
Preston

hi
->Also please let me know how the Oracle Lite is licenced if I have 300 odd users of this offline applications
you should speak to your local oracle rep about that, for us for example they gave a pretty cheap packet for olite 10gr3+ 11g repository
->need to use only database part of Oracle Lite and not mobile server.
you cant do that. mobile server is the application server that handles the synchronization process from the server side. when a client tries to sync he actually connects to the mobile server and asks for the database changes , the mobile server know what the client must get so he packs it and give it to him
->you can ofc make lightweight .net apps using olite. we make olite apps even for wince handhelds. so yes ofcourse you can. olite had a win32 client also.
->can it run from usb.
ok here to be honest ive never tried that myself, looks kinda weird as a requirement to be honest but i dont see why it shouldnt run. if you set up the paths correctly you shouldnt have a problem i think.
->offline application will have more or less similar data entry forms and storage structure
yes ofcourse , if i have 3 tables in server i can choose to have 2(or all ) of them on the client too. i can even separate which client gets what. for instance if client a sells houses in new york he will get only the house table rows for new york. if another sells for chicago he will get those instead and etc.
->all client apps are offline and sync periodically (when you choose) to the server

Specify File Encoding(Japanese Characters) for UTL_FILE in Oracle 10g

Hi All,
I am creating a text file using the UTL_FILE package. The database is Oracle 10G and the charset of DB is UTF-8.
The file is created on the DB Server machine itself which is a Windows 2003 machine with Japanese OS. Further, some tables contain Japanese characters which I need to write to the file.
When these Japanese characters are written to the text file they occupy 3 bytes instead of 1 and distort the format of the file, I need to stick to.
Can somebody suggest, is there a way to write the Japanese character in 1 byte or change the encoding of the file type to something else viz. ShiftJIS etc.
Thanking in advance,
Regards,
Tushar

Are you using the UTL_FILE.FOPEN_NCHAR function to open the files?
Cheers, APC

Problem with Gui_download using ASC File type - japanese characters

Hi,
During upgrade,while downloading data for japanese characters using GUI_DOWNLOAD Function module with file type as 'ASC', the space between 2 fields data getting much wider compared to 4.6C Version ws_download Function module's data.
Example: the gap between first field data and second field data in ECC 6.0 is 6 characters length,but in 4.6C it is 2 characters length.
Is there any possibility to get the results similar to 4.6c version.Please give your valueable suggestions.
Thanks
BalaNarasimman

Hi Sandra
Please find the detailed information for your questions.
1.Internal table content before download:During Debugging,it was observed that internal table content was same in both versions.For testing,i used only brand new data(Transaction entry).
2.Download with code Page conversion:Yes,codepage parameter 4103 was explicitly passed into GUI_DOWNLOAD Function module.Also the front end code page which is used by system is 4110 . No errors occured.
3.System is an Unicode system only.
4.Actually this 6 character does not refer the byte value,only the gap between 2 fields data is getting referred in ECC 6.0.Please find the below example.
Example - File data after Download:
ECC 6.0: Field1            Field2      (gap - 6 characters space between 2 fields data) Using GUI_Download
data       u0152©Ïu201DÔu2020      EN
     4.6C: Field1            Field2       (gap - 2 characters space between 2 fields data) Using WS_Download
     data    u0152©Ïu201DÔu2020 EN
Note:Special characters are Japanese characters:

Japanese Characters are showing as Question Marks '?'

Hi Experts,
We are using Oracle Database with below nls_database_parameters:
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CHARACTERSET WE8MSWIN1252
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RR
NLS_DATE_LANGUAGE AMERICAN
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_NCHAR_CONV_EXCP FALSE
NLS_CSMIG_SCHEMA_VERSION 3
NLS_RDBMS_VERSION 11.1.0.7.0
When we are trying to view the Japanese characters (windows 7) in SQLdeveloper, toad or sqlPlus, we are getting data like '????'.
Can anybody please explain us the setups required to view the Japanese characters from the local machine and database.
Thanks in advance.

user542601 wrote:
[Note: If I insert the Japanese characters from Sql Developer or Toad, I am unable to see proper results.]For JDBC connections in Oracle SQL Developer, I believe a different parameter setting is required.
Try running Sql Dveloper with jvm option: -Doracle.jdbc.convertNcharLiterals=true.
I need to use this data in Oracle 6i Reports now.
When I am creating reports using the table where I have Japanese characters stored in NVARCHAR2 column, the value is not displaying correctly in Report Regardless of Reports support for nchar columns, 6i is very very old and based on equally ancient database client libraries (8.0.x if memory serves me). Earliest version of Oracle database software that support the N literal replacement feature is 10.2. So, obviously not available for Reports 6i.
I'm guessing only way to fully support Japanese language symbols is to move to a UTF8 database (if not migrating to a current version of Report Services).
Please help to provide a workaround for this. Or do I need to post this question in any other forums?There is a Reports forum around here somewhere. Look in the dev tools section or maybe Middleware categories.
Edit: here it is: {forum:id=84}
Edited by: orafad on Feb 25, 2012 11:12 PM
Edited by: orafad on Feb 25, 2012 11:16 PM

Problem viewing Japanese characters in Excel sent via Email attachment

Hi All
I am using FM '''SO_DOCUMENT_SEND_API1'' to send out an e-mail attachment (Excel file). I am able to receive the Excel file successfully. However I am not able to display the Japanese characters properly in my Excel file.
I tried to display some Japanese characters in my e-mail contents and I have no problem viewing these characters in my e-mail (I am using MS Outlook 2003). These statements becomes something illegible when I transfer it over to Excel as an attachment. In my Internal Table, these characters are displayed correctly as well.
Anyone has any advice to solve this issue?
Thanks for your replies.

Hi Divya,
refer to the link below:
http://www.sapdevelopment.co.uk/reporting/email/attach_xls.htm
The code in this demonstrates how to send an email to an external email addresswhere the data is stored within a .xls attachment.
Hope this helps.
Reward if helpful.
Regards,
Sipra

SmartForms Printing Error for Japanese Characters

Hi,
I have an issue with Printing Japanese Characters in the SmartForm.
When I Print Preview the smartform I can see the characters in Japanese but when I Print the actual smartform in the printer, all I get is some junk characters like # and !.
I have the following setting in the smartform.
   Multi-Byte functionality enabled
   Character set used is Shift JIS
   Smartform created in language EN
   Translate to All Languages set to true
   No restriction on Language Control
   Smartstyles used has JPMINCHO fonts only
   Default Para is JPMINCHO
   Characters formatting has explicit JPMINCHO as font
   Login language is JA
   Printer is Local Printer with device type JPSAPWIN
When I debug the Smartform function module, the OTF Data Table (within the function) shows characters in Japanese (with codepage 8000) but then I look at the actual spool as a result of processing the function, it comes up as garbled.
Interestingly, I have exactly the same settings on another test smartform to print just 2 columns and it works well.
Not sure where I am going wrong.
Appreciate help on this issue.
Many Thanks
Krishnan

hi,
has anyone solved this problem? even i m facing the problem. me restating the problem as below:
in VF02 transaction, for billing document we are trying to print smartform. Our logon languagae is EN and the Output Type langugae is EN. In spool, we are getting the CORRECT format ie the texts that are maintained in Japanese are showing as Japanese characters while the whole smartform is in English format.
Our requirement is to PRINT (from printer) the texts that are maintained in Japanese should get printed in Japanese format. I know that this is problem with printer settings. The printer is printing perfectly fine as other documents in Japanese characters so its ensured that printer is enabled for Japanese.
I suppose some kind of setting needs to be done at SAP end so that printer recognises double-byte character set of Japanese.
Waiting for confirmation/ solution.
Thanks
Debs

Issue inputting japanese characters in sqlplus?

(Reposting this from Linux forum)
Hey all, looking for some feedback on this issue.
Background: Using a tool to process an external data file (with a Japanese filename and contents which contain Japanese as well). This tool ultimately places the converted data into an Oracle database.
When this tool is run from our Windows environment, everything works correctly.
However, when running the Linux version of the tool, talking to the same database, the following is output:
WARNING: underlying database error.
SDE Code (-51): Underlying DBMS error
Extended DBMS error code: 911
ORA-00911: invalid character
(駅)
Not able to create business table 駅
Delete layer "駅" ...
SDE Code (-51): Underlying DBMS error
Extended DBMS error code: 911
ORA-00911: invalid character
Unable to delete layer "駅"
When we re-run the tool with an English filename it seems to work fine regardless of the contents of the file.
Relevant environment variables:
LANG=ja_JP.UTF8
NLS_LANG=Japanese_Japan.UTF8
NLS_LANGUAGE=japanese
And I have also tried with:
LANG=ja_JP.eucJP
NLS_LANG=japanese_japan.JA16EUC
NLS_LANGUAGE=japanese
I have noticed that if I run sqlplus directly and attempt to create a simple table with a Japanese name, I get the ORA-00911 error with japanese_japan.JA16EUC as NLS_LANG. If I change NLS_LANG to Japanese_Japan.UTF8 I can then successfully create a table with japanese characters in its name. I still cannot run our conversion tool above.
Any ideas or tips? I believe the database is set up correctly as it does work with Windows as the client... perhaps the conversion tool is overriding my NLS_LANG settings? Any other possibilities you can think of?
TIA,
Ray

Thanks for all the input guys.
Here's output from when I have an error creating a table:
ORACLE_HOME is /goa1/ora10gr2/app/oracle/product/10.2.0
ORACLE_SID is jap10b
SDEHOME is /goa1/sde/sde93_jap/sdeexe93
NLS_LANG is japanese_japan.JA16EUC
LANG is ja_JP.eucJP
jap10b on goa.esri.com >sqlplus user/pw
SQL*Plus: Release 10.2.0.2.0 - Production on �� 5�� 8 17:06:56 2008
Copyright (c) 1982, 2005, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.2.0 - Production
With the Partitioning and Data Mining options
SQL> CREATE TABLE 文 (
2 I INT);
CREATE TABLE 文 (
ERROR at line 1:
ORA-00911: invalid character
SQL> SELECT * from NLS_SESSION_PARAMETERS;
PARAMETER
VALUE
NLS_LANGUAGE
JAPANESE
NLS_TERRITORY
JAPAN
NLS_CURRENCY
PARAMETER
VALUE
NLS_ISO_CURRENCY
JAPAN
NLS_NUMERIC_CHARACTERS
NLS_CALENDAR
GREGORIAN
PARAMETER
VALUE
NLS_DATE_FORMAT
RR-MM-DD
NLS_DATE_LANGUAGE
JAPANESE
NLS_SORT
BINARY
PARAMETER
VALUE
NLS_TIME_FORMAT
HH24:MI:SSXFF
NLS_TIMESTAMP_FORMAT
RR-MM-DD HH24:MI:SSXFF
NLS_TIME_TZ_FORMAT
HH24:MI:SSXFF TZR
PARAMETER
VALUE
NLS_TIMESTAMP_TZ_FORMAT
RR-MM-DD HH24:MI:SSXFF TZR
NLS_DUAL_CURRENCY
NLS_COMP
BINARY
PARAMETER
VALUE
NLS_LENGTH_SEMANTICS
BYTE
NLS_NCHAR_CONV_EXCP
FALSE
17 rows selected.
SQL> select value from nls_database_parameters where parameter = 'NLS_CHARACTERSET';
VALUE
JA16EUC
SQL> select value from nls_database_parameters where parameter = 'NLS_NCHAR_CHARACTERSET';
VALUE
AL16UTF16
SQL> And here it works fine (using UTF8 in shell environment)
ORACLE_HOME is /goa1/ora10gr2/app/oracle/product/10.2.0
ORACLE_SID is jap10b
SDEHOME is /goa1/sde/sde93_jap/sdeexe93
NLS_LANG is Japanese_Japan.UTF8
LANG is ja_JP.UTF8
jap10b on goa.esri.com >
jap10b on goa.esri.com >sqlplus user/pass
SQL*Plus: Release 10.2.0.2.0 - Production on 木 5月 8 17:09:12 2008
Copyright (c) 1982, 2005, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.2.0 - Production
With the Partitioning and Data Mining options
SQL> CREATE TABLE 文 (
2 I INT);
Table created.
SQL> SELECT * from NLS_SESSION_PARAMETERS;
PARAMETER
VALUE
NLS_LANGUAGE
JAPANESE
NLS_TERRITORY
JAPAN
NLS_CURRENCY
PARAMETER
VALUE
NLS_ISO_CURRENCY
JAPAN
NLS_NUMERIC_CHARACTERS
NLS_CALENDAR
GREGORIAN
PARAMETER
VALUE
NLS_DATE_FORMAT
RR-MM-DD
NLS_DATE_LANGUAGE
JAPANESE
NLS_SORT
BINARY
PARAMETER
VALUE
NLS_TIME_FORMAT
HH24:MI:SSXFF
NLS_TIMESTAMP_FORMAT
RR-MM-DD HH24:MI:SSXFF
NLS_TIME_TZ_FORMAT
HH24:MI:SSXFF TZR
PARAMETER
VALUE
NLS_TIMESTAMP_TZ_FORMAT
RR-MM-DD HH24:MI:SSXFF TZR
NLS_DUAL_CURRENCY
NLS_COMP
BINARY
PARAMETER
VALUE
NLS_LENGTH_SEMANTICS
BYTE
NLS_NCHAR_CONV_EXCP
FALSE
17 rows selected.
SQL> select value from nls_database_parameters where parameter = 'NLS_CHARACTERSET';
VALUE
JA16EUC
SQL> select value from nls_database_parameters where parameter = 'NLS_NCHAR_CHARACTERSET';
VALUE
AL16UTF16Input is being done from an SSH session from an XFCE Terminal on a Fedora 8 machine. I am cutting and pasting the Japanese character. Everything seems to match up, save for the shell environment variable. I wonder if somehow the character transmitted by my paste changes based on the character set setting.

# is appearing instead of Japanese characters.

Hi All,
While loading data which comprises of japanese characters, '#' values are appearing in the infocube(infoprovider) instead of japanese characters in the production system.When I load the same data in developmennt system I can see the japanese characters.Is there any setting that we need to make in production system so that we can see japanese characters instead of #.If not then please suggest any other method.
Thanks,
Ishdeep.

Yogesh,
Actually the thing is that we dont have any option of code page setting during secong level loading i.e.in our case we have 2 ODS and 2 masters as base tables. On these tables we had a view and above view we have Infocube (2nd level loads) and on infocube we are reporting. If you know any code page setting or any other setting for this please let me know.
Ishdeep.

Printing Japanese characters in English form

Hi all,
i have a requirement to print the japanese characters in the from which is printed in english.
Actually the customer adress is in japanese but when i am printing it,the japanese characters are not getting printed (text is printed as some square boxes).
Any ideas to print the japanese text as it is...
Thanks and Regards,
Sravanthi Chilal

Hi,
Is your text coming from an internal table or it is a standard text that you want to print in adobe form?
If it is standard text or long text,Creating a Text node by right clicking on the top node and then
CREATE -> TEXT.
In the properties of the TEXT node select Text type Include Text and then provide the Text Name, Text Object, Text ID and Text Language values as shown below.
Drag the structure from the interface into the Context.
We can pass the Text node properties values dynamically as well.
If our application makes no use of the Business Address Services but you still want to have country-specific addresses, we must use ABAP coding to achieve this. It is possible to do this in the application program or in the initialization coding of the interface.
In interface,
Define Import parameter under Form Interface.
     IS_CUSTOMER_ID     TYPE     S_CUSTOMER
     IS_COUNTRY          TYPE     LAND1
TYPES:
BEGIN OF ty_adr_printform_table_line,
    line_type      TYPE ad_line_tp,
    address_line LIKE adrs-line0,
END OF ty_adr_printform_table_line.TYPES:
    ty_adr_printform_table TYPE TABLE OF ty_adr_printform_table_line.
Define Global Work Areas and Internal Tables under Global definition as shown below.
IS_ADDRESS           TYPE     ADRS1.
IT_ADDRESS_LINES     TYPE     TY_ADR_PRINTFORM_TABLE.
IS_ADDRESS_LINE     TYPE     TY_ADR_PRINTFORM_TABLE_LINE.
IS_SCUSTOM          TYPE     SCUSTOM.
SELECT SINGLE *
FROM scustom
    INTO is_scustom
WHERE id EQ is_customer_id.is_address-title_text     = is_scustom-form.
is_address-name1     = is_scustom-name.
is_address-street     = is_scustom-street.
is_address-po_box     = is_scustom-postbox.
is_address-post_code1     = is_scustom-postcode.
is_address-city1     = is_scustom-city.
is_address-region     = is_scustom-region.
is_address-country     = is_scustom-country.* Address Format According to Post Office Guidelines
Note : Refer Function module documentation for more information.CALL FUNCTION 'ADDRESS_INTO_PRINTFORM'
EXPORTING
      address_1          = is_address
      address_type     = '1'            "normal/company
       sender_country     = is_country
       number_of_lines     = 8
IMPORTING
address_printform_table = it_address_lines.
*******If the customer address in the database table is in english, you cannot print in Japanese.

Japanese characters in JTable cell

Hi,
We have a situation where text is entered via a JTextArea in a dialog, and is then displayed in a JTable.
We can enter japanese characters into the JTextArea in the dialog, these are displayed correctly. However, when we display the data in the JTable, the japanese characters are displayed as squares. Is it possible to get these characters to display properly?
We have installed the MS Mincho font on our system, we're running Windows NT 4.0 (not a japanese version), and we're running version JDK-1.2.2_007 of the JDK.
We've tried updating our font.properties to use the japanese locale, and also setting the font of the table to MS Mincho but without any luck. Some of the other messages in this forum deal with Java 1.3, but we only support 1.2, is it possible with this version of the JDK?
Thanks!
Mark

Hi,
you must have the font set on the Renderer.
JRG

Japanese characters, outputstreamwriter, unicode to utf-8

Hello,
I have a problem with OutputStreamWriter's encoding of japanese characters into utf-8...if you have any ideas please let me know! This is what is going on:
static public String convert2UTF8(String iso2022Str) {
   String utf8Str = "";
   try {
      //convert string to byte array stream
      ByteArrayInputStream is = new     ByteArrayInputStream(iso2022Str.getBytes());
      ByteArrayOutputStream os = new ByteArrayOutputStream();
      //decode iso2022Str byte stream with iso-2022-jp
      InputStreamReader in = new InputStreamReader(is, "ISO2022JP");
      //reencode to utf-8
      OutputStreamWriter out = new OutputStreamWriter(os, "UTF-8");
      //get each character c from the input stream (will be in unicode) and write to output stream
      int c;
      while((c=in.read())!=-1) out.write(c);
      out.flush();
     //get the utf-8 encoded output byte stream as string
     utf8Str = os.toString();
      is.close();
      os.close();
      in.close();
      out.close();
   } catch (UnsupportedEncodingException e1) {
      return    e1.toString();
   } catch (IOException e2) {
      return e2.toString();
   return utf8Str;
}I am passing a string received from a database query to this function and the string it returns is saved in an xml file. Opening the xml file in my browser, some Japanese characters are converted but some, particularly hiragana characters come up as ???. For example:
屋台骨田家は時間目離れ拠り所那覇市矢田亜希子ナタハアサカラマ楢葉さマヤア
shows up as this:
屋�?�骨田家�?�時間目離れ拠り所那覇市矢田亜希�?ナタ�?アサカラマ楢葉�?�マヤア
(sorry that's absolute nonsense in Japanese but it was just an example)
To note:
- i am specifying the utf-8 encoding in my xml header
- my OS, browser, etc... everything is set to support japanese characters (to the best of my knowledge)
Also, I ran a test with a string, looking at its characters' hex values at several points and comparing them with iso-2022-jp, unicode, and utf-8 mapping tables. Basically:
- if I don't use this function at all...write the original iso-2022-jp string to an xml file...it IS iso-2022-jp
- I also looked at the hex values of "c" being read from the InputStreamReader here:
while((c=in.read())!=-1) out.write(c);and have verified (using character value mapping table) that in a problem string, all characters are still being properly converted from iso-2022-jp to unicode
- I checked another table (http://www.utf8-chartable.de/) for the unicode values received and all of them have valid mappings to a utf-8 value
So it appears that when characters are written to the OutputStreamWriter, not all characters can be mapped from Unicode to utf-8 even though their Unicode values are correct and there should be utf-8 equivalents. Instead they are converted to (hex value) EF BF BD 3F EF BF BD which from my understanding is utf-8 for "I don't know what to do with this one".
The characters that are not working - most hiragana (thought not all) and a few kanji characters. I have yet to find a pattern/relationship between the characters that cannot be converted.
If I am missing some....or someone has a clue....oh...and I am developing in Eclipse but really don't have a clue about it beyond setting up a project, editing it and hitting build/run. It is possible that I may have missed some needed configuration??
Thank you!!

It's worse than that, Rene; the OP is trying to create a UTF-8 encoded string from a (supposedly) iso-2022 encoded string. The whole method would be just an expensive no-op if it weren't for this line: utf8Str = os.toString(); That converts the (apparently valid) UTF-8 encoded byte array to a string, using the system default encoding (which seems to be iso-2022-jp, BTW). Result: garbage.
@meggomyeggo, many people make this kind of mistake when they first start dealing with encodings and charset conversions. Until you gain a good understanding of these matters, a few rules of thumb will help steer you away from frustrating dead ends.
* Never do charset conversions within your application. Only do them when you're communicating with an external entity like a filesystem, a socket, etc. (i.e., when you create your InputStreamReaders and OutputStreamWriters).
* Forget that the String/byte[] conversion methods (new String(byte[]), getBytes(), etc.) exist. The same advice applies to the ByteArray[Input/Output]Stream classes.
* You don't need to know how Java strings are encoded. All you need to know is that they always use the same encoding, so phrases like "iso-2022-jp string" or "UTF-8 string" (or even "UTF-16 string") are meaningless and misleading. Streams and byte arrays have encodings, strings do not.
You will of course run into situations where one or more of these rules don't apply. Hopefully, by then you'll understand why they don't apply.

Oracle Report Server Issue with Japanese Characters

We are trying to setup a Oracle Report Server to print the Japanese characters in the PDF format.
We have separate Oracle Report servers for printing English, Chinese and Vietnamese characters in PDF formats using Oracle Reports in the production which are running properly with Unix AIX version 5.3. Now we have a requirement to print the Japanese characters. Hence we tried to setup the new server for the same and the configurations are done as same as Chinese/Vietnamese report servers. But we are not able to print the Japanese characters.
I am providing the details which we followed to configure this new server.
1.     We have modified the reports.sh to map the proper NLS_LANG (JAPANESE_AMERICA.UTF8) and other Admin folder settings.
2.     We have configured the new report server via OPMN admin.
3.     We have copied the arialuni.ttf to Printers folder and we have converted this same .ttf file in AFM format. This AFM file has been copied to $ORACLE_HOME/guicommon/gk/JP_Admin/AFM folder.
4.     We have modified the uifont.ali (JP_admin folder) file for font subsetting.
5.     We have put an entry in JP_admin/PPD/datap462.ppd as *Font ArialUnicodeMS: Standard "(Version 1.01)" Standard ROM
6.     We have modified the Tk2Motif.rgb (JP_admin folder) file for character set mapping (Tk2Motif*fontMapCs: iso8859-1=UTF8) as we have enabled this one for other report servers as well.
Environment Details:-
Unix AIX version : 5300-07-05-0831
Oracle Version : 10.1.0.4.2
NLS_LANG : JAPANESE_AMERICA.UTF8
Font Mapping : Font Sub Setting in uifont.ali
Font Used for Printing : arialuni.ttf (Font Name : Arial Unicode MS)
The error thrown in the rwEng trace (rwEng-0.trc) file is as below
[2011/9/7 8:11:4:488] Error 50103 (C Engine): 20:11:04 ERR REP-3000: Internal error starting Oracle Toolkit.
The error thrown when trying to execute the reports is…
REP-0177: Error while running in remote server
Engine rwEng-0 crashed, job Id: 67
Our investigations and findings…
1.     We disabled the entry Tk2Motif*fontMapCs: iso8859-1=UTF8 in Tk2Motif.rgb then started the server. We found that no error is thrown in the rwEng trace file and we are able to print the report also in PDF format… (Please see the attached japarial.pdf for your verification) but we are able to see only junk characters. We verified the document settings in the PDF file for ensuring the font sub set. We are able to see the font sub setting is used.
2.     If we enable the above entry then the rwEng trace throwing the above error (oracle toolkit error) and reports engine is crashed.
It will be a great help from you if you can assist us to resolve this issue…

Maybe 7zip or another tool has workarounds for broken file names, you could try that.
Or you could try to go over the files in the zip archive one-by-one and write it to files out-1, out-2, ..., out-$n without concerning yourself with the file names. You could get file endings back via the mimetype.
This script might work:
#include <stdio.h>
#include <zip.h>
static const char *template = "./out-%04d.bin";
int main(int argc, char**argv)
int err = 0;
zip_t *arc = zip_open((const char*)argv[1], ZIP_RDONLY, &err);
if(arc == NULL)
printf("Failed to open ZIP, error %d\n", err);
return -1;
zip_int64_t n = zip_get_num_entries(arc, 0);
printf("%s: # of packed files: %d\n", argv[1], n);
for(int i = 0; i < n; i++)
zip_stat_t stat;
zip_stat_index(arc, i, ZIP_FL_UNCHANGED, &stat);
char buf[stat.size];
char oname[sizeof(template)];
zip_file_t *f = zip_fopen_index(arc, (zip_int64_t)i, ZIP_FL_UNCHANGED);
zip_fread(f, (void*)&buf[0], stat.size);
snprintf(&oname[0], sizeof(template), template, i);
FILE *of = fopen(oname, "wb");
fwrite(&buf[0], stat.size, 1, of);
printf("%s: %s => %lu bytes\n", argv[1], oname, stat.size);
zip_fclose(f);
fclose(of);
zip_close(arc);
return 0;
Compile with
gcc -std=gnu99 -O3 -o unzip unzip.c -lzip
and run as
./unzip $funnyzipfile
You should get template-named, numbered output files in the current directory.
Last edited by 2ion (2015-05-21 23:09:29)

Japanese Characters in Tables

Similar Messages

Maybe you are looking for