Converting to Unicode

Hello, we currently have a 9.2 database that is set up as Western European. We are upgrading to 10g and we now need to support unicode. I was planning on changing all the CHAR and VARCHAR2 data types to NCHAR and NVARCHAR2. I was also going to change the character set to AL32UTF8 and the national character set to UTF8. Is this what is necessary to support unicode characters?
Can I upgrade a 9i database to 10g and convert everything? Would I be able to export the 9i database and import it in to the 10g UTF8 database? At that point all the data would be normal characters.
If any one has had experience doing this I would appreciate whatever you could provide.
Thank you kindly,
David

- Kieth's recollection about changing the character set of a database is correct. Basically, in order for the ALTER DATABASE CHARACTER SET command to work, the destination character set has to be a strict binary superset of the source character set. That means that every character in the source character set has to have exactly the same binary representation in the destination character set. For any of the Western European character sets, characters 0-127 will generally match US7ASCII and have the same binary representation in UTF8, but characters 128-255 will have a different binary representation in UTF8. That means that direct upgrades are not possible.
- I would generally avoid using NCHAR/ NVARCHAR2 data types except as a last resort. Those data types are more appropriate where you have a need to store data in multiple different character sets in the same database (i.e. you want to use Windows-1252 for your base tables and you want to support Unicode in a handful of new tables, you want to reduce storage requirements by storing some Asian data in the UTF16 character set rather than in UTF8). If you want the entire database to be able to store Unicode data, using national character set data types is generally the wrong approach architecturally.
Practically, it's also a problem to move everything to NCHAR/ NVARCHAR2 because you'd have to re-create all your tables (or add and drop a lot of new columns) which is going to be at least as slow as the Data Pump export & import approach. It would also entail lots of code changes, and some APIs don't support NCHAR/ NVARCHAR2 data types, so you might have to make substantial architectural changes to the apps.
- In addition to being aware of the character/ byte semantics Keith mentioned (some_column varchar2(3) by default allocates 3 bytes not 3 characters of storage space which becomes important when you have a variable length character set like UTF8 where 1 character can be up to 3 bytes long), be aware that there is an initialization parameter NLS_LENGTH_SEMANTICS that can be set to CHAR to change the default for new DDL to specifying lengths in characters rather than bytes.
- Finally, you cannot (in recent Oracle versions) use UTF8 as the national character set. You pretty much need to use UTF16. And it wouldn't be useful to have the national character set and the database character set be the same.
Justin

Similar Messages

Cannot convert between unicode and non-unicode string datatypes

My source is having 3 fields :
ItemCode nvarchar(50)
DivisionCode nvarchar(50)
Salesplan (float)
My destination is :
ItemCode nvarchar(50)
DivisionCode nvarchar(50)
Salesplan (float)
But still I am getting this error :
Column ItemCode cannot convert between unicode and non-unicode string datatypes.
As I am new to SSIS , please show me step by step.
Thanks In Advance.

My source is having 3 fields :
ItemCode nvarchar(50)
DivisionCode nvarchar(50)
Salesplan (float)
My destination is :
ItemCode nvarchar(50)
DivisionCode nvarchar(50)
Salesplan (float)
But still I am getting this error :
Column ItemCode cannot convert between unicode and non-unicode string datatypes.
As I am new to SSIS , please show me step by step.
Thanks In Advance.
HI Subu ,
there is some information gap , what is your source ? are there any transformation in between ?
If its SQL server source and destination and the datatype is as you have mentioned I dont think you should be getting such errors ... to be sure check advance properties of your source and check metada of your source columns
just check simple oledb source as
SELECT TOP 1 ItemCode = cast('111' as nvarchar(50)),DivisionCode = cast('222' AS nvarchar(50)), Salesplan = cast(3.3 As float) FROM sys.sysobjects
and destination as you mentioned ... it should work ...
somewher in your package the source columns metadata is not right .. and you need to convert it or fix the source.
Hope that helps
-- Kunal
Hope that helps ... Kunal

Convert 2byte Unicode inside of XI 2.0 into 1byte ascii

Hi,
does anyone know if there is the possibility inside of XI 2.0 to convert 2byte Unicode into 1byte ascii?
We are sending IDOCs from an UC R/3 system down to XI 2.0 and from
there to a MS-SQL db through JDBC Adapters. Unfortunately the MS-SQL db cannot handle UC.
Thanks and regards
Nesimi
+49.4321.871.428

Duhh....
Here's the code I ended up using, in case others are having the same question.
byte[] b = Encoding.ASCII.GetBytes("ABC");
blob.Write(b,0,b.Length);

SSIS Package : While Extracting Sharepoint Lookup column, getting error 'Cannnot convert between unicode and non-unicode string data types'

Hello,
I am working on one project and there is need to extract Sharepoint list data and import them to SQL Server table. I have few lookup columns in the list.
Steps in my Data Flow :
Sharepoint List Source
Derived Column
its formula : SUBSTRING([BusinessUnit],FINDSTRING([BusinessUnit],"#",1)+1,LEN([BusinessUnit])-FINDSTRING([BusinessUnit],"#",1))
Data Conversion
OLE DB Destination
But I am getting the error of not converting between unicode and non-unicode string data types.
I am not sure what I am missing here.
In Data Conversion, what should be the Data Type for the Look up column?
Please suggest here.
Thank you,
Mittal.

You have a data conversion transformation. Now, in the destination are you assigning the results of the derived column transformation or the data conversion transformation. To avoid this error you need use the data conversion output.
You can eliminate the need for the data conversion with the following in the derived column (creating a new column):
(DT_STR,100,1252)(SUBSTRING([BusinessUnit],FINDSTRING([BusinessUnit],"#",1)+1,LEN([BusinessUnit])-FINDSTRING([BusinessUnit],"#",1)))
The 100 is the length and 1252 is the code page (I almost always use 1252) for interpreting the string.
Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.com

Column "A" cannot convert between unicode and non-unicode string data types

I am following the SSIS overview video-
https://secure.cbtnuggets.com/it-training-videos/series/microsoft-sql-server-2008-business-development/6143?autostart=true
I have a flat file that i want to import the contents onto a SQL database.
I created a Dataflow task, source file and oledb destination.
I am getting the folliwung error -
"column "A" cannot convert between unicode and non-unicode string data types"
in the origin file the data type is coming as string[DT_STR] and in the destination object it is coming as "Unicode string [DT_WSTR]"
I used a data conversion object in between, dosent works very well
Please help what to do

I see this often.
Right Click on FlatFileSource --> Show Advanced Editor --> 'Input and Output Properties' tab --> Expand 'Flat File Source Output' --> Expand 'Output Columns' --> Select your field and set the datatype to DT_WSTR.
Let me know if you still have issues.
Thank You,
Jay

Column cannot convert between unicode and non-unicode string data types

I am converting SSIS jobs from SQL Server 2005 running on a Windows 2003 server to 2008R2 running on a Windows 2008 server.  I have a dataflow with an OLE DB Source which is selecting from an Oracle view.  This of course worked fine in
2005.   This OLE DB Source will not even read the data from Oracle without the error "Column "UWI" cannot convert between unicode and non-unicode. The select is:
SELECT SOME_VIEW.UWI AS UWI,
CAST(SOME_VIEW.OIL_NET AS NUMERIC(9,8)) AS OIL_NET
FROM SOME_SCHEMA.SOME_VIEW
WHERE OIL_NET IS NOT NULL AND UWI IS NOT NULL
ORDER BY UWI
When I do "Show Advanced Editor" on this component, in the Input and Output Properties, I show the OLE DB External Column as DT_STR length 40 for the UWI column and for the Output Columns I see the UWI as the same DT_STR.
How can I get past this? I have tried doing a cast...cast(SOME_VIEW.UWI AS VARCHAR(40)) AS UWI and this gives the same error. The column in Oracle is a varchar2(40).
Any help is greatly appreciated. Thanks.

Please check the data type for UWI using advanced editor for Oledb Source under
external columns and output columns. Are the data types same?
If not, try changing the data type (underoutput columns) same as data type shown under
external columns
Nitesh Rai- Please mark the post as answered if it answers your question

Cannot convert between unicode and non-unicode string data types.

I'm trying to copy the data from 21 tables in a SQL 2005 database to a MS Access database using SSIS. Before converting the SQL database from 2000 to 2005 we had this process set up as a DTS package that ran every month for years with no problem. The only way I can get it to work now is to delete all of the tables from the Access DB and have SSIS create new tables each time. But when I try to create an SSIS package using the SSIS Import and Export Wizard to copy the SQL 2005 data to the same tables that SSIS itself created in Access I get the "cannot convert between unicode and non-unicode string data types" error message. The first few columns I hit this problem on were created by SSIS as the Memo datatype in Access and when I changed them to Text in Access they started to work. The column I'm stuck on now is defined as Text in the SQL 2005 DB and in Access, but it still gives me the "cannot convert" error.

I was getting same error while tranfering data from SQL 2005 to Excel , but using following method i was able to tranfer data. Hopefully it may also help you.
1) Using Data Conversion transformation
data types you need to select is DT_WSTR (unicode in terms of SQL: 2005)
2) derived coloumn transformation
expression you need to use is :
(DT_WSTR, 20) (note : 20 can be replace by your character size)
Note:
Above teo method create replica of your esting coloumn (default name will be copy of <coloumn name>).
while mapping data do not map actual coloumn to the destination but select the coloumn that were created by any of above data transformer (replicated coloumn).

How to fix "cannot convert between unicode and non-unicode string data types" :/

Environment: SQL Server 2008 R2
Introduction:Staging_table is a table where data is being stored from source file. Individual and ind_subject_scores are destination tables.
Purpose: To load the data from a source file .csv while SSIS define table fields with 50 varchar, I can still transfer the data to the entity table/ destination and keeping the table definition.
I'm getting validation error "Cannot convert between a unicode and a non-unicode string data types" for all the columns.
Please help

Hi ,
NVARCHAR = DT_WSTR
VARCHAR = DT_STR
Try below links:
http://social.msdn.microsoft.com/Forums/sqlserver/en-US/ed1caf36-7a62-44c8-9b67-127cb4a7b747/error-on-package-can-not-convert-from-unicode-to-non-unicode-string-type?forum=sqlintegrationservices
http://social.msdn.microsoft.com/Forums/en-US/eb0d1519-4be3-427d-bd30-ae4004ea9e8d/data-conversion-error-how-to-fix-this
http://technet.microsoft.com/en-us/library/aa337316(v=sql.105).aspx
http://social.technet.microsoft.com/wiki/contents/articles/19612.ssis-import-excel-to-table-cannot-convert-between-unicode-and-non-unicode-string-data-types.aspx
sathya - www.allaboutmssql.com ** Mark as answered if my post solved your problem and Vote as helpful if my post was useful **.

Problem in converting the unicode to local language

Hi all,
i am trying to convert unicode char to my local language and print it on label but i am not getting the proper output.
the code is given below
package test;
import java.util.*;
import java.awt.*;
public class KanProp extends Frame{
Locale lcl;
ResourceBundle resbdl;
Label lbl;
public KanProp(){
          setLayout(null);
          setSize(400,400);
          lcl =new Locale("kn","IN");
          resbdl=ResourceBundle.getBundle("LocalStrings_kn",lcl);
          String str=resbdl.getString("Name");
          lbl=new Label();
          lbl.setFont(new Font("JanaKannada",Font.BOLD,24));
          lbl.setBounds(0,100,100,20);
          lbl.setText(str);
          add(lbl);
          setVisible(true);
   static public void main(String[] args) {
      new KanProp();
   } // main
} // classand i am reading a key (ie Name) from the .properties file
can any one please give me the solution?

1. Square box means font does not have a glyph to render your char, or
2. Label is not a lightweight component and may strongly depend on OS - it may result in problems with changing font or using non-OS native characters. Try move to Swing and use JLabel.
3. Try to hardcode unicode text in your source - use editor which supports UTF-8 format and add -encoding UTF-8 in javac command line. This will show if there is any problem with reading resource.

Convertion from Unicode to UTF8

I want to convert some string having Unicode chars into a string with UTF8 char. I used following code snippet:
try {
String str = new String(givenString);
String utfStr = new String(str.getBytes("UTF-8"), "UTF-8");
System.out.println("Converted:" + str + " to:" + utfStr);
} catch (Exception e) {
e.printStackTrace(System.out);
I also tried :
Charset utf8Charset = Charset.forName("UTF-8");
CharsetEncoder encoder = utf8Charset.newEncoder();
CharsetDecoder decoder = utf8Charset.newDecoder();
ByteBuffer bbuf = encoder.encode(CharBuffer.wrap(givenString));
CharBuffer cbuf = decoder.decode(bbuf);
String dest = cbuf.toString();
When Java tries to encode Unicode to UTF-8 and it runs into an unknown character (typically a character that is in the High Ascii range) it substitutes it with '?' or some other wierd character.
How do I prevent this.

Where is this string coming from? Are you initializing it in your source code as a String literal? String str = "A�roport Princesse B�atrix"; If so, you need to make sure the .java file is saved in an encoding that can handle all of the characters. ISO-8859-1, windows-1252, and of course, UTF-8 will all suffice. You also need to make sure the compiler reads the source file with the correct encoding. For example, if you saved your source files as UTF-8, you would do this: javac -encoding UTF-8 *.java Finally, before you print the text to the console, you need to make sure the console is using an encoding that can handle it. On my WinXP box, the default encoding (or codepage, as they call it) for console windows is cp437, which doesn't support accented characters. You can change it with the "chcp" command, like so: chcp 1252 Unfortunately, chcp won't accept UTF-8 or any other Unicode encoding, but cp1252 can handle the accented characters in your string. Note that you don't need to specify that encoding in your code; the Java runtime detects it automatically.
>
If you see question marks or some other placeholder character when viewing output, that's probably because the terminal or whatever doesn't have the fonts available to render those characters.>
No, question marks always indicate an encoding problem. If the character is valid but the font lacks a glyph for it, it shows up as a little rectangle.

Errors in PDF converter for Unicode systems

Hello experts,
I have some problems with the PDF conversion in a Unicode system.
I have to convert a smartform to PDF using CONVERT_OTF. I have implemented SAP notes 812821 and 999712.
The problem is that the special characters(diacritics, language specific characters) are overlapped in the generated PDF document. Does anyone knows what is the problem? I don't know what to do anymore...

Hi,
Try Below code
*& Report ZTEST_NREDDY1
REPORT ztest_nreddy1 NO STANDARD PAGE HEADING.
DATA: it_otf   TYPE STANDARD TABLE OF itcoo,
      it_docs TYPE STANDARD TABLE OF docs,
      it_lines TYPE STANDARD TABLE OF tline,
      st_job_output_info      TYPE ssfcrescl,
      st_document_output_info TYPE ssfcrespd,
      st_job_output_options   TYPE ssfcresop,
      st_output_options       TYPE ssfcompop,
      st_control_parameters   TYPE ssfctrlop,
      v_len_in                TYPE so_obj_len,
      v_language              TYPE sflangu VALUE 'E',
      v_e_devtype             TYPE rspoptype,
      v_bin_filesize          TYPE i,
      v_name                  TYPE string,
      v_path                  TYPE string,
      v_fullpath              TYPE string,
      v_filter                TYPE string,
      v_uact                  TYPE i,
      v_guiobj                TYPE REF TO cl_gui_frontend_services,
      v_filename              TYPE string,
      v_fm_name               TYPE rs38l_fnam.
CONSTANTS c_formname          TYPE tdsfname VALUE 'ZTEST'.
CALL FUNCTION 'SSF_GET_DEVICE_TYPE'
EXPORTING
    i_language    = v_language
    i_application = 'SAPDEFAULT'
IMPORTING
    e_devtype     = v_e_devtype.
st_output_options-tdprinter = v_e_devtype.
*st_output_options-tdprinter = 'locl'.
st_control_parameters-no_dialog = 'X'.
st_control_parameters-getotf = 'X'.
.................GET SMARTFORM FUNCTION MODULE NAME.................
CALL FUNCTION 'SSF_FUNCTION_MODULE_NAME'
EXPORTING
    formname           = c_formname
IMPORTING
    fm_name            = v_fm_name
EXCEPTIONS
    no_form            = 1
    no_function_module = 2
    OTHERS             = 3.
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
...........................CALL SMARTFORM............................
CALL FUNCTION v_fm_name
EXPORTING
    control_parameters   = st_control_parameters
    output_options       = st_output_options
IMPORTING
    document_output_info = st_document_output_info
    job_output_info      = st_job_output_info
    job_output_options   = st_job_output_options
EXCEPTIONS
    formatting_error     = 1
    internal_error       = 2
    send_error           = 3
    user_canceled        = 4
    OTHERS               = 5.
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ELSE.
.........................CONVERT TO OTF TO PDF.......................
CALL FUNCTION 'CONVERT_OTF_2_PDF'
    IMPORTING
      bin_filesize           = v_bin_filesize
    TABLES
      otf                    = st_job_output_info-otfdata
      doctab_archive         = it_docs
      lines                  = it_lines
    EXCEPTIONS
      err_conv_not_possible = 1
      err_otf_mc_noendmarker = 2
      OTHERS                 = 3.
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
Regards
jana

Convert non-unicode file to unicode.

Hi All,
I have to convert a file (in presentation server), which is non-unicode, to unicode compatible. Will reading the file in binary mode and writing in ASC mode be sufficient.
Appreciate the help,
Regards,
Sai

Hi,
try translate .. from code page ... to code page ...
Or may be some of these classes..
CL_ABAP_CONV_IN_CE : Reading binary data
CL_ABAP_CONV_OUT_CE : exporting binary data
CL_ABAP_CONV_X2X_CE : reading and exporting binary data and changing the format
regards
Siggi
Message was edited by: Siegfried Szameitat

How to convert non-unicode app into unicode ASO app

I am using hyperion 11.1.1
I would like to convert ASOsamp app into unicode app. can someone give me some direction. I have looked into DBAG but couldnt find anything useful.
thanks in advance.

thanks John,
Also i would like to know about the Alias table in multiple language ? In Samp_U i see *.alt files on the server, how do we create that ? and then how do we view using those alias table. I set the Japanese alias table as active on the Samp_U database...and i was trying to view the retrieve in Japanese in Web Analysis but it didn't work. What wrong am I doing ?
thanks for the help again.

Convert to Unicode DB

We are planning to upgrade 3.0B to 3.5, in this process we wanted to know:
Our R/3 Source System is 4.6C with MDMP, when we extract data which contains double byte languages, it causes an error in the BW system, thus we are unable to read the data. Will converting to a UNICODE DB (Oracle) resolve this issue?

Sorry Karl for not giving you a real answer, but at this moment I've never been involved in your situation and I haven't personal experience on this topic, so I think that a good document (that I can't read for myself to then give you a real answer simply cause I've a lot of things to do, other than trying help someone here ) is better than nothing (especially if you will be able to find a solution inside it !!!)
Good luck (or maybe someone else can give you a definitive contribution!)...
Bye,
Roberto

Convert arabic unicode to hexidecimal String .

HI ,
In Java, how to Unicode string convert in to hexadecimal String or UCS2.
Thanks and Regards,
Prathap Reddy.
Edited by: prathapreddypolu on Aug 21, 2008 2:34 AM

somebody gave this to me one day. It has helped me tremendously to understand this thing.
Hope it helps you too.
cheers

Converting to Unicode

Similar Messages

Maybe you are looking for