Nvarchar fields using Transparent Gateway for MS SQL

Question

We are using TG to make a number of tables/views available in an Oracle database. The tables in question contain a number of nvarchar fields which are expected to contain Unicode data. These fields cause a number of problems when accessed in Oracle. Firstly, queries returning any nvarchar(x) (for some x) fields hang when they are run in Oracle and it is necessary to reset the connection before running another query! Fields with type nvarchar(max) do come through, however they come through as type LONG, which is very difficult to do anything useful with in Oracle and cannot be searched on.
We are working around the problem by creating views in MS SQL on top of our original tables/views which cast nvarchar(x) to varchar(x). We are then able to query these new views in Oracle, search on text fields etc. However, MS SQL does not properly support UTF-8 in varchar fields and the conversion results in downgrading our data to the latin-1 character set. Note casting nvarchar(x) to nvarchar(max) in MS SQL does not work, the queries still hang. We also have a problem where the length of the nvarchar field in MS SQL is longer than the max supported varchar2 length in Oracle. In this case TG will again provide a LONG converted field.
We would like to know how to get full Unicode data to come across Transparent Gateway. We would be happy to get our data in the Oracle nvarchar2 type, or varchar2 with utf8 would also be fine, ideally with clob/nclob for over-length fields. Alternatively any solution which allows us to access full Unicode data would probably be acceptable. We only need to query the data from inside one ETL package, which will store the transformed data in Oracle, so a solution which involves a cast at this stage would also be acceptable, although obviously having the data show up correctly directly through TG would be better.

Answer 1

Heya,
Here is an example table in SQL Server:
CREATE TABLE [dbo].[tblPnLCostNames](
     [ID] [int] IDENTITY(1,1) NOT NULL,
     [Name] [nvarchar](2048) NOT NULL,
     [Group] [nvarchar](50) NULL,
      [nvarchar](50) NULL,
     [Description] [nvarchar](1024) NULL,
CONSTRAINT [PK_tblPnLCostNames] PRIMARY KEY CLUSTERED
     [ID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]
And in Oracle I did this:
  CREATE TABLE "FOR_DW_DEV01"."UNICODE_COST_TEST"
   (     "ID_" NUMBER,
     "NAME_" LONG,
     "GROUP_" VARCHAR2(50 CHAR),
     "CODE_" VARCHAR2(50 CHAR),
     "DESCRIPTION_" VARCHAR2(1024 CHAR)
Now I want to be able to say:
insert into unicode_cost_test ( id_, name_, group_, code_, description )
select "ID", to_lob( "Name" ), "Group", "Code", "Description" from tblPnlCostNames@my_link;
The other fields are shorter and will go straight in to varchar2 fields as UTF-8 with no problems following some tweaks to the gateway ini file. Anything over 2000 characters comes across as a LONG, but trying to use the to_lob function results in "ORA-00997: illegal use of LONG datatype". Doing the same thing using a local table with a LONG field as the source works fine. Is to_lob supposed to work on LONG fields that come across the gateway?
I found this in the 11.2.0.2.2 bugfix list:
10157402 - lob segment has null data after long to lob conversion in parallel mode
(ftp://ftp.hj.se/Oracle/Patchar/11.2.0.2/psu_april_2011/download.htm#BABDIEDD)
Could this be related?
The only way I have found to read the LONG fields so far that works is to use select into, and read each value individually!