Database independant ETL in ODI for SQL Server and Oracle

Hi,
We have a requirement to create ETL which can be run for both SQL Server and Oracle.
We have some tables to be filled from operational database to data warehouse using business requirement.Our goal is to create code which should be database independant.
We will provide all DB details at the start of execution and it will code for as per provided details.
The same can be achieved by Talend.
I wanted to know if same can be achieved in ODI.
Any pointers regarding this would be helpful.
Thanks,
Mahesh

A quick trick: Provide one more value at the start of execution. It is something like a flag. For example db_flag. It can be either 'O' or 'S'.
Open both oracle and sql server KM. Copy one of the KM code to other KM and put an if condition to check the flag.
if (#DB_FLAG.equalsTo("O")){
Oracle code goes here...........
}else{
SQL Server code goes here...........
Its a pure KM customization.
Bhabani
http://dwteam.in

Similar Messages

NVARCHAR - NVARCHAR2 datatype mapping between sql server and oracle.

Question regarding size allocation that occurs for NVARCHAR columns in sql server to NVARCHAR2 in Oracle in the migration tool.
The migration tool has converted NVARCHAR columns to be twice their size as NVARCH2 in Oracle. Example, a table whose column is mycol NVARCHAR(40) gets migrated as mycol NVARCHAR2(80).
In SQL Server land, the designation of a column as NVARCHAR(40) will physically store the column as 80 bytes. However, as I understand from this excerpt (taken from here: http://www.oracle.com/technology/oramag/oracle/03-nov/o63tech_glob.html), that Oracle in essence will do the same thing for NVARCHAR2:
To make it easy to allocate proper storage for Unicode values, Oracle9i Database introduced character semantics. You can now use a declaration such as VARCHAR2(3 CHAR), and Oracle will set aside the correct number of bytes to accommodate three characters in the underlying character set. In the case of AL32UTF8, Oracle will allocate 12 bytes, because the maximum length of a UTF-8 character encoding is four bytes (3 characters * 4 bytes/character = 12 bytes). On the other hand, if you're using AL16UTF16, in which case your declaration would be NVARCHAR2(3), Oracle allocates just six bytes (3 characters * 2 bytes/character = 6 bytes). One difference worth noting is that for UTF-8, a declaration using character semantics allows enough room for surrogate characters, whereas for UTF-16 that is not the case. A declaration such as NVARCHAR2(3) provides room for three UTF-16 code units, but a single supplementary character may consume two of those code units.
As I read this, NVARCHAR(40) in sql server should be equivalent to NVARCHAR2(40) in Oracle and not NVARCHAR2(80) as it gets translated to - am I missing something? We have a rather large database to migrate so I need to get this size allocation right (and I obviously don't want to truncate any data).
Thanks -

Right - well, that's what I'm suggesting is that NVARCHAR(8) in SQL Server is equivalent to NVARCHAR2(8) in Oracle. Truncation will occur after 8 of any character, including, say, double byte Kanjii characters, which would physically store 16 bytes in the NVARCHAR(8) NVARCHAR2(8) column defn. I tried that in both SQL Server and Oracle, and the behavior is the same.
Also, technically, NVARCHAR2(8) and VARCHAR2(8 CHAR) is roughly the same in terms of behavior, except that in the first case, the physical storage is 16 bytes, while in the second case, it's 24 bytes (to account for 'supplemental' characters or true UTF-8 compliance). Same truncation occurs after 8 of -any- character be it single byte ascii or double byte kanjii, which is the behavior I expect.
Thanks for looking into this. What I decided to do was the following:
What was originally defined in SQL Server as varchar (and translated to VARCHAR2 with the CHAR designation), I removed the CHAR designation since I truly want that number of bytes to be stored, and not characters (for columns I do not care about UTF8 or 16 compliance - that will also mimic the same behavior I currently have on the SQL Server side. For anything that was nvarchar in SQL Server, I went ahead and created them as VARCHAR2 with the CHAR designation, with the same size (not double the value as was in SQL Server which the documentation says it gets translated to). I did this since we probably will need true UTF-8 compliance down the line, and since I had the hood open, it was easy enough to do. I could also have converted NVARCHAR(n) to NVARCHAR2(n) (and not n*2) as what was happening in the migration.
I tested strings in both cases, and I think I have my solution.
Edited by: kpw on Sep 24, 2008 11:21 AM

DSNless connection to SQL Server and Oracle

hi,
how can i connect to SQl Server and Oracle without specifying the DSN.
zulfgi

My favorite reply again:
Someone wrote in a newsgroup about configuring some kind of "temporary DSN".
His example was for Excel, but it will maybe work the same way for other drivers:
db = DriverManager.getConnection(
"jdbc:odbc:Driver={Microsoft Excel Driver (*.xls)};DBQ=c:/temp/test2.xls;DriverID=22;READONLY=false","","");
Also someone else suggested for SQLServer:
db = DriverManager.getConnection("jdbc:odbc:Driver={SQL Server};Server=MyServerName;Database=MyDataBase","","");
Find out the right driver clause for Oracle; look at the names the control panel ODBC manager shows.
Regards,
Hartmut

Entity Framework - Code First - Migration - How to access SQL Server and Oracle using the same context?

Hello,
I use Entity Framework code first approach.
My project is working fine with SQL Server. But, I want to access Oracle too. I want to switch SQL Server and Oracle in run time.
I am able to access Oracle using "Oracle.ManagedDataAccess.EntityFramework.dl" in a new project.
But, Is this possible to access SQL Server and Oracle in the same project.
Thanks,
Murugan

This should be possible with a Code-First workflow. In Code-First the database mapping layer is generated at runtime.
David
David http://blogs.msdn.com/b/dbrowne/

Different output of same query in SQL Server and Oracle

I have two tables table1 and table2
--table1 has two columns c1 int and c2 varchar. there are not constraints added in it. it has data as given below
c1     c2
6     d
5     j
102     g
4     g
103     f
3     h
501     j
1     g
601     n
2     m
--table2 has only one column c1 int. there are not constraints added in it. it has data as given below
c1
6
1
4
3
2
now when i run below given query in sql server and oracle it gives me different result
select *
from table1
     inner join (SELECT ROW_NUMBER() OVER (order by c1 ASC) AS c1 from table2) table2 on table2.c1=table1.c1
sql server output
c1     c2     c1
1     g     1
2     m     2
3     h     3
4     g     4
5     j     5
oracle output
C1 C2 C1
5 j 5
4 g 4
3 h 3
1 g 1
2 m 2
If you notice the first column in both output. It is sorted in sql server and not in oracle.
Why it is behaving differently in oracle? Is there any way I can solve this in oracle?
Thanks,
Jigs

It is NOT behaving "differently" in Oracle; you just haven't specified an order that you expect your results to be in, so you're going to get output in whatever order the database fancies displaying it (ie. no guarenteed order). This is an artifact of how the database chooses to put together the data, and different databases (or even datasets within the same database) can and most likely will behave differently.
Even SQL Server won't guarentee to always get your data in an ordered fashion if you exclude the order by clause, even if you think it has always output the data in an ordered fashion.
Your solution is to add an order by clause, in BOTH databases, to force the order of the output data.

Compare SQL Server and Oracle syntax

Hello, I am very new to the Oracle DB as I have been working with MS SQL Server for many years. I have a SQL statement that works fine in MSSQL but seems to not work in Oracle. Is this a syntax problem or do I need to look elsewhere?
DECLARE @as_master_key varchar(26)
EXEC [dbo].[GET_UNIQUE_MASTER_KEY]
     @as_project_name = N'CMDEMO',
     @as_database_site = N'CMPL',
     @as_user_initials = N'SK',
     @as_table = N'SUBMITTAL_CATEGORY',
     @as_master_key =
     @as_master_key
OUTPUT SELECT @as_master_key as N'master_key'
This is what I use in MSSQL and it works fine - but I am porting this application to an Oracle backend and it fails. Any suggestions?
Thanks in advance to this newbie. :)

SQL Server and Oracle are two completely different RDBMS'.
You'd be best not to try and port SQL Server code to Oracle SQL or PL/SQL directly as it will not favour the power of Oracle's features correctly.
I'm not sure what your code is supposed to be doing (I find SQL Server code awful to read, and never understand the need for "@" before variable names, that just seems to show a lazy parser (or lazy coder who designed the parser)).
From what I can guess it's going to be something like...
SQL code...
create or replace sequence my_master_key;PL/SQL code...
DECLARE
as_master_key number;
as_project_name varchar2(20) := 'CMDEMO';
as_database_site varchar2(20) := 'CMPL';
as_user_initials varckar2(5) := 'SK';
as_table varchar2(30) := 'SUBMITTAL_CATEGORY';
BEGIN
select my_master_key.nextval
into as_master_key
from dual;
-- In 11g you could replace this select with just:
-- as_master_key = my_master_key.nextval;
END;This get's a unique sequence value into a variable and declares all the other variables. What you want to do with them after that I don't know.

Outer join query for SQL server from Oracle

Hi All,
My question is regarding making queries from Oracle to SQL Server database thorugh DBLink.
In my oracle database I have a DBLink emp.world for SQL Server database.
I need to query SQL Server data from oracle (so that this query can be combined with other oracle tables).
Query is given below:
SELECT
        a."EmpID" as "Employee ID",
        a."EmpStatus" "Employee Status"
        b."EmpSub" as "Employee Subjects"
FROM
        [email protected] a
        left outer join [email protected] b on a."EmpID" = b."suEmpID"
ORDER BY a."EmpID";My problem is when I run the same query from oracle, it does not show the EmpID that does not exist in Subjects table, but when run from actual SQL Server database, it shows all the records.
Samples are given below:
Run from Oracle
Employee ID      Employee Status     Employee Subjects
101                     Active                     Maths
102                     Active                     Maths
102                     Active                     Physics
104                   Inactive                  Chemistry
Run form SQL Server
Employee ID      Employee Status     Employee Subjects
101                     Active                     Maths
102                     Active                     Maths
102                     Active                     Physics
103                 Active                       NULL
104             Inactive            ChemistryI am not sure why in oracle outer join for SQL server tables is not working. What is the right way for outer join in this case.
I am using oracle database 10gR2 and SQL Server 2005.
Please Help.
Thanks.

SELECT
a."EmpID" as "Employee ID",
a."EmpStatus" "Employee Status"
b."EmpSub" as "Employee Subjects"
FROM
[email protected] a
left outer join [email protected] b on a."EmpID" = b."suEmpID"
ORDER BY a."EmpID";
http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/queries006.htm#sthref3175
From your description, it appears you may need a right outer join. You want to get back all the rows from 'B', not from 'A'. Try a right join and let us know.

Bidirectional Database Sync b/w SQL server and ORACLE

Hi,
We have a Oracle 10g database and we have multiple SQL Server 7.0 databases.
1. The data changes made at Oracle database should be in sync with Sql Servers and vice versa depending upon several Conditions and vice versa (Sql Server - Oracle)
2.This has to happen automatically (not manually) at some particular interval of time everyday.
3. Only the “Delta” data(added/modified/deleted) has to be updated in both databases.
Does Oracle streams provide the above. If so how do we implement it.
We found Database Sync but it is uni directional from Oracle to SQL server.
Is there any other tool available other than Oracle Streams
Thanks...

If the data is available in Oracle of what value is there in duplicating it in another product. Just learn how to do heterogeneous queries and leave the data where it is.

SQL query against tables on MS SQL Server and Oracle using Oracle BI

Hi All
I have two databases Oracle and MS SQL Server. I want to write a query statement which will retrieve data from both databases and publish this data into one report. That is I want to join/merge data from tables in both database into one report with one SQL Statement something like:
Select o.F1,o.F2, m.F1,m.F2
from
table@oracleServer o,
table@microsoftSQLServer m
where o.F1=m.F1;
Can Oracle BI Publisher do this?
Basically I want to query simultaneously two heterogeneous data sources in one SQL statement. Is there such a tool? I don't want to build a warehouse by moving data from SQL Server into Oracle or vice-versa. I don't want to duplicate data. I want to be able to query data at its source via 1 SQL statement.
How would one go about doing such a thing?
Many Thanks
Kubilay.

Hi Matt and Thomas
Many thanks for your replies.
Yes I have to do the join. Even worse I have to write an Aggregate query like this!
Select o.F1,o.F2, m.F1,m.F2, count(*)
from
table@oracleServer o,
table@microsoftSQLServer m
where o.F1=m.F1
group by o.F1,o.F2, m.F1,m.F2;
These are 2 different systems (hardware & software) which actually do the same job, they produce transactions for a common business entity. Kind of, both sell tickets for the same ABC ferry company.
I can not put them side by side on the dashboard, as I need to aggregate the data and present it maybe in a Oracle BI Report for Accounting and Financial reconciliation purposes.
The truth is, as it is. I can't change it, is too late.
I have to device a way to harvest both systems as they are. I don't want to move around data. I thought looking at Oracle BI I could write SQL against multiple Data Sources. I am struggling to find a way. As it seems that it can "SQL query" 1 data source at a time.
I have been hinted on another forum (OracleTURK) to use Oracle Transparent Gateways and Generic Connectivity. http://download.oracle.com/docs/cd/B19306_01/server.102/b14232/toc.htm
Shame, I have to pay licenses for OWB and Oracle Transparent Gateways. I thought DB vendors can do better. Why do I have to pay to access the other 50% of the market.
I can understand the performance implications this might have. Because of it I might even be forced at the end to move data (ETL) into a separate database, slice it into partitions and manage it that way. Pitty because currenlty I only need one report out of these systems but seems like will be dear to get.
Thank you for all your help.
Kubilay

SQL Server and Oracle

is there a SQL server tool or database utility that allows to automatically synchronize data between a table in SQL server and a table(same structure) residing in Oracle on a different server...
Ashish

SQL Server has DTS for this sort of thing. Oracle has Heterogenous Services.
Justin
Distributed Database Consulting, Inc.
http://www.ddbcinc.com/askDDBC

List of OOTB Reports for SQL Server and Windows Server

Hi,
I am struggling to find a list of OTB reports from Operations Manager for SQL and Windows. Ideally I need the name of the reports and a list of counters that they report on.
Can anybody give me an idea where I can find this?
Thanks,
Pete
Peter Carter http://sqlserverdownanddirty.blogspot.com/

Hi Pete,
I don't believe that there are any Windows or SQL Server specific reports out of the box from SCOM.
However, when you import the Windows (either Server or Client, unsure which you are referring to) and/or the SQL Server Management Packs, within the documentation it will tell you the reports that will also be imported and thus available.

Monitoring/ tuning tool for sql server and os environment

What is a good tool for monitoring/tuning OS and SQL layers.

OS and SQL are different things, and I am not sure what you mean under layers.
To further expand my question, what OS build, what SQL Server edition?
All these have enormous impact on strategy.
Then the question is for what? OLTP need one tuning, DW another.
Monitoring is also a question of budget, how much allowed to spend?
Secondly, this forum is not dedicated to these kinds of questions.
Arthur My Blog

Dual contection between SQL Server and Oracle 10g

Hi.
I have the following scenario:
- a MS SQL Server database with some data i'd like to tranfer to an Oracle XE or SE database.
- The idea would be to start a process (possibly on the SQL Server DB) to gather and transform certain data and then, connect to an Oracle Db an insert the data.
I'm good working with Oracle Forms, and i know i can connect to an MS SQL Server using odbc (forms 6i). So obviously, my first choice would be using forms6i to create some text files (using text_io) as a temporary storage and later connect to Oracle DB read and process text files. Another option which i'm not familiar in 6i would be using the EXEC_SQL package.
Apart from this solution, i wonder whether it's possible to create a stored procedure somewhere and have it execute from a deamon or something. This of course depends on the possibility of establish a dual connection between the two DBs.
Thanks in advance ...!

Luis,
The documentation for Generic Connectivity is here:
http://download-east.oracle.com/docs/cd/B19306_01/server.102/b14232/gencon.htm#sthref343
But it is pretty confusing. There is a nice easy and complete example here: http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:4406709207206#18830681837358
It shows a connection to MS Excel, but connecting to SQL Server is the same - just use a different ODBC driver.
Doug

Case sensitive within character literals in SQL Server and Oracle

I am migrating SQL Server database to Oracle database by using Oracle Workbench.
SQL Server is not case sensitive, but Oracle PL/SQL is case sensitive within character literals. for example, if sort the following string ('UN','Win', 'iS' ,'Ne','zS'), SQL Server will get ('iS', 'Ne','UN','Win','zS') order, but Oracle PL/SQL will return ('Ne','UN','Win','iS','zS') order.
Anyone has idea how I can get same result as SQL Server without change PL/SQL code (I don't want to change "ORDER BY typename" to "ORDER BY UPPER(typename)" for every stored procedure).
Thanks

Jimmy,
alter session set NLS_SORT=West_European;
is the closest thing I know of, but UPPER() is the only sure way.
West_European sort order is AaBbCc ... 0123456789
usually sort order is 0123456789 ABC ... abc ...

Use of SQL Server and Oracle Simultaneously

Hello,
Is it possible to have Oracle and SQL Server used simultaneously? Here, developers will be accessing both databases at the same time, whereas they are some ease of implementations at one Database, which can't be found in another database.
Regards,

user650358 wrote:
I am able to retrieve with simple OUT Parameters from the stored procedure in the application without having to use refcursor. Is this a bad thing in the use of other data types instead of refcursors?
I was able to have only a varchar and this worked fine.PL/SQL is a formal procedural language with a number of object orientated programming features. Of course functions can return all kinds of data types - even objects and arrays and collections and pointers (called LOB locators).
All SQL in Oracle are parsed and stored in the server as cursors. The ref cursor data type in PL/SQL is a variable type that contains the handle of a cursor in the server - its "special ability" is passing this PL/SQL variable structure to a client (like Java or .Net) and have the client consume the output of the cursor.
Ref cursors allow you to move a lot of business logic and validation and processing into PL/SQL - and have PL/SQL do the complex stuff and create the SQL cursor and then simply pass the cursor handle to the client as a ref cursor data type. You can easily maintain this code, introduces changes, tune for performance, without having to touch a single byte of client code.
But functions are not there simply to pass ref cursor handles to a client. They can do various other stuff and return all kinds of data types and structures.

Database independant ETL in ODI for SQL Server and Oracle

Similar Messages

Maybe you are looking for