Staging Area Schema

My source and target table is same and I am trying to update certain column in table XXX, I have set staging area different then target and specify the staging schema where I do have the create table prevelige but when I run this interface its trying to create temp table in target schema where I do not have preveilege to create table.Any clue ...?
In the interface I have checked all my mapping process is in staging area not source or target.
Any clue ...?
Thanks

HI,
It depends on the KM that you are using... If you are using a IKM Incremental Update then the process will create a temp table at target because it needs it to make the PK comparation.
My suggestions to work around it:
1) if your ETL allows, work with the IKM Control Append
2) If you need of a Incremental Update ELT just point the "Work Area" at the target Physical Schema to the "Staging Area" Schema, change the Database connection user to the user of Staging Area and give him the necessary grants at the tables of the target schema (insert and update i imagine),
Just to be sure, are you using Oracle DB as source, staging and target?

Similar Messages

Staging area and base are in different schema/tablespace

Please can someone give me some advantages and disadvantages to keep staging tables and base tables in same schema. For eg we have staging area where we daily truncate staging table, load in the stg table and do transformation process. Once done we will move the staging table data to base tables. Base tables sizes are huge volume like 6 to 8 million rows as there is no purging done.
I want to suggest to my team that we should keep them in separate schemas as I understand it will be good from I/O point.
Is there any other reason to keep staging and base tables in separate schema/tablespace.

Hi,
Definitely I agree with previous answers. You wrote that staging area, transformation etc.. So I think it's a datawarehouse. Staging and base tables should be stored in different data files also in different harddrives. During the ETL task there can be high load on the Disk subsystem. Storing them in different schemas is another subject which is not related with the disk performance.
When it comes to blocksize of the datafile, there is a approach like this:
If the database is used for OLTP systems, it would be good idea to setup smaller size blocks. But when we're talking about a datawarehouse, it's highly recommended to use larger block sizes like 16KB or 32KB. This setting has an advantage related to reading, accessing data blocks.
Regards,
Cuneyt

Work Schema Vs Staging Area

Can anybody explain what's the difference between Work Schema and Staging Area in ODI.
Work schema we specify while creating Physical schema in Topology manager.
Is Staging area same as work schema?
Thanks

Hi,
you have 3 step
1-source
2-staging area
3-target
in any mapping (designer9 you may choose in which step execute the operation (according to step technology). The work schema is the place in which you execute operation of this step. So, if you say to
LOGICAL : MY_SRC_DB
schema: ora_schema
work_schema: ora_w_schema
LOGICAL : MY_TARGET_DB
schema: ora_t_schema
work_schema: ora_wt_schema
you could say to odi having staging area different from target (for example SUNOPSIS MEMORY ENGINE)
the operation on source would be executed on ora_w_schema
the operation on staging would be executed on SUNOPSIS_MEMORY ENGINE's work
the operation on target would be executed on ora_wt_schema
At Oracle University teacher said me to put work=schema. You may fill in them different only for security issue (or performance)
i hope to be usefull, my english is not so good;P
Decaxd

Staging area and target area

hi..friends.
should staging area and target area be same or different???
if same what are the advantages and dis-advantages???
and if not same what are the advantages and dis-advantages????
I want your help..
Thanks.

Staging area is the place where transformation takes place.
Basically c$,I$ tables gets created at this place and once processing is done , the same gets dropped .
By defalut staging area and traget area are the same . But most of the DBA does not like to have the temp tables created at the target as this cause dbms performance, maintance issue in long term.
Instead you can have a dedicated schema created just for staging and use the same in your interface.
Plus when your target is non dbms then you need to go for a dedicated staging area which can be a separate schema in your database or it could be in memory schema. You will find performance issue if you are using in memory schema as your staging area in case the data volume increase.
Thanks,
Sutirtha

Staging area different from target - which KM???

Hi,
I need to transfer data from CSV to DB (only new inserts operation)
I am working the following KM's
LKM File to SQL
IKM SQL Control Append
It's working fine. However now i need to keep the staging area (where C$, I$ etc table are created) in a separate schema.
I have created a diff schema for staging area and selected in the interface "Staging area different from target"
However not sure of which KM's to use.
Please let me know how to achieve this.
Thanks,
Rosh

Hi
1st thing you have to give the workschema for temp tables when creating physical schema in dataserver for target.
Then you have to select the workschema in overview of interface as "staging different from target".
After doing this when you will use predefined KM for the interface it will create temp tables in workschema.
Now suppose you are not giving the workschma at the time of creating physical schema and you have selected "staging different than target" in interface.Here your C$ table will be created in your workschema but I$ table that is used by IKM is going to be created on target schema.So for this again you have change the IKM KM i.e. where to create I$ table (wokrschema) by selecting the corresponding logical schema.
So its better you give the workschema at the time of creating physical schema.
Here is the query to give privilege by the sys_dba
Grant create any table to ODI_TEMP.
Hope you got it
(Please mark the answer as correct or helpful and close the thread)
Thanks

Lookup issue when staging area diff from target

Hi,
I am using staging area different from target (csv file to database)
Everything is working fine the source data is moved to target. The interface runs suiceessfuly.
Now i want to add a lookup during transformation using a lookup table.
When i do that using lookups in quick edit section, i am getting an error.
When i run the interface, the C$ gets created in staging area - that is correct.
However when inserting from staging to target, now the ODI searches for the C$ table in target area instead of the staging..
Please help.
Thanks,
Rosh

Hi
1st thing you have to give the workschema for temp tables when creating physical schema in dataserver for target.
Then you have to select the workschema in overview of interface as "staging different from target".
After doing this when you will use predefined KM for the interface it will create temp tables in workschema.
Now suppose you are not giving the workschma at the time of creating physical schema and you have selected "staging different than target" in interface.Here your C$ table will be created in your workschema but I$ table that is used by IKM is going to be created on target schema.So for this again you have change the IKM KM i.e. where to create I$ table (wokrschema) by selecting the corresponding logical schema.
So its better you give the workschema at the time of creating physical schema.
Here is the query to give privilege by the sys_dba
Grant create any table to ODI_TEMP.
Hope you got it
(Please mark the answer as correct or helpful and close the thread)
Thanks

Staging area on source in Oracle

Hi,
My source and target, both are Oracle DBs.
a) I created one interface with STAGING AREA ON TARGET. For this I used LKM SQL to SQL and IKM SQL Incremental Update.
Its working fine and all the work tables, flow table etc are gettting created in the
Work Schema (ie. ODI_TRG) I have given while defining physical schema of my target.
I want to try with staging area on source....
b) I checked the "Staging area different from Target" option and selected the logical schema of the source in the Definitions tab of my interface.
In the corresponding physical schema, Work Schema I gave is ODI_SRC.
Now I expected all the work tables, flow tables etc to be created ODI_SRC. But still they are getting created in ODI_TRG!!! Why???
But finally to insert/update the target table, its trying to read the flow table from ODI_SRC, even though it created the flow table in ODI_TRG!!!!
And as a result its failing with this error code...
942 : 42000 : java.sql.SQLException: ORA-00942: table or view does not exist
Please clarify.

I think you didn't change your KM.
Step1: import the "IKM SQL to SQL Append" into your project
Step2: In your interface, on the definition table, select the "Staging Area Different From Target" checkbox, and select the schema of your source data in the drop-down list box.
Step3: go to the flow tab, and select the target, and select the IKM SQL to SQL Append.
Step4: Execute.
Refer this thread what KM to change.
Optimizing the KM

E$ table to be created in STAGING Area

Hello All,
I have an interface that has Staging Area different from TARGET. I have created an ODI Constraint in the target table in my model. Currently, when I run the scenario, the E$ table is created in the target database.
Because we are planning to use E$ for the tracking purpose for which the INtegration team would have access.
1) How can I force the E$ table be created in the ODI STAGING AREA (Database) rather than creating at the target database ?
2) What is the use of RECYCLING ERROR in IKM ?
Kindly clarify.
Thanks

Hi Nag,
Unfortunately a huge KM modification is necessary to accomplish this or divide in more than one interface (process). Let me try to explain:
An IKM Incremental Update create the I$ at target even when the stage area is another Logical Schema because it needs the target table to "decide" what records are to update and what are to insert.
Because of that, the internal the ODI algorithmic set the connections to the target once the I$ will be created there anyway.
To accomplish what you need it will be necessary to change the way that this kind of ETL is done and force the connections to the new staging area.
That is possible but will take some time.
An alternative solution, by process, could be;
1) make a copy of the target at staging area (if possible, I mean, if the amount of records allows it)
2) add 2 columns to this copy to flag the kind of dml (flg_ins, flg_upd for instance)
3) copy all desired constraints to the temporary table
4) use the same original interface but change the target to the temporary table.
5) at mapping check just "insert" to column flg_ins (value "Y" for instance)
6) at mapping check just "update" to column flg_upd (value "Y" for instance)
7) execute the interface
8) create a procedure with 2 steps:
9) step 1 - tab source: select all records from temporary table where flag_ins = 'Y'
- tab target; insert into original target
10)step 2 - tab source: select all records from temporary table where flag_upd = 'Y'
- tab target; update into original target
for the itens 9 and 10 will be necessary to choose the technology and respective Logical Schema.
If the amount of records at target doesn't allow this technique than:
Or will be necessary change the IKM
Or work with DBLINK from target to staging if both technologies are Oracle and to create a dblink will be possible. Any way IKM customization will be necessary, but a small one.
Does it make any sense to you?

Unable to load staging area table in teradata from ODI

Hi
i'm unable to load data in Staging area in teradata. while loading i'm geeting following Error
org.apache.bsf.BSFException: exception from Jython:
Traceback (innermost last):
File "<string>", line 9, in ?
Load Error: See D:\CRM_MOD\/CRM_PRODUCT_SS_0.log for details
     at org.apache.bsf.engines.jython.JythonEngine.exec(JythonEngine.java:146)
     at com.sunopsis.dwg.codeinterpretor.k.a(k.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.scripting(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.execScriptingOrders(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.execScriptingOrders(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTaskTrt(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSqlC.treatTaskTrt(SnpSessTaskSqlC.java)
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
     at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
     at com.sunopsis.dwg.cmd.e.i(e.java)
     at com.sunopsis.dwg.cmd.g.y(g.java)
     at com.sunopsis.dwg.cmd.e.run(e.java)
     at java.lang.Thread.run(Unknown Source)
please any one help me
Thanks
Edited by: user12079238 on Oct 19, 2009 1:17 AM

Hi,
I believe you made some mistake in the Phisycal Schema, quit the last "\" from there....
Cezar Santos
[www.odiexperts.com]

XML as staging area

Hi All,
I am using SQL server as my source and XML as my target. I am using target area same as staging area
Now I want to look up fior all my work table i.e. E$.C$ and I$...where can I see these tables...i mean where are these tables stored when my staging area is XML?
Please help
Thanks

Hi,
ODI creates these temporary tables (E$, I$, C$) in the work schema of your staging area. So check your Staging area (on the definition tab), this will be the name of one of your logical schemas for SQL Server, then go to Topology Manager, find the logical schema and the physical schema which is associated through the Context in which you are running your process. Review the Physical Schema. There are two sections with two drop-down lists each: one section is for the for the data catalog/schema, the other one for the work catalog/schema. The temporary tables are created in the work schema (this might be "dbo" or what ever user you defined) of the work catalog.
Hope this helps,
Matt

Maintaining the structure of a SQL Server staging area and dwh aligned with the Oracle data source

Hi,
I'm working in a context where the data source system, in Oracle, is a continuos work in progress. Each 1-2-3 weeks the data source system in the prod environment is updated with new tables or updated table (with new columns or altered columns). So, it is
important to apply in a rapid manner the related changes in the data structure of the SQL Server staging area and dwh.
The issues to solve are:
a. maintaining SQL Server data structure of the staging area aligned with the data structure of the Oracle data source;
b. maintaing SQL Server data structure of the staging area aligned with the data structure of the SQL Server staging area.
In order to solve these issues it could be useful to think to an authomatic manner to alert when a data structure change occurs and a simple manner to apply the changes on SQL Server data structure.
Any suggests, please? Many thanks

We use Oracle CDC service in SQLServer. It has a flag to indicate the schema changes happening at the Oracle end. We track the schema changes using it and then apply changes to sqlserver side. And regarding automation you can use Biml as suggested or .NET
scripts inside script task.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Load file into Oracle without staging area

Is it possible to change LKM to do this?
Thanks

Yes it is.
You can set the 'Work Schema' (staging) of your topology object to the same as your target (Data Schema), then all objects will be created in the target (C$ I$ etc), therefore no staging area is required - although this is not best practice.
If you want to remove the staging tables altogether, then you can alter the knowledge module to load directly into the target in your LKM. It depends on the method you are using (external tables, sqlloader for example). You IKM will also need amending to basically do nothing, but has to be present for the interface to work.
Cheers
Bos
Edited by: Bos on Apr 7, 2011 8:54 AM

Oracle 8.1.5 Invalid Staging Area Error

I get an error message: Invalid staging area. There is no top level components for Windows NT available for installing
from this staging area. I installed this version successfully running win98, but i get this error with XP. I tried renaming the symcjit.dll
as suggested in some of the earlier posting with no success. Any help is appreciated.
Thanks,
Will

copy disk to hardrive in a temporary directory. go to properties for setup.exe (right click on file used to install oracle). Select the compatiblity tab. make sure the check box "run this program in compatibility mode" is checked. In combo box select windows 98.

Lookup in staging area

We have a column on the source table which contain a person's country information. A single country name could be represented in different formats (for e.g. United States of America, USA, U.S.A). The target database has a table which stores the equivalence mapping for all these different possibilities (i.e. United States of America, USA, U.S.A are mapped to "USA")
We plan to have a design like this
Step 1. Load the country column to the staging area
Step 2. In the staging area, lookup the equivalences table, identify the country name and populate it on the staging area
Step 3. Transfer the data into target table.
The point where we have discussion is the Step-2. What would be an efficient way of resolving the equivalences and how we proceed in this step.
Any thoughts/suggestions on what would be a good or bad design here ?

Hi,
Let me try to help...
If I understood right you can achieve that in just one interface by check the join between the column "country" and the table where the all possible match are at staging area, it means:
1) create an interface
2) drag and drop all souce tables plus the table where are the countries are, from target.
3) drag and drop the "final" target table at target position
4) create a join between the "target countries tables" and "person table" by "country" column and check it to be executed at Staging Area.
Does it make any sense to you?
Cezar

How to print staging areas and doors on the pick doc. ?

Dear experts!
Thank you for your attention!
In our system, we use Lean WM.
We know staging areas and doors can aslo be printed on the pick doc. How to do that?
Best regard!
Tangdark

I'm not a WM expert, but am pretty sure nothing special is required for that. If the fields, where this information is stored, are used by the output processing program and the data is passed to the form and then printed by the form, there should be no problem. If data is there, but it's not printed, then you might need to adjust the output program and/or form.
You might want at least to try it first and use Search whenever possible, then come back with more specific questions, if necessary.

Staging Area Schema

Similar Messages

Maybe you are looking for