Replicate Tables as Staging Tables in a DWH

Hi
We are in process of building a Data Mart, for the staging area we are considering to use GG as the Change Data Capture tool.
The idea is in the source (OLTP Database) we have a table called T1, this table has no PK and it's partitioned by day, generates around 20 million of rows per day and a few hundreds of thousands of changes such as Updates and Deletes. In the staging area we will setup a table called STG_T1 with same structure as T1 plus a few columns such as
@GETENV("TRANSACTION" , "CSN")
@GETENV("GGHEADER", "COMMITTIMESTAMP")
@GETENV("GGHEADER", "LOGRBA")
@GETENV("GGHEADER", "LOGPOSITION")
@GETENV("GGHEADER", "OPTYPE")
@GETENV("GGHEADER", "BEFOREAFTERINDICATOR")
All the changes will be comverted to INSERTS using INSERTALLRECORDS in the replicat. This has a problem, since we dont have PK in the source we dont know how to identify a row's change history in the source in STG_T1.
Has anyone got experience replicating OLTP to a Staging Area using OGG and the ETL basics to propage the changes to the Fact Tables from the Staging Area?
Thanks

If there is no primary key on the source, when you do the ADD TRANDATA, all columns will be supplementally logged. This is probably what you want so that you will have all columns when you apply the operation as an insert on the target.
Even if you don't have a primary key on the target table, you can give Replicat a KEYCOLS on the MAP of one of the target columns - it won't really make any difference what column you pick since you are only going to only be applying inserts so Replicat does not have to format a WHERE clause. However, with no primary key on the target side, you do want to make sure you have enough information on the record to make each row unique.
I would suggest you take a look at the following MOS articles to help guide you:
What Tokens need to included in the transaction to make it unique for Insertallrecords to be used in the replicat [ID 1340823.1]
Oracle GoldenGate - Best Practice: Creating History Tables [ID 1314698.1]
Oracle GoldenGate Best Practice - Oracle GoldenGate for ETL Tools [ID 1371706.1]
Let us know if you still have further questions.
Best regards,
Marie

Similar Messages

How can I replicate table from a user to another user?

I want replicate a table from a user to another user, I use DBMS_STREAMS_ADM.RENAME_SCHEMA, but it didn't work.
who can tell this reason?
scripts as following:
create user strmadmin IDENTIFIED BY strmadmin;
grant dba to strmadmin;
BEGIN
DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE(
grantee => 'STRMADMIN',
grant_privileges => TRUE);
END;
connect strmadmin/strmadmin
--queue
BEGIN
DBMS_STREAMS_ADM.SET_UP_QUEUE(
queue_table => 'STREAMS_QUEUE_TABLE',
queue_name => 'STREAMS_QUEUE');
END;
--capture
BEGIN
DBMS_STREAMS_ADM.ADD_SCHEMA_RULES(
schema_name => 'SCOTT',
streams_type => 'CAPTURE',
streams_name => 'S_CAPTURE',
queue_name => 'STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
include_tagged_lcr => false,
source_database => 'ORCL.REGRESS.RDBMS.DEV.US.ORACLE.COM',
inclusion_rule => true);
END;
--instantiation
DECLARE
iscn NUMBER;
BEGIN
iscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();
DBMS_APPLY_ADM.SET_SCHEMA_INSTANTIATION_SCN(
source_schema_name=>'SCOTT',
source_database_name => 'ORCL.REGRESS.RDBMS.DEV.US.ORACLE.COM',
instantiation_scn=>iscn);
DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(
source_object_name=>'SCOTT.DEPT',
source_database_name => 'ORCL.REGRESS.RDBMS.DEV.US.ORACLE.COM',
instantiation_scn=>iscn);
END;
--apply
--transformation
DECLARE
s_dml VARCHAR2(30);
s_ddl VARCHAR2(30);
BEGIN
DBMS_STREAMS_ADM.ADD_SCHEMA_RULES(
          schema_name => 'SCOTT',
streams_type => 'APPLY',
streams_name => 'S_APPLY',
queue_name => 'STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
source_database => 'ORCL.REGRESS.RDBMS.DEV.US.ORACLE.COM',
dml_rule_name => s_dml,
ddl_rule_name => s_ddl);
DBMS_STREAMS_ADM.RENAME_SCHEMA(
          rule_name => s_dml,
          from_schema_name => 'SCOTT',
          to_schema_name => 'TEST',
          step_number => 0,
          operation => 'ADD');
END;
--start
BEGIN
DBMS_APPLY_ADM.SET_PARAMETER(
apply_name => 'S_APPLY',
parameter => 'DISABLE_ON_ERROR',
value => 'N');
END;
BEGIN
DBMS_APPLY_ADM.START_APPLY(
apply_name => 'S_APPLY');
END;
BEGIN
DBMS_CAPTURE_ADM.START_CAPTURE(
capture_name => 'S_CAPTURE');
END;
/

select * from dba_streams_rename_schemaWhat do you have there?

Replicate Table with spatial columns

I'd like to know how to
replicate (updatable snapshot) a table
with spatial columns.
when i try to create a master repobject
i've got ORA-23318 error.
What can i do?
null

Hi Vincent,
Oracle does not support replication of object in 8.1.6 or 8.1.7. I believe replication support is expected for the version after.
Sorry,
Dan

Using Archiver to replicate tables

Hi,
I'm trying to replicate a singular table between two instances content server using archiver.
CREATE TABLE TEST
ID NVARCHAR2(20) NOT NULL,
SCHCREATETIMESTAMP TIMESTAMP(6),
SCHMODIFYTIMESTAMP TIMESTAMP(6),
SCHSOURCEID NVARCHAR2(50)
When I insert or update records using "Export Archiver" in manual mode works good!, but in this mode, the archiver doesn't get the records deleted.
I try to use "export Archiver" in automatic mode, but seems don't work because don't get the changes on the records (update/insert or delete) .
Does anybody using this feature ?.
Content Server 7.5.1
Thanks.

Be careful before engaging in dml apply handler doing such weird things... Take time to experiment basic functionality before.
I have not had the time to do such experiment yet myself... but I'd rather be careful with new features until I saw them work satisfactorily.
I personnally would investigate using materialized views for what you describe.
Regards,

Why could not Oracle replicates tables which includes Spatial Data to Logical Standby?

Hi all,
Nowadays, i am working on Oracle Logical Standby Databases which includes Spatial Data? But i could not achieve to transport a table with multimedia data (with spatial data types ). But Logical Standby Databases can not support this type of tables? Is this a restriction of modelling Spatial and Geometry in data blocks? And how do i achieve this because i should use this secondary for query-only?
Kind Regards,

Some data types are simply not supported by logical standby replication. Depending on the version, this might or might not include Spatial.
11.2: https://docs.oracle.com/cd/E11882_01/server.112/e41134/data_support.htm#SBYDB4936
12.1: https://docs.oracle.com/database/121/SBYDB/data_support.htm#SBYDB4936
If you need to replicate Spatial data, you need to look at other mechanisms. E.g., physical standby, materialized views, or GoldenGate.

Can we replicate table data from R/3 to BI

Hi All
I have a tablke called JEST which captures status(OBJNR) from VBAP table. We have same table available in BW also. My question is can we replicate data from R/3 to BW for this table. Since this table in BW is not being loaded.
My requirement is to display the status OBJNR in BW
Can anyone advice.

Hello Vani,
Replication of data in some tables from ECC to BW is possiable. Data in tables like T00* (Currency conversion, unit etc ) is replicated into BW from ECC. I think we could also replicate data in table JEST. Please consult your BASIS team, they could do it for you.
If your requirement is to display the status OBJNR in BW, one good way is to create InfoObject Object Number, with attributes Object Status, Indicator & Change number. Data can then be loaded in BW using a Generic master datasource based on DataBase table JEST. Or enhance a Datasource to include OBJNR.
Please let me know if you need any more information.
Regards,
Pankaj

Replicate Table within LiveCycle form

I manage to create a dynamic table which capable to grow on user demand for the use of company's internal purchase requisition. It works perfectly, until someday somebody in the management ask me if I can generate Purchase Order based on company's internal purchase requisition.
I need to populate the dynamic table (in the requisition form) to be used in Purchase Order form. My question is, what should I do in order to
replicate the same dynamic table
without using external application / accessing database. I'm currently using Designer 8.
Thank you in advance.

Look here or wherever you installed it and there is a really good example....
C:\Program Files\Designer 8.0\EN\Samples\Purchase Order

No tables after creating DWH tables

Hi,
After configering the DAC there are no tables created.
The contents of the generate_clt log are as below.
Schema will be created from the following containers:
obiawarehouse
Success!
>>>>>>>>>>>>>>>>>>>
and the content of the createwtables.log
=====================================
STD OUTPUT
=====================================
CREATING SIEBEL DATABASE OBJECTS
E:\oracle\product\10.2.0\OracleDB\DAC\bifoundation\dac\UTILITIES\BIN\DDLIMP /I N /s N /u hr /p ******* /c obia /G "SSE_ROLE" /f E:\oracle\product\10.2.0\OracleDB\DAC\bifoundation\dac/conf/sqlgen/ctl-file/oracle_bi_dw.ctl /b "" /K "" /X "" /W N
The Siebel Database Objects were created successfully.
=====================================
ERROR OUTPUT
=====================================
Siebel Enterprise Applications ODBC DDL Import Utility, Version 7.7 [18030] ENU
Copyright (c) 2001 Siebel Systems, Inc. All rights reserved.
This software is the property of Siebel Systems, Inc., 2207 Bridgepointe Parkway,
San Mateo, CA 94404.
User agrees that any use of this software is governed by: (1) the applicable
user limitations and other terms and conditions of the license agreement which
has been entered into with Siebel Systems or its authorized distributors; and
(2) the proprietary and restricted rights notices included in this software.
WARNING: THIS COMPUTER PROGRAM IS PROTECTED BY U.S. AND INTERNATIONAL LAW.
UNAUTHORIZED REPRODUCTION, DISTRIBUTION OR USE OF THIS PROGRAM, OR ANY PORTION
OF IT, MAY RESULT IN SEVERE CIVIL AND CRIMINAL PENALTIES, AND WILL BE
PROSECUTED TO THE MAXIMUM EXTENT POSSIBLE UNDER THE LAW.
If you have received this software in error, please notify Siebel Systems
immediately at (650) 295-5000.
E:\oracle\product\10.2.0\OracleDB\DAC\bifoundation\dac\UTILITIES\BIN\DDLIMP /I N /s N /u hr /p ***** /c obia /G SSE_ROLE /f E:\oracle\product\10.2.0\OracleDB\DAC\bifoundation\dac/conf/sqlgen/ctl-file/oracle_bi_dw.ctl /b /K /X /W N
Connecting to the database...
Connected.
Reading tables and indexes from DDL file...
Read 0 tables and 0 indexes from DDL file...
Reading existing schema...
Read 0 tablespaces, 0 tables and 0 indexes from existing schema...
Running SQL statements against the database...
TABLES CREATED : 0
TABLES GRANTED : 0
TABLES DELETED : 0
TABLES MERGED : 0
TABLES REBUILT : 0
TABLES IGNORED : 0
COLUMNS ADDED : 0
COLUMNS DELETED : 0
COLUMNS MODIFIED: 0
INDEXES CREATED : 0
INDEXES DELETED : 0
INDEXES MODIFIED: 0
TOTAL ERRORS : 0
Disconnecting from the database.
>>>>>>>>>>>>>>>>>>
Thanks & Regards
Raj
Edited by: user10189442 on Aug 27, 2008 10:41 PM

Every release of Apps comes with metadata, and DAC as well (atleast for now). You have to install DAC from the apps install, and restore data. Now if you happen to have a later version of DAC, then you can use it to upgrade the repository that was created with the DAC that came with apps install. For example, lets say the latest apps release is V1. The latest DAC that you can download is V2. You cannot restore the data using V2 DAC from the V1 Apps. Use V1 apps DAC to restore the data, and if necessary, use V2 to upgrade it.
Please say which applications are you trying to install, and what version of DAC are u using to restore... DAC version can be obtained from DAC Client->Help->About DAC.

Replicate Table DDL Changes to other schema..!!

Hello All,
I have two schemas SCH1 and SCH2
SCH1 : TBL1
SCH2 : A_TBL1
Our requirement is whenever we create a table in SCH1 for Eg TBL2 there should be some code which will create A_TBL2 table automatically in SCH2.. Both TBL2 and A_TBL2 should have same structure.. This functionality i can manage using "ON CREATE OR ALTER" Trigger..
But now if in SCH1 we do ALTER TABLE... on TBL2.. same changes has to be applied to A_TBL2 in SCH2.. I have no idea how i could manage this ALTER TABLE change..
Please any help would be greatly appreciated..
Here's the trigger i did for create table..
create or replace
TRIGGER DDL_APPLY_TO_SCH2
AFTER ALTER OR CREATE ON SCH1.SCHEMA
DECLARE
VSTR VARCHAR2(255);
VJOB NUMBER;
BEGIN
        IF ORA_DICT_OBJ_TYPE = 'TABLE' THEN
        VSTR := 'EXECUTE IMMEDIATE ''CREATE TABLE SCH2.A_'||ORA_DICT_OBJ_NAME||' AS SELECT * FROM '||ORA_DICT_OBJ_NAME||''';';
        DBMS_OUTPUT.PUT_LINE(VSTR);
        DBMS_JOB.SUBMIT( VJOB, VSTR );
        END IF;
END;Thanks - HP

Ora_User_2 wrote:
Hello All,
I have two schemas SCH1 and SCH2
SCH1 : TBL1
SCH2 : A_TBL1
Our requirement is whenever we create a table in SCH1 for Eg TBL2 there should be some code which will create A_TBL2 table automatically in SCH2.. Both TBL2 and A_TBL2 should have same structure.. This functionality i can manage using "ON CREATE OR ALTER" Trigger..
But now if in SCH1 we do ALTER TABLE... on TBL2.. same changes has to be applied to A_TBL2 in SCH2.. I have no idea how i could manage this ALTER TABLE change..
Please any help would be greatly appreciated..
Here's the trigger i did for create table..
create or replace
TRIGGER DDL_APPLY_TO_SCH2
AFTER ALTER OR CREATE ON SCH1.SCHEMA
DECLARE
VSTR VARCHAR2(255);
VJOB NUMBER;
BEGIN
IF ORA_DICT_OBJ_TYPE = 'TABLE' THEN
VSTR := 'EXECUTE IMMEDIATE ''CREATE TABLE SCH2.A_'||ORA_DICT_OBJ_NAME||' AS SELECT * FROM '||ORA_DICT_OBJ_NAME||''';';
DBMS_OUTPUT.PUT_LINE(VSTR);
DBMS_JOB.SUBMIT( VJOB, VSTR );
END IF;
END;Thanks - HPThis is not a good idea but you can use that,first you can check if object(table) exits then you can drop this table and can create again using VSTR := 'EXECUTE IMMEDIATE ''CREATE TABLE SCH2.A_'||ORA_DICT_OBJ_NAME||' AS SELECT FROM '||ORA_DICT_OBJ_NAME||''';'* it means trigger will fire after alter event du to there will not any problem,so first check if this table exits then drop and create again ,so you will get changed table.

How to Replicate tables at different servers??? Please Help!!!

Hi all,
Basically i have two tables with same table structure (one is master & slave )both on different server.
I just want to update slave table whenever Master table is updated with new records.Can this activity be scheduled???
Could somebody guide me how to achieve this...
Thanks in advance

Hi,
you could write a TRIGGER in the master table which would update the other table or you could have a procedure which you could run every hour or depending upon how frequently you want to update the other table. Every thing depnds upon how frequently the MASTER is updated, how quickly you want the information transferred to the other table ....
http://download.oracle.com/docs/cd/B14117_01/server.101/b10739/jobtosched.htm
Thanks

How to reset table in Status "Replicate - Error'?

Hi,
we have a HANA sidecar scenario (HANA revision 82) where we replicate tables from ECC non-HANA towards HANA with SLT for operational reporting (using DMIS_2011_1_731 SP7).
In total we have about 550 tables that are being replicated fine. There are however 2 tables that went into status "Replicate - Error": for instance table KONV. There are other large tables that replicate fine. When I have a look at the transaction LTRC, I find the following message:
- Error while generating runtime objects for Z_KONV_002
- Syntax error in function module /1CADMC/ACS_00100000000240
- Error message: Method "ADJUST_INTERVAL_LOWER_BOUNDARY" is unknown or private or protected
In attempt to solve this, we will load SAP notes 2041240 + 2016512 and test again.
But how can we reset a single table as if it has not been selected for replication before? Is this via a specific expert function in transaction LTRC and a physical deletion in RS-tables on HANA? This is needed to be able to start replication again for that single table, right?
Thanks in advance!
Stijn

Hi,
I'm assuming that you want to reload table KONV completely into HANA? Through transaction LTRC, once in the required mass transfer ID/Configuration, on the Table Overview Tab, press the Data Provisioning button and then enter the table name in the appropriate field. Select the 'Stop Load/Replication' option and execute (make sure the master job is running).
This will remove the triggers/logging tables in ECC and set the appropriate status for the table in HANA. You can check progress on the Table Overview tab...once KONV disappears, press the use the Data Provisioning button again and change the option to 'Start Replication' for KONV and execute. This will drop the data in HANA and then create the logging table and triggers again, then start the initial load phase and when the load is complete, the table should switch to 'replicate'.
Word of caution, you might want to maintain any required performance options/filters before you reload.
Hope this helps.
Kris

How streams replicate to a different table?

How can I replicate a shared object from source database to another table that has the different name of the source table ?
anyone who has this experience please give me a example, thanks .

In our environment
OS:solaris 5.9
DB:oracle 10.2.1
we use oracle streams to replicate tables from DB1 to DB2.for example
DB1: table site on test schema,
replicate to
DB2: table site_opc on xxrpt schema
because the shared database object (site) on DB1 have a different name and be in a different schema at the source database and destination database.I try to configure rule-based transformation to convert it.
on destination database(DB2),I execute following
BEGIN
DBMS_STREAMS_ADM.RENAME_TABLE(
rule_name => 'STRMADMIN.WBXSITE102',
from_table_name => 'test.SITE',
to_table_name => 'XXRPT.SITE_OPC',
step_number => 0,
operation => 'ADD');
END;
After configure capture, propagation on source and apply on destination database, I make a update on source database, but I don't find any change apply on DB2. And I query dba_apply_error, I can't find any error message. Why?

Best practice of metadata table in data warehouse environment ?

Hi guru's,
In datawarehouse, we have 1. Stage schema 2. DWH(Data warehouse reporting schema). In stageing we have about 300 source tables. In DWH schema, we are creating the tables which are only required from reporting prespective . some of the tables in stageing schema, have been created in DWH schema as well with different table name and column names. The naming convention for these same tables and columns in DWH schema is more based on business names.
In order to keep track of these tables we are creating metadata table in DWH schema say for example
Stage                DWH_schema
Table_1             Table_A
Table_2             Table_b
Table_3             Table_c
Table_4              Table_DMy question is how do we handle the column names in each of these tables. The stage_1, stage_2 and stage_3 column names have been renamed in DWH_schema which are part of Table_A, Table_B, Table_c.
As said earlier, we have about 300 tables in stage and may be around 200 tables in DWH schema. Lot of the column names have been renamed in DWH schema from stage tables. In some of the tables we have 200 column's
so my concern is how do we handle the column names in metadata table ? Do we need to keep only table names in metadata table not column names ?
Any idea will be greatly appriciated.
Thanks!

hi
seems quite a buzzing question.
In our project we designed a hub and spoke like architecture.
Thus we have 3 layer, L0 is the one closest to the source and L0 table's name are linked to the corresponding sources names by mean of naming standard (like tabA EXT_tabA tabA_OK1 so on based on implementation of load procedures).
At L1 we have the ODS , normalized model , we use business names for table there and standard names for temporary structures and artifacts
Both L0 an L1 keep source's column names as general rule, new columns like calculated one are business driven and metadata are standard driven.
Datamodeler fits perfect for modelling L1 purpose.
L2 is the dimensional schema business names take place for tables and columns eventually rewritten at presentation layer ( front end tool )
hope this helps D.

Procedure with multiple Loops and Insert into table

Hello All,
I am trying to create the Procedure to create a new table for our DWH.
In this Case, I have 2 tables and I need to create Loop for each value from one table and get the respective data from second table based on first table data.
Please find the below example:
First table: TABLE_A
X             Y              Z
1              a              10
1              b             abc
1              c              xy
1              e             $
2              a              11
2              c              asf
2              d             tal
2              f              ghs
Second Table: TABLE_B
A             B             C             D             E              F
10           abc         xy           sd           ew          100
10           jhy          xy           sd           ew          100
11           ght         asf          tal           ss            ghs
11           ght         afr          tal           ss            ghs
O/P Table from Procedure: OUTPUT_TABLE
A             B             C             D             E              F              X
10           abc         xy           sd           ew          100         1
11           ght         asf          tal           ss            ghs         2
Business Logic In the Procedure:
CREATE OR REPLACE PROCEDURE OP_TAB_PROCEDURE IS
BEGIN
                -- First get the DISTINCT values from the TABLE_A to generate the Loop
                FOR ID (SEELCT DISTINCT X FROM TABLE_A_
                LOOP
                                -- For Each ID get the Y ,Z values from TABLE_A other than $ in Z
                                                TEMP:
                                                SELECT Y, Z FROM TABLE_A WHERE X = ID.X AND Z <> '$' ;
                                                Based on above SELECT statement, I need to construct dynamic SQL query to get the data from TABLE_B
                                                OP_EACH_ID:
                                                SELECT * , ID.X AS X FROM TABLE_A WHERE [In this place I need generate sql cond dynamically. For Example, For X=1 in TABLE_A , the where cond must A=10 AND B='abc' AND C='xy'
                                                                                                                                                                                                                 For X=2 , the Where cond must be WHERE A=11 AND C='asf' AND D='tal' AND F='ghs']
                                                -- I need to INSERT all the values from OP_EACH_ID into OUT_PUT_TABLE
                END LOOP ;
END;
I am new to PL/SQL , so please help me on the above case.

duplicate post

Replication of S071 Table in CRM

Hi,
My requirement is to Replicate table S071 Condition Update Table from ECC to CRM in custom table (ZS071) through BDOC. Can you please guide me for the same.
As i want the cumulative entry to be stored in CRM, it should be updated when ever there is a change or update in ECC S071 table.
Thanks in advance.

Hi,
My requirement is to Replicate table S071 Condition Update Table from ECC to CRM in custom table (ZS071) through BDOC. Can you please guide me for the same.
As i want the cumulative entry to be stored in CRM, it should be updated when ever there is a change or update in ECC S071 table.
Thanks in advance.

Replicate Tables as Staging Tables in a DWH

Similar Messages

Maybe you are looking for