Loading data into Fact/Cube with surrogate keys from SCD2

We have created 2 dimensions, CUSTOMER & PRODUCT with surrogate keys to reflect SCD Type 2.
We now have the transactional data that we need to load.
The data has a customer id that relates to the natural key of the customer dimension and a product id that relates to the natural key of the product dimension.
Can anyone point us in the direction of some documentation that explains the steps necessary to populate our fact table with the appropriate surrgoate key?
We assume that we need to have an lookup table between the current version of the customer and the incoming transaction data - but not sure how to go about this.
Thanks in advance for your help.
Laura

Hi Laura
There is another way to handling SCD and changing Facts. This is to use a different table for the history. Let me explain.
The standard approach has these three steps:
1. Determine if a change has occurred
2. End Date the existing record
3. Insert a new record into the same table with a new Start Date and dummy End Date, using a new surrogate key
The modified approach also has three steps:
1. Determine if a change has occurred
2. Copy the existing record to a history table, setting the appropriate End Date en route
3. Update the existing record with the changed information giving the record a new Start Date, but retaining the original surrogate key
What else do you need to do?
The current table can use the surrogate key as the primary key with the natural key being being a unique key. The history table has the surrogate key and the end date in the primary key, with a unique key on the natural key and the end date. For end user queries which in more than 90% of the time go against current data this method is much faster because only current records are in the main table and no filters are needed on dates. If a user wants to query history and current combined then a view which uses a union of the main and historical data can be used. One more thing to note is that if you adopt this approach for your dimension tables then they always keep the same surrogate key for the index. This means that if you follow a strict Kimball approach to make the primary key of the fact table be a composite key made up of the foreign keys from each dimension, you NEVER have to rework this primary key. It always points to the correct dimension, thereby eliminating the need for a surrogate key on the fact table!
I am using this technique to great effect in my current contract and performance is excellent. The splitter at the end of the map splits the data into three sets. Set one is for an insert into the main table when there is no match on the natural key. Set two is when there is a match on the natural key and the delta comparison has determined that a change occurred. In this case the current row needs to be copied into history, setting the End Date to the system date en route. Set three is also when there is a match on the natural key and the delta comparison has determined that a change occurred. In this case the main record is simply updated with the Start Date being reset to the system date.
By the way, I intend to put a white paper together on this approach if anyone is interested.
Hope this helps
Regards
Michael

Similar Messages

  • Getting error while loading  Data into ASO cube by flat file.

    Hi All,
    i am getting this error Essbase error 1270040: Data load buffer[1] does not exist while loading data into ASO cube.
    does anyone have solution.
    Regards,
    VM

    Are you using ODI to load the data or maxl? If you are using an ODI interface, are you using a load rule also which version of essbase and ODI are you using
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • Unable to load data into Planning cube

    Hi,
    I am trying to load data into a planning cube using a DTP.
    But the system throws a message saying that, real time data loading cannot be performed to the planning cube.
    What would be the possible reason for the same?
    Thanks & Regards,
    Surjit P

    Hi Surjit,
    To load data into cube using DTP, it should be put in loading mode only.
    The real time cube behaviour is set to planning mode only when the data is input to the cube via the planning layouts / or through the Fileupload program.
    You can change the behaviour of the cube in RSA1 -> Right click on the cube -> Change Real-time load behaviour and select the first option(real time cube can be loaded with data; planning not allowed)
    Best Rgds
    Shyam
    Edited by: Syam K on Mar 14, 2008 7:57 AM
    Edited by: Syam K on Mar 14, 2008 7:59 AM

  • Loading data into two cubes

    We want to set up a delta refresh from R/3 data that will pull data into two cubes.  One cube I want all the records to be loaded, in the second cube I want to filter the records that are loaded into that cube.  I can figure out how to load the data into two cubes, but can I place a filter on one of the loads.  (i.e. only load records of where type = 'X')
    Thanks,
    Chris

    You can do that in the Update Rules to the second cube... In the Start Routine...
    DELETE FROM DATA_PACKAGE
    WHERE type EQ 'X'.
    (Please, verify the right syntax)
    Then with only one Infopackage, you can load both cube with different conditions.
    Hope it helps.
    Regards,
    Luis

  • Loading data into infocube in bi 7.0 from flat file.

    Hello  All,
    I need the complete procedure for loading data into the infocube from flat file in bi 7.0 i.e.using transformation,DTP etc.
    Please help me with some gud documents.

    Hi Pratighya,
    Step by step procedure for loading data from flat file.
    1. Create the infoobjects you might need in BI
    2. Create your target infoprovider
    3. . Create a source system
    4. Create a datasource
    5. Create and configure an Infopackage that will bring your records to the PSA
    6. Create a transformation from the datasource to the Infoprovider
    7. Create a Data Transfer Process (DTP) from the datasource to the Infoprovider
    8. Schedule the infopackage
    9. Once succesful, run the DTP
    10. This will fill your target.
    Hope this helps
    Regards
    Karthik

  • Error while loading data into the cube

    Hi,
    I loaded data on to PSA and when I am loading the data to the cube through DataTransferProcess, I get an error (red color).
    Through "Manage", I could see teh request in red. How do I get to knoe the exact error? Also what could be the possibel reason for this?
    Also can some one explain the Datatransfer process(not in process chain)?
    Regards,
    Sam

    Hi Sam
        after you load the data  through DTP(after click on execute button..) > just go to monitor screen.. in that press  the refresh button..> in that it self.. you can find the  logs..
       otherwise.. in the request  screen also..  beside of the request number... you can see the logs icon.. you can click on this..
    DTP  means..
    DTP-used for data transfer process from psa to data target..
    check thi link..for DTP:
    http://help.sap.com/saphelp_nw04s/helpdata/en/42/f98e07cc483255e10000000a1553f7/frameset.htm
    to load data in to the datatargets or infoproviders like DSO, cube....
    in your  case.. the problem may be.. check the date formats.. special cherectrs... and
    REGARDS
    @JAY

  • Unable to load data into Essbase cube using Essbase Studio

    Hi
    We are creating an essbase cube using Essbase Studio using flat files as data sources.
    We have taken different hierarchies into different flat files and created one fact file having dimension intersection along with data.
    We are able to create the cube and the hierarchy but not able to load any data.
    We are getting the following error
    Failed to deploy Essbase cube.
    Caused by: Unable to perform dataload from more than one flat file.
    Could anyone please help on this?

    Oh this was killing me, so I did this test in 11.1.1.3:
    1) Excel 2007 format -- no go, Essbase didn't see it
    2) Excel 2003 format, three sheets -- only the first sheet was read into an empty rule
    3) Excel 2003 format, one sheet -- the first sheet was read into an empty rule
    4) Excel 95 format, one sheet -- the first sheet was read into an empty rule
    The lesson?
    1) Excel 2007/2010 sheets don't work (no surprise there as the .xlsx format isn't supported).
    2) Excel 2003 and lower (hey, if you have Excel 4, I'll bet that works as well) work, but only the first sheet is recognized.
    Regards,
    Cameron Lackpour

  • Error while loading data into cube

    hi BW gurus,
    when ever i am trying to load data into the cube from flat file after scheduling iam getting short dump in BW system. I checked it in st22 it is giving me a error as exception add_partition_failed. please help me to sort out this problem. If you know the error recovery please give me the answer in detail.
    I will assign points for good answers.

    This is what the note says:
    Symptom
    The process of loading transaction data fails because a new partition cannot be added to the F fact table.The loading process terminates with a short dump.
    Other terms
    RSDU_TABLE_ADD_PARTITION_ORA, RSDU_TABLE_ADD_PARTITION_FAILED, TABART_INCONSITENCY, TSORA, TAORA , CATALOG
    Reason and Prerequisites
    The possible causes are:
    SQL errors when creating the partition
    Inconsistencies in the Data Dictionary control tables TAORA and TSORA
    Solution
    BW 3.0A & BW 3.0B
    In the case of SQL errors:Analyze the SQL code in the system log or short dump and if possible, eliminate the cause. The cause is often a disk space problem or lock situations on the database catalog or, less frequently: the partitioning option in the ORACLE database is not installed.
    The most common cause of the problem is inconsistencies in the TAORA and TSORA tables. As of Support Package 14 for BW 3.0B/Support Package 8 for BW 3.1C, the TABART_INCONSITENCY exception is issued in this case. The reason is almost always missing entries in TSORA for the tablespaces of the DDIM, DFACT and DODS data classes.
    The TAORA table contains the assignment of data classes to data tablespaces and their attributes, for example:
    Tablespace Data class
    DDIM PSAPDIMD ........
    DFACT PSAPFACTD ........
    DODS PSAPODSD .......
    Foreach data tablespace, the TSORA table must contain an entry for the corresponding index tablespace, for example:
    TABSPACE INDSPACE
    PSAPDIMD PSAPDIMD
    PSAPFACTD PSAPFACTD
    PSAPODSD PSAPODSD
    In most cases, these entries are missing and have to be added. See also notes 502989 and 46272.

  • Short Dump when loading Data into Cube UNCAUGHT_EXCEPTION(Pls Help)

    Dear Guru's
    i am trying to load data into a Cube from a ODS and the load gets dump...
    In Dump Analysis.....
    The exception 'ZCX_EDW_MAP_ROOT' was raised, but it was not caught anywhere         
    along   
    "UNCAUGHT_EXCEPTION" "ZCX_EDW_MAP_ROOT"                                       
    "ZCL_EDW_MAP_ROOT==============CP" or "ZCL_EDW_MAP_ROOT==============CM002"   
    "LOAD_MAPPING_INFOPROV"                                                                               
    So it would be great if some one can give me some suggestion to handle this issue
    Thanks in Adv
    Dev

    Dear Arun,
    i have checked the routine there exists some code
    Eg: data: vref_monitor type ref to zcl_edw_monitor,
    But to say fact i ddnot i understood it....
    Any suggestions dude...pls
    thanks in adv
    Dev

  • Best Practice loading Dimension Table with Surrogate Keys for Levels

    Hi Experts,
    how would you load an Oracle dimension table with a hierarchy of at least 5 levels with surrogate keys in each level and a unique dimension key for the dimension table.
    With OWB it is an integrated feature to use surrogate keys in every level of a hierarchy. You don't have to care about
    the parent child relation. The load process of the mapping generates the right keys and cares about the relation between the parent and child inside the dimension key.
    I tried to use one interface per Level and created a surrogate key with a native Oracle sequence.
    After that I put all the interfaces in to one big Interface with a union data set per level and added look ups for the right parent child relation.
    I think it is a bit too complicated making the interface like that.
    I will be more than happy for any suggestions? Thank you in advance!
    negib
    Edited by: nmarhoul on Jun 14, 2012 2:26 AM

    Hi,
    I do like the level keys feature of OWB - It makes aggregate tables very easy to implement if your sticking with a star schema.
    Sadly there is nothing off the shelf with the built in knowledge modules with ODI , It doesnt support creating dimension objects in the database by default but there is nothing stopping you coding up your own knowledge module (use flex fields maybe on the datastore to tag column attributes as needed)
    Your approach is what I would have done, possibly use a view (if you dont mind having it external to ODI) to make the interface simpler.

  • Error encountered while loading data into an Essbase cube from oracle DB

    Hi All,
    I am trying to load data into an Essbase cube from Oracle db. Essbase is installed in an 64 bit HP-UX machine and oracle database on HP -UX 11.23 B server. While performing the above I am getting the below error messages:
    K/INFO - 1021013 - ODBC Layer Error: [08001] ==> [[DataDirect][ODBC Oracle Wire Protocol driver][Oracle]ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    HP-UX Error: 2: No such file or directory].
    OK/INFO - 1021014 - ODBC Layer Error: Native Error code [1034] .
    ERROR - 1021001 - Failed to Establish Connection With SQL Database Server. See log for more information.
    Can you please help me out in identifying the issue, as why it could have occured , because of network problem or is there anything related to databse?
    Regards,
    Priya

    The error is related to Oracle, I would check that it is all up and running
    ORA-01034: ORACLE not available
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • Loading Data into Table with Complex Transformations

    Hello Guys
    I am trying to load data into one of the Dimension table and it has quite a few Transformations and i created 6 temp tables
    1. It has 7 Columns , Gets 935 rows based on where condition
    2. It has 10 Columns , Gets 935 rows with but it has nulls in it i.e for column 1 there are 500 fields and columns 2 there are 300 etc ...
    3 , 4 , 5 , 6 all the same as the 2 table
    and at the end when i am trying to join all the temp tables with the Product_id into the target , which is in each temp table ...
    I am Getting Error Saying Not Obeying Primary key Constraints i.e unique values are not been inserting into the Product_id Column of Target Table and the Job is running for Hours
    and the main Problem comes at , some of the Columns have the same Product_id
    Please help me
    I have been Trying for 1 week and i am in Full pressure
    Thanks
    Sriks
    Edited by: Sriks on Oct 16, 2008 6:43 PM

    Hi,
    If you are creating a warehouse and product_key is ur PK then it should come only once and so u might have to think ur logic in getting the data. To get over the isue u can disable the constraint and load with out the cosntraint, but i would have u look at the logic and make sure u have only 1 product_key in the table.
    Regards
    Bharath

  • Steps for loading data into the infocube in BI7, with dso in between

    Dear All,
    I am loading data into the infocube in BI7, with dso in between. By data flow looks like....
    Top to bottom:
    InfoCube (Customized)
    Transformation
    DSO (Customized)
    Transformation
    DataSource (Customized).
    The mapping and everything else looks fine and data is also seen in the cube on the FULL Load.
    But due to some minor error (i guess), i am unable to see the DELTA data in DSO, although it is loaded in DataSource through process chains.
    Kindly advise me, where did i miss.
    OR .. Step by step instructions for loading data into the infocube in BI7, with dso in between would be really helpful.
    Regards,

    Hi,
    my first impulse would be to check if the DSO is set to "direct update". In this case there is no Delta possible, because the Change log is not maintained.
    My second thought would be to check the DTP moving data between the DSO and the target cube. If this is set to full, you will not get a delta. It is only possible to create one DTP. So if you created one in FULL mode you can't switch to Delta. Just create the DTP in Delta mode.
    Hope this helps.
    Kind regards,
    Jürgen

  • How can I load data into table with SQL*LOADER

    how can I load data into table with SQL*LOADER
    when column data length more than 255 bytes?
    when column exceed 255 ,data can not be insert into table by SQL*LOADER
    CREATE TABLE A (
    A VARCHAR2 ( 10 ) ,
    B VARCHAR2 ( 10 ) ,
    C VARCHAR2 ( 10 ) ,
    E VARCHAR2 ( 2000 ) );
    control file:
    load data
    append into table A
    fields terminated by X'09'
    (A , B , C , E )
    SQL*LOADER command:
    sqlldr test/test control=A_ctl.txt data=A.xls log=b.log
    datafile:
    column E is more than 255bytes
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)
    1     1     1     1234567------(more than 255bytes)

    Check this out.
    http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96652/ch06.htm#1006961

  • Getting an error while trying to load data into cube

    Hello,
    I am trying to load data into cube but it gives me following error:
    fiscal year variant not expected.
    Can someone tell what could be done on fiscal year
    Cheers
    Jim.

    Hi,
    May be you have not mapped anything to fiscal variant or not included fiscal variant in the cube but you have used fiscal period inside it.
    You should load fiscal variant if you wanna use fiscal period.
    Map fiscal variant to a constant K4 or from the source if present and then see the result.
    Generally it is K4 except for FI-CO cubes .
    In the transfer rules, select field '0fiscvarnt' 'Tp' click
    Enter K4 for Constant................Save and Activate
    Iam sure you'd fix the issue.
    Regards,
    Ray

Maybe you are looking for

  • Moved Premiere Elements 10 project Error: "Project is not compatible with current version of Premiere Elements"

    After moving an entire client folder from an external hard drive to an internal one, the client's video, created in adobe premiere elements 10, will no longer open in adobe premiere elements 10. It appears to be loading the project, and instead of co

  • Purchase order output error....

    Hi, Purchase order is created from a shopping cart.But PO output(via email) is not created.I am getting output error.Error log is showing message as "Incorrectly processed". Error message description is "SUM OF PERCENTAGE EXCEEDS..." and "TABLE TEM_D

  • Is there a file size limit for App Store downloads over wifi?

    I just gifted my friend a navigation app that is 1.4 Gb in size. Unbeknown to me, her iPhone is not tethered to a computer and therefore she only downloads apps directly to the phone via wifi. Is it possible to download a file of this size directly t

  • How do I use a shape as a Path in Photoshop?

    i've tried creating a path in Photoshop and tried dragging the shape from the library but it will not drag to a path. When I drag a shape to a layer in Photoshop it is rasterized. How do I get a shape to go onto a path in Photoshop? Also, If this can

  • Abnormal increase in the oracle data base of XI dev

    Hi Experts, The size of the oracle data base for SAP-XI is increased by 9% abnormally in 1 day, where as 1% increase in the oracle data base size normally takes 1 to 2 weeks. What could be the problem? Please give me some links or some idea how to ch