Best Practice loading Dimension Table with Surrogate Keys for Levels

Hi Experts,
how would you load an Oracle dimension table with a hierarchy of at least 5 levels with surrogate keys in each level and a unique dimension key for the dimension table.
With OWB it is an integrated feature to use surrogate keys in every level of a hierarchy. You don't have to care about
the parent child relation. The load process of the mapping generates the right keys and cares about the relation between the parent and child inside the dimension key.
I tried to use one interface per Level and created a surrogate key with a native Oracle sequence.
After that I put all the interfaces in to one big Interface with a union data set per level and added look ups for the right parent child relation.
I think it is a bit too complicated making the interface like that.
I will be more than happy for any suggestions? Thank you in advance!
negib
Edited by: nmarhoul on Jun 14, 2012 2:26 AM

Hi,
I do like the level keys feature of OWB - It makes aggregate tables very easy to implement if your sticking with a star schema.
Sadly there is nothing off the shelf with the built in knowledge modules with ODI , It doesnt support creating dimension objects in the database by default but there is nothing stopping you coding up your own knowledge module (use flex fields maybe on the datastore to tag column attributes as needed)
Your approach is what I would have done, possibly use a view (if you dont mind having it external to ODI) to make the interface simpler.

Similar Messages

Update/Load Dimension table attributes(non-key elements)

Hi,
Is there a configuration change in DAC or Informatica where I can opt to load all the dimension tables during an incremental load, irrespective of an associated transaction in the fact table? I observed that for all those accounts with no corresponding transactions are extracted but dropped by the time they reach the actual dimension table. So in the final target DWH, only those unique accounts are brought which have an associated transaction.
I know we can create an aux mapping to include some kind of a flag to trick DAC into thinking the corresponding base fact table also has changed and then bring over the dimension data. But is there a simple alternative?
For example, In GL we have an account dimension which has an associated line transaction details for each account in the fact table. I just change the Account name from A to B. No change or updates to the associated line entry in the FACT table. Now I run an incremental load. Will the change ( new account name) be recognized and brought over? OR will it be brought over only when the associated line entry also changes?
Usually when the dollar amounts dont change, nobody cares but in our case the client wants it. I have googled extensively to understand incremental loading but cud not get a clear response to the above question. Does SCD2 play any role in this?
Thanks,
Dan

Ahsan,
The question primarily revolves around dimension tables and incremental load. If we take out SCD1/2 out of the scene for now, could you please answer Yes or No for the below questions? Please it is kind of urgent as the client is really worried about these aspects of data transfer in incremental loads. As always, appreciate your quick response. Assume that first full load has been done.
1. If we change a key field (effective date) of an account that does not have a new transaction associated with it in the ledger – will the change come over after an incremental load?
2. If we change a non key field (title) on a prior effective date of an account that does not have a new transaction associated with it in the ledger – will the change come over after an incremental load?
3. If we change a non key field (title) on a prior effective date of an account that does have a new transaction associated with it in the ledger – will it reflect after an incremental load?
4. During a load process Full/Incremental, dimension tables get loaded first right? IF not, and facts are loaded first, wont they error out seeing that no associated dimension details exist?
5. We have a few custom tables which dont have any sort of a datetimestamp columns. Is bringing them all over everytime load happens, the only option? What is the best method to handle these kind of situations?
Thanks,
Dan

Load fact table with null dimension keys

Dear All,
We have OWB 10g R2 and ROLAP star schema. In our source system some rows don’t have all attributes populated with values (null value), and this empty attributes are dimension (business) keys in star schema. Is it possible to load fact table with such rows (some dimension keys are null) in the OWB mappings? We use cube operator in mappings.
Thanks And Regards
Miran

The dimension should have a row indicating UNKNOWN, this will have a business key outside of the normal range e.g. -999999.
In the mapping the missing business keys can then be NVL'd to -999999.
Cheers
Si

Loading data into Fact/Cube with surrogate keys from SCD2

We have created 2 dimensions, CUSTOMER & PRODUCT with surrogate keys to reflect SCD Type 2.
We now have the transactional data that we need to load.
The data has a customer id that relates to the natural key of the customer dimension and a product id that relates to the natural key of the product dimension.
Can anyone point us in the direction of some documentation that explains the steps necessary to populate our fact table with the appropriate surrgoate key?
We assume that we need to have an lookup table between the current version of the customer and the incoming transaction data - but not sure how to go about this.
Thanks in advance for your help.
Laura

Hi Laura
There is another way to handling SCD and changing Facts. This is to use a different table for the history. Let me explain.
The standard approach has these three steps:
1. Determine if a change has occurred
2. End Date the existing record
3. Insert a new record into the same table with a new Start Date and dummy End Date, using a new surrogate key
The modified approach also has three steps:
1. Determine if a change has occurred
2. Copy the existing record to a history table, setting the appropriate End Date en route
3. Update the existing record with the changed information giving the record a new Start Date, but retaining the original surrogate key
What else do you need to do?
The current table can use the surrogate key as the primary key with the natural key being being a unique key. The history table has the surrogate key and the end date in the primary key, with a unique key on the natural key and the end date. For end user queries which in more than 90% of the time go against current data this method is much faster because only current records are in the main table and no filters are needed on dates. If a user wants to query history and current combined then a view which uses a union of the main and historical data can be used. One more thing to note is that if you adopt this approach for your dimension tables then they always keep the same surrogate key for the index. This means that if you follow a strict Kimball approach to make the primary key of the fact table be a composite key made up of the foreign keys from each dimension, you NEVER have to rework this primary key. It always points to the correct dimension, thereby eliminating the need for a surrogate key on the fact table!
I am using this technique to great effect in my current contract and performance is excellent. The splitter at the end of the map splits the data into three sets. Set one is for an insert into the main table when there is no match on the natural key. Set two is when there is a match on the natural key and the delta comparison has determined that a change occurred. In this case the current row needs to be copied into history, setting the End Date to the system date en route. Set three is also when there is a match on the natural key and the delta comparison has determined that a change occurred. In this case the main record is simply updated with the Start Date being reset to the system date.
By the way, I intend to put a white paper together on this approach if anyone is interested.
Hope this helps
Regards
Michael

Best practices in Queue table maintenance

Hi Fellow AQ Users,
I am looking to hear from the community about best practices in queue table maintenance.
I have been mining through metalink about various Oracle recommendations and putting
together a set of recommendations as a starting point for my DBAs.
I am looking to answer questions like these --
How often (in relation to messaging load) would you coalesce and rebuild the indexes?
How often would you rebuild the table itself to get rid of the high water mark issues ?
and what procedure would you use to do that?
Would really love to learn from your experiences in this area. We are using 9.2.0.7
64 bit DB and have plans to go to 10g over the next year. So, I am looking at 9i related
stuff and then 10g.
Thanks
Vijay

Hello,
In general you coalesce once per day ideally during a quiet time to avoid ORA-54 errors as per <Note:271855.1>. Some customers do it more often than that but once per day is a good starting point.
In terms of shrinking the queue tables you can use the procedure in <Note:304522.1> with a null 3rd parameter. This is an offline procedure so you could only run it during a maintenance window. In 10.2 onwards you can dynamically shrink the queue table and IOTS. Again it depends on exactly what you are doing with your queue tables how often you might need to do this.
Thanks
Peter

Best practice identifying ERT modules with SAP / IS-Utilities

Hi everybody,
I'm looking for the best practice identifying ERT modules with SAP / IS-Utilities (electricity).
Here's the physical device set up :
The ERT modules are internal to the electricity meter. They're integrated into a multi purpose electronic circuit. So they can't be remove physically as a separate device.
The ERT modules are used to transmit data from the meter to a radio frequency receiver (handheld or drive-by). The main data that is transmitted is the consumption reading. So the receiver stores the ERT module number and the reading value.
They may be one or more ERT modules in a single meter, and each ERT module transmit his own specific consumption reading (energy reading, demand reading, etc...).
Each ERT module has is own manufacturer number.
My issue is :
To find a way to identify in IS-U the ERT module within the meter's register group (or somewhere else???) in order to relate each register to his ERT module number.
The purpose of all this is to create reading orders with the ERT module number following for each register.
This way we can match, using a unique key, each reading order and his corresponding reading value uploaded from the radio frequency receiver (handheld or drive-by).
Thanks for your help and ideas on best practice.

Hi,
1) The system (application) environment of BI (what is integrated in it - e. g. within the portal, there is a storage for unstructured information like documents or virtual rooms for collaboration between departments - and what does it make)
Document management from RSA1 transaction of BI helps to attach any unstructured documents at specific level in BI.
2) How does development in BI works (development environment, coding, debugging, building, deployment and test) and what is used stronger (ABAP or ABAP OO)? Here, I don't mean how to write ABAP or ABAP OO programs, only the infrastructure from development to transport to a target system
BI has got a separate tool and GUI to perform all the Extract, Transform and load related activities. ABAP is part of BI but you don't need much extensive ABAP learning. Basic ABAP is sufficient to write routines and extractors.
3) How is a BI system to configure as default after installation?
May be a BASIS person can help you out here about the configuration but this is not the job of BI person.
4) Good guides (e/books) to learn ABAP and ABAP OO (as far as possible oriented on the practive)
You can search for SAM Series learn ABAP in 24 days book. This book is sufficient to learn the ABAP required for working in BI.
But except ABAP you will have to completly learn the BI system to work efficiently.
Regards,
Durgesh.

How we use Surrogate Keys for snowflake dimension

Hi All,
my question is - How we use Surrogate Keys for snowflake dimension
i heard from some body Surrogate Keys only work with star schema.
please correct me if i wrong.
Regards,
Manish

Hi manishcal16PPS,
According to your description, you can only create natural key in your dimension. But it's not working when using surrogate key. Right?
In Analysis Services, the snowflake schema of the dimensions are represented by more than one dimension table in other words its takes multiple dimension tables to define a dimension. Surrogate key are just some extra, redundant, unique key based on the
natural key. So there's no direct relationship or some limitations between surrogate keys and snowflake schema.
In this scenario, since there's relationship between the two dimensions, you should create natural key. For using natural key or surrogate key. Please refer to an article below:
Surrogate Key vs. Natural Key
For understanding star/snowflake schema, please see:
Understanding Star and Snowflake Schemas
Regards,
Simon Hou
TechNet Community Support

Best practice when using Tangosol with an app server

Hi,
I'm wondering what is the best practice when using Tangosol with an app server (Websphere 6.1 in this case). I've been able to set it up using the resource adapter, tried using distributed transactions and it appears to work as expected - I've also been able to see cache data from another app server instance.
However, it appears that cache data vanishes after a while. I've not yet been able to put my finger on when, but garbage collection is a possibility I've come to suspect.
Data in the cache survives the removal of the EJB, but somewhere later down the line it appear to vanish. I'm not aware of any expiry settings for the cache that would explain this (to the best of my understanding the default is "no expiry"), so GC came to mind. Would this be the explanation?
If that would be the explanation, what would be a better way to keep the cache from being subject to GC - to have a "startup class" in the app server that holds on to the cache object, or would there be other ways? Currently the EJB calls getCacheAdapter, so I guess Bad Things may happen when the EJB is removed...
Best regards,
/Per

Hi Gene,
I found the configuration file embedded in coherence.jar. Am I supposed to replace it and re-package coherence.jar?
If I put it elsewhere (in the "classpath") - is there a way I can be sure that it has been found by Coherence (like a message in the standard output stream)? My experience with Websphere is that "classpath" is a rather ...vague concept, we use the J2CA adapter which most probably has a different class loader than the EAR that contains the EJB, and I would rather avoid to do a lot of trial/error corrections to a file just to find that it's not actually been used.
Anyway, at this stage my tests are still focused on distributed transactions/2PC/commit/rollback/recovery, and we're nowhere near 10,000 objects. As a matter of fact, we haven't had more than 1024 objects in these app servers. In the typical scenario where I've seen objects "fade away", there has been only one or two objects in the test data. And they both disappear...
Still confused,
/Per

Replacing natural keys with surrogate keys

Hi,
We have database that is runnig with some old applications since many years. In the database, all the primary key are natural keys (composed with many columns). The applications are developped with mod pl/sql and oracle forms6i / designer6i.
We are planning to develop a new system (replacing all the old applications), the database tables will ramain the same but we are thinking about replacing the natural keys with surrogate keys. Since we are planning to develop and migrate into the new system gradually (oracle forms applications will not be redeveloped first), my question is:
- will the oracle forms applications still works?
- what is the risk and precautions to take ?
- is the decision to convert natural keys into surrogate a good one?
All your comments and propositions are welcome.
Thanks.

Surrogate keys in general is a good idea. Many generators cannot handle multiple column PK's very well, or not at all. I believe, for instance, that Apex can handle 2 columns only.
But, doing this on an existing datamodel can be a lot of work. You have to change your tables, of course, but also the foreign keys, triggers, constraints etc. A PK is protected from an update, but I guess you also want your old, natural PK, to be protected as well.
Forms generates master-detail synchronization based on FK relations. If these change, you may have to change your Forms too. As long as you don't do anything in your existing form, my guess is that it will still work. The forms will maintain the FK relation between old table definitions. This works independently of the actual FK relation in the database. You can create triggers on the tables to actually populate the new PK and FK.
For something like this: give it a try in a small test setup first. Than calculate the risk and impact.

Maintenance View for custom table with foreign key relationship

Hi Folks,
I have created a custom table with foreign key relationship with other check tables. I want to create a maintenance view / tablemaintenance generator. What all things I need to take care for the foreign keys related fields while creating the maintenance view / tablemaintenance generator.
Regards,
santosh

Hi,
You do not have to do anything explicitely for the foreign key relationships in the table maintainance generator.
Create the table maintainance generator via SE11 and it will take care of all teh foreign key checks by itself.
Regards,
Ankur Parab

How to insert record in child table with foreign key

Hi,
I am using Jdeveloper 11.1.2.0. I have two master table one child table.
How to insert and update a record in child table with foreign key ?
I have created VO based on three EO(one eo is updatable other two eo are references) by using joined query.
Thanks in Advance
Edited by: 890233 on Dec 24, 2011 10:40 PM

... And here is the example to insert using sequenceimpl by getting the primary key of the master record and insert master and detail together.
Re: Unable to insert a new row with a sequence generated column id
-Arun

Best practices in Internal table naming convention GT , GS , LT , LS ??

Hi Gurus,
 Are GT_ , GS_ , LT_ , LS_ --- the Best practices in Internal table naming convention ????
 I have seen this naming convetions adhered in standard programs .
 What each one of the below signify
 GT_ , GS_ , LT_ , LS_ ???????
Regards
Jaman
Message was edited by:
 ABAP Techie

Hello
I use the following naming conventions:
- G = global variable
- L = local variable
- T = internal table
- S = structure
- D = field
That's how the combinations look like:
- GT_ITAB = global itab
- GS_STRUC = global structure
- GD_FIELD = global field
- LT_ITAB = local itab
- LS_STRUC = local structure
- LD_FIELD = local field
Function module parameters have to stick to the following rules:
- I = importing
- E = exporting
- [C = changing -> never used]
- IT_ITAB = imported table type (itab)
- IS_STRUC = imported structure
- ID_FIELD = imported field
- ET_ITAB = exported table type (itab)
- ES_STRUC = exported structure
- ED_FIELD = exported field
Depending on their semantics TABLES parameters look like:
- IT_ITAB = imported data
- ET_ITAB = exported data
- XT_ITAB = changing data (import & export)
Here are the conventions for FORM routine parameters:
- UT_ITAB = using itab (data are usually treated like constants; no changes will be transfer - although possible - to the calling program)
- CT_ITAB = changing itab (if it is semantically an exporting itab then one of the very
first statements in the routine is: REFRESH ct_itab. )
- US_STRUCT
- UD_FIELD
- CS_STRUCT
- CS_FIELD
Conventions for class/interface parameters:
- IT_ITAB = importing table type
- IS_STRUC = importing structure
- ID_FIELD = importing field
- ET_ITAB = exporting table type
- ES_STRUC = exporting structure
- ED_FIELD = exporting field
- RT_ITAB = returning table type
- RS_STRUC = returning structure
- RD_FIELD = returning field
Conventions for class/interface attributes:
- MT_ITAB = table type
- MS_STRUC = structure
- MD_FIELD = field
- MC_CONST = constant
Question: Are there any advantages of such elaborated naming conventions?
My answer to this question is: Yes, definitively.
I believe that the advantage of semantically differentiating TABLES parameters of function modules is quite obvious:
CALL FUNCTION 'Z_BAD_NAMING'
 TABLES
 itab1 = ...
 itab2 = ...
 itab3 = ... .
CALL FUNCTION 'Z_GOOD_NAMING'
 TABLES
 it_itab1 = ...
 et_itab2 = ...
 xt_itab3 = ... .
I also believe that my naming conventions clearly enhance readability and maintainability of my programs.
Regards
Uwe

How to convert the table with separate fields for every period

Hi there,
I have the table in R/3 called COSP. The table contains the separate fields for values of separate reporting periods, let's say Amount01, Amount02 etc. I need to extract this data to BW and load it into ODS, which contains the data like common key figure for amount and info object Fiscal Period. How to extract and tansform the data to be able to load it there?
I mean, how to change the row like:
100 100 200 200....
for rows like:
001 100
002 100
003 200
004 200
I hope you'll help,
Kooyot

Hi,
you need to implement a bit of coding for that.
With the 'do varying' statement abap provides a loop over those fields. Within this loop you can build up a new internal table with one entry for each period and amount.
kind regards
Siggi
PS: I am not very familiar with CO, but I guess there will be a standard extractor for that.

Surrogate key for fact

Hi all,
i don't know that in ETL update of existing records in fact can happen or not. IF it happen then how can i handle it? is it recommendable to implement surrogate key for fact tables to update the existing records of fact? That work like primary key for fact table. This doesn't have any relation with OWB. This is a general concept about ETL.
Regards,
Sumanta

Yes, you can update a fact.
You generally do this by adding a unique constraint to the fact table which comprises the identifying primary key columns from the source tables, but exclusive of the surrogate key. The surrogate key will be the actual primary key of the table.
Now, when you make an insert/update mapping you include a sequence that generates your surrogate key, and then select the 'match by' on the fact table to match by the unique key (actual source table values which determine uniqueness) rather than the primary key. Then, on the surrogate key column on the fact table which is being populated by the sequence, you set its property to not update the surrogate key when an update is being applied to the row.
This way the update statement will not update the surrogate key when updating the fact based upon the source identifiers that identify this fact.
I'm sure I could explain this more clearly... but I'm very pressed for time right now.
Cheers,
Mike

Cdot with ssh keys for domain accounts

Has anyone on this board got ssh working with domain keys for cdot??

I use multiple keys; it is easy enough to manage them using keyring, and it means that I am able to compartmentalize them according to use case: work (which I obviously have a professional interest as well as personal in protecting) home (one for each box), and then keys for specific tasks (eg., automated backups, acess to particular services like github, mercurial etc).
This means that if one key is compromised, the others are unaffected and I can revoke the compromised key and, after cleaning up the mess as best I can, generate another and move on.
The only system I employ is to give each key a meaningful name (having multiple keys named id_{d,r}sa doesn't scale at all) and a policy of only adding the minimum necessary keys to each box's keyring; again, entering all the passphrases with any frequency helps manage this tendency.
I am also very careful about the key on my android as I see this as the most obvious risk: losing your phone is a pain; losing your phone and potentially relinquishing the key on it would be catastrophically asinine...

Best Practice loading Dimension Table with Surrogate Keys for Levels

Similar Messages

Maybe you are looking for