Ssis to load dimension table

i want to be able to generate a dimension out of fact table
the attribute is the name of a store which can be spelt anyway.
but in the dimension table, i want all the member values with similar spelling to correspond to a single store and mapped to a unique key.
how is this one using ssis
the fact table may have the followng for walmart
walmart
walmartab
walmart123
how to map all these walmarts to a single walmart and also associate a unique key
the dimension table should have
ID  storename  displayname
1 walmart    walmart123
2 walmart   walmart
3 walmart   walmartab
OR, would it be a better idea to clean the source and then have a dimension like below?
1 walmart
please advice wahat is the general best practice for designing as well as loading the dimension table
thanks very much

do you mean, the fact table needs to be cleansed? or can it be left alone with diffent values within the fact tablebeing mapped to a single value, which requires cleaning the dimension?
which is the preferred approach and why?
if the dimension is cleaned this would be the resulting dimension
ID  storename  displayname
1 walmart    walmart123
2 walmart   walmart
3 walmart   walmartab
OR, would it be a better idea to clean the source and then have a dimension like below?
1 walmart
Its dimension that needs cleansing. The fact will only have fk to dimension using key value isnt it? So you need to make sure you complete the cleansing in the dimension and based on which you map the key values correctly in fact to correspond to survived key
value for collapsed values
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Similar Messages

  • Best Practice loading Dimension Table with Surrogate Keys for Levels

    Hi Experts,
    how would you load an Oracle dimension table with a hierarchy of at least 5 levels with surrogate keys in each level and a unique dimension key for the dimension table.
    With OWB it is an integrated feature to use surrogate keys in every level of a hierarchy. You don't have to care about
    the parent child relation. The load process of the mapping generates the right keys and cares about the relation between the parent and child inside the dimension key.
    I tried to use one interface per Level and created a surrogate key with a native Oracle sequence.
    After that I put all the interfaces in to one big Interface with a union data set per level and added look ups for the right parent child relation.
    I think it is a bit too complicated making the interface like that.
    I will be more than happy for any suggestions? Thank you in advance!
    negib
    Edited by: nmarhoul on Jun 14, 2012 2:26 AM

    Hi,
    I do like the level keys feature of OWB - It makes aggregate tables very easy to implement if your sticking with a star schema.
    Sadly there is nothing off the shelf with the built in knowledge modules with ODI , It doesnt support creating dimension objects in the database by default but there is nothing stopping you coding up your own knowledge module (use flex fields maybe on the datastore to tag column attributes as needed)
    Your approach is what I would have done, possibly use a view (if you dont mind having it external to ODI) to make the interface simpler.

  • Update/Load Dimension table attributes(non-key elements)

    Hi,
    Is there a configuration change in DAC or Informatica where I can opt to load all the dimension tables during an incremental load, irrespective of an associated transaction in the fact table? I observed that for all those accounts with no corresponding transactions are extracted but dropped by the time they reach the actual dimension table. So in the final target DWH, only those unique accounts are brought which have an associated transaction.
    I know we can create an aux mapping to include some kind of a flag to trick DAC into thinking the corresponding base fact table also has changed and then bring over the dimension data. But is there a simple alternative?
    For example, In GL we have an account dimension which has an associated line transaction details for each account in the fact table. I just change the Account name from A to B. No change or updates to the associated line entry in the FACT table. Now I run an incremental load. Will the change ( new account name) be recognized and brought over? OR will it be brought over only when the associated line entry also changes?
    Usually when the dollar amounts dont change, nobody cares but in our case the client wants it. I have googled extensively to understand incremental loading but cud not get a clear response to the above question. Does SCD2 play any role in this?
    Thanks,
    Dan

    Ahsan,
    The question primarily revolves around dimension tables and incremental load. If we take out SCD1/2 out of the scene for now, could you please answer Yes or No for the below questions? Please it is kind of urgent as the client is really worried about these aspects of data transfer in incremental loads. As always, appreciate your quick response. Assume that first full load has been done.
    1. If we change a key field (effective date) of an account that does not have a new transaction associated with it in the ledger – will the change come over after an incremental load?
    2. If we change a non key field (title) on a prior effective date of an account that does not have a new transaction associated with it in the ledger – will the change come over after an incremental load?
    3. If we change a non key field (title) on a prior effective date of an account that does have a new transaction associated with it in the ledger – will it reflect after an incremental load?
    4. During a load process Full/Incremental, dimension tables get loaded first right? IF not, and facts are loaded first, wont they error out seeing that no associated dimension details exist?
    5. We have a few custom tables which dont have any sort of a datetimestamp columns. Is bringing them all over everytime load happens, the only option? What is the best method to handle these kind of situations?
    Thanks,
    Dan

  • Load fact table with null dimension keys

    Dear All,
    We have OWB 10g R2 and ROLAP star schema. In our source system some rows don’t have all attributes populated with values (null value), and this empty attributes are dimension (business) keys in star schema. Is it possible to load fact table with such rows (some dimension keys are null) in the OWB mappings? We use cube operator in mappings.
    Thanks And Regards
    Miran

    The dimension should have a row indicating UNKNOWN, this will have a business key outside of the normal range e.g. -999999.
    In the mapping the missing business keys can then be NVL'd to -999999.
    Cheers
    Si

  • ERROR while loading time dimension table

    i need to load time dimension from csv to oracle table, while loading i got the error.
    my source data type is date and target is date.
    ODI-1226: Step sample day fails after 1 attempt(s).
    ODI-1240: Flow sample day fails while performing a Loading operation. This flow loads target table W_SAMPLE_DATE.
    ODI-1228: Task SrcSet0 (Loading) fails on the target ORACLE connection Target_Oracle.
    Caused By: java.sql.SQLException: ORA-30088: datetime/interval precision is out of range
    while creating c$ table
    create table WORKSCHEMA.C$_0W_SAMPLE_DATE
         C3_ROW_WID     NUMBER(10) NULL,
         C1_CALENDAR_DATE     TIMESTAMP() NULL,
         C2_DAY_DT     TIMESTAMP() NULL
    )

    check the source data and use the correct function eg TO_DATE(SRC.DATE, 'MM?DD/YYYY') use NVL if required.

  • Incremental load into the Dimension table

    Hi,
    I have the problem in doing the incremental load of the dimension table.Before loading into the dimension table,i would like to check the data in the dimnesion table.
    In my dimension table i have one not null surrogate key and the other null dimension tables.The not null surrogate key, i am populating with the Sequence Generator.
    To do the incremental load i have done the following.
    I made lookup into the dimension table and looked for a key.The key from the lookup table i have passed to the expression operator.In the expression operator i have created one field and hard coded one flag based on the key from the lookup table.I passed this flag to the filter operator and rest of the fields from the source.
    By doing this i am not able to pass the new records to the dimension table.
    Can you please help me.
    I have another question also.
    How do i update one not null key in the fact table.
    Thanks
    Vinay

    Hi Mark,
    Thanks for your help to solve my problem.I thought i share more information by giving the sql.
    I am giving below the 2 sqls, i would like to achieve through OWB.
    Both the following tasks need to be accomplished after loading the fact table.
    task1:
    UPDATE fact_table c
    SET c.dimension_table_key =
    (SELECT nvl(dimension_table.dimension_table_key,0)
    FROM src_dimension_table t,
    dimension_table dimension_table
    WHERE c.ssn = t.ssn(+)
    AND c.date_src_key = to_number(t.date_src(+), '99999999')
    AND c.time_src_key = to_number(substr(t.time_src(+), 1, 4), '99999999')
    AND c.wk_src = to_number(concat(t.wk_src_year(+), concat(t.wk_src_month(+), t.wk_src_day(+))), '99999999')
    AND nvl(t.field1, 'Y') = nvl(dimension_table.field1, 'Y')
    AND nvl(t.field2, 'Y') = nvl(dimension_table.field2, 'Y')
    AND nvl(t.field3, 'Y') = nvl(dimension_table.field3, 'Y')
    AND nvl(t.field4, 'Y') = nvl(dimension_table.field4, 'Y')
    AND nvl(t.field5, 'Y') = nvl(dimension_table.field5, 'Y')
    AND nvl(t.field6, 'Y') = nvl(dimension_table.field6, 'Y')
    AND nvl(t.field7, 'Y') = nvl(dimension_table.field7, 'Y')
    AND nvl(t.field8, 'Y') = nvl(dimension_table.field8, 'Y')
    AND nvl(t.field9, 'Y') = nvl(dimension_table.field9, 'Y')
    WHERE c.dimension_table_key = 0;
    fact table in the above sql is fact_table
    dimesion table in the above sql is dimension_table
    source table for the dimension table is src_dimension_table
    dimension_table_key is a not null key in the fact table
    task2:
    update fact_table cf
    set cf.key_1 =
    (select nvl(max(p.key_1),0) from dimension_table p
         where p.field1 = cf.field1
    and p.source='YY')
    where cf.key_1 = 0;
    fact table in the above sql is fact_table
    dimesion table in the above sql is dimension_table
    key_1 is a not null key in the fact table
    Is it possible to achieve the above tasks through Oracle Warehouse builder(OWB).I created the mappings for loading the dimension table and fact table and they are working fine.But the above two queries i am not able to achieve through OWB.I would be thankful if you can help me out.
    Thanks
    Vinay

  • Loading dimensions using interface table

    Hello,
    I am trying to load dimensions into EPMA 11.1.1.3 using interface tables and need some direction. I have an accounts dimension file with the following columns ( Parent, Child, Alias, Description, DataType,Aggregation). I configured a data source using configuration wizard (Interface tables).
    I see in the database that some sample tables are created. Now, should I create a separate table with the columns listed above?
    Can someone let me know what are the next steps to successfully import accounts dimension?
    Thanks

    Can someone plz respond?
    This is important for me. I have gone thru' the documentation and also the sample tables but don't understand why are there multiple tables for a particular dimension viz. hierarchy, property etc?
    Please tell me the right way.
    Thanks again!!

  • Should my dimension table be this big

    Im in the process of building my first product dimension for a star schema and not sure if im doing this correctly. A brief explanation of our setup
    We have products (dresses) made by designers for specific collections each of which has a range of colors and each color can come in many sizes. For our UK market this equates to some 1.9 million
    product variants. Flattening the structure out to Product+designer+collection gives about 33,000 records but when you add all the colors and then all the colors sizes it pumps that figure up to 1.9million. My “rolled own” incremental ETL load runs
    ok just now, but we are expanding into the US market and our projections indicate that our products variants could multiple 10 fold. Im a bit worried about performance of the ETL just now and in the future.
    Is 1.9m records not an awful lot of records to incrementally load (well analyse) for a dimension table nightly as it is never mind what that figure may grow to when we go to US?
    I thought of somehow reducing this by using a snowflake but would this not just reduce the number of columns in the dimensions and not the row count?
    I then thought of separating the colors and size into their own dimension, but this doesn’t seem right as they are attributes of products and also I would lose the relationship between products,  size & color I.e. I would have to go through the
    fact table (which ive read is not a good choice.) for any analysis.
    Am I correct in thinking these are big numbers for a dimension table? Is it even possible to reduce the number somhow?
    Still learning so welcome any help.
    Thanks

    Hi Plip71,
    In my opinion, It is always good to reduce the Dimension Volume as much as possible for better performance.
    Is there any Hierarchy in you product Dimension?.. Going for a snowflake for this problem is a bad idea..
    Solution 1:
    From the details given by, It is good to Split the Colour and Size as seperate dimension. This will reduce the vloume of dimension and increase the column count in the fact(seperate WID has to be maintained in the fact table). but, this will improve the
    performance of the cube. before doint this please check the layout requirement from the business.
    Solution 2:
    Check the distinct count of Item varient used in fact table. If it is very less, they try creating a linear product dimension. i.e, create an view for the product dimension doing a inner join with the fact table. so that only the used Dimension member will
    be loaded in the Cube Product Dimension. hence volume is reduced with improvement in performance and stability of the cube.
    Thanks in advance, Sorry for the delayed reply ;)
    Anand
    Please vote as helpful or mark as answer, if it helps Regards, Anand

  • Static and small dimension table should I use lookup

    Hi Gurus,
    I am creating interface to load my fact table.
    Generally for different codes to replace with surrogate keys I use dimension table for look-up in interface.
    but I have question if dimension table is very small and has only static values like
    Sur_key, Code, Description.
    1, C, Create
    2 , R, Redeem
    in such case if in interface for fact table, I just write expression "if source_table.column=C then 1 else 2".
    Please advice if this is better way or not.
    Thanks in advance.

    I agree - it is much better practice to use lookups to populate surrogate keys. Using hard-coded values in code requires the developer to always remember that dependency on dimensional data and can cause unexpected results down the line if the keys are updated or are inconsistent across environments.

  • What is '#Distinct values' in Index on dimension table

    Gurus!
    I have loaded my BW Quality system (master data and transaction data) with almost equivalent volume as in Production.
    I am comparing the sizes of dimension and fact tables of one of the cubes in Quality and PROD.
    I am taking one of the dimension tables into consideration here.
    Quality:
    /BIC/DCUBENAME2 Volume of records: 4,286,259
    Index /BIC/ECUBENAME~050 on the E fact table /BIC/ECUBENAME for this dimension key KEY_CUBENAME2 shows #Distinct values as  4,286,259
    Prod:
    /BIC/DCUBENAME2 Volume of records: 5,817,463
    Index /BIC/ECUBENAME~050 on the E fact table /BIC/ECUBENAME for this dimension key KEY_CUBENAME2 shows #Distinct values as 937,844
    I would want to know why the distinct value is different from the dimension table count in PROD
    I am getting this information from the SQL execution plan, if I click on the /BIC/ECUBENAME table in the code. This screen gives me all details about the fact table volumes, indexes etc..
    The index and statistics on the cube is up to date.
    Quality:
    E fact table:
    Table   /BIC/ECUBENAME                    
    Last statistics date                  03.11.2008
    Analyze Method               9,767,732 Rows
    Number of rows                         9,767,732
    Number of blocks allocated         136,596
    Number of empty blocks              0
    Average space                            0
    Chain count                                0
    Average row length                      95
    Partitioned                                  YES
    NONUNIQUE  Index   /BIC/ECUBENAME~P:
    Column Name                     #Distinct                                       
    KEY_CUBENAMEP                                  1
    KEY_CUBENAMET                                  7
    KEY_CUBENAMEU                                  1
    KEY_CUBENAME1                            148,647
    KEY_CUBENAME2                          4,286,259
    KEY_CUBENAME3                                  6
    KEY_CUBENAME4                                322
    KEY_CUBENAME5                          1,891,706
    KEY_CUBENAME6                            254,668
    KEY_CUBENAME7                                  5
    KEY_CUBENAME8                              9,430
    KEY_CUBENAME9                                122
    KEY_CUBENAMEA                                 10
    KEY_CUBENAMEB                                  6
    KEY_CUBENAMEC                              1,224
    KEY_CUBENAMED                                328
    Prod:
    Table   /BIC/ECUBENAME
    Last statistics date                  13.11.2008
    Analyze Method                      1,379,086 Rows
    Number of rows                       13,790,860
    Number of blocks allocated       187,880
    Number of empty blocks            0
    Average space                          0
    Chain count                              0
    Average row length                    92
    Partitioned                               YES
    NONUNIQUE Index /BIC/ECUBENAME~P:
    Column Name                     #Distinct                                                      
    KEY_CUBENAMEP                                  1
    KEY_CUBENAMET                                 10
    KEY_CUBENAMEU                                  1
    KEY_CUBENAME1                            123,319
    KEY_CUBENAME2                            937,844
    KEY_CUBENAME3                                  6
    KEY_CUBENAME4                                363
    KEY_CUBENAME5                            691,303
    KEY_CUBENAME6                            226,470
    KEY_CUBENAME7                                  5
    KEY_CUBENAME8                              8,835
    KEY_CUBENAME9                                124
    KEY_CUBENAMEA                                 14
    KEY_CUBENAMEB                                  6
    KEY_CUBENAMEC                                295
    KEY_CUBENAMED                                381

    Arun,
    The cube in QA and PROD are compressed. Index building and statistics are also up to date.
    But I am not sure what other jobs are run by BASIS as far as this cube in production is concerned.
    Is there any other Tcode/ Func Mod etc which can give information about the #distinct values of this Index or dimension table?
    One basic question, As the DIM key is the primary key in the dimension table, there cant be duplicates.
    So, how would the index on Ftable on this dimension table show #distinct values less than the entries in that dimension table?
    Should the entries in dimension table not exactly match with the #Distinct entries shown in
    Index /BIC/ECUBENAME~P on this DIM KEY?

  • Dimension key 16 missing in dimension table /BIC/DZPP_CP1P

    Hi all,
    I have a problem with an infocube ZPP_CP1. I am not able to delete nor load any data. It was working fine till some time back.
    Below is the outcome of running RSRV check on this cube. I tried to run the error correction in RSRV. But n o use
    Please help.
    Dimension key 16 missing in dimension table /BIC/DZPP_CP1P
    Message no. RSRV018
    Diagnosis
    The dimension key 16 that appears as field KEY_ZPP_CP1P in the fact table, does not appear as a value of the DIMID field in the dimensions table /BIC/DZPP_CP1P.
    There are 17580 fact records that use the dimension key 16.
    The facts belonging to dimension key 16 are therefore no longer connected to the master data of the characteristic in dimension.
    Note that errors that are reported for the package dimension are not serious (They are thus shown as warnings (yellow) and not errors (red). When deleting transaction data requests, it can arise that the associated entries in the package dimension have already been deleted. As a result, the system terminates when deleting what can be a very large number of fact records. At the moment, we are working on a correction which will delete such data which remains after deletion of the request. Under no circumstances must you do this manually. Also note that data for request 0 cannot generally be deleted.
    The test investigates whether all the facts are zero. If this is the case, the system is able to remove the inconsistency by deleting these fact records. If the error cannot be removed, the only way to re-establish a consistent status is to reconstruct the InfoCube. It may be possible for SAP to correct the inconsistency, for which you should create an error message.
    Procedure
    This inconsistency can occur if you use methods other than those found in BW to delete data from the SAP BW tables (for example, maintaining tables manually, using your own coding or database tools).

    Hi Ansel,
                 There has been no changes in the cube. I am getting this problem in my QA server. So I retransported the cube again from Dev to QA. But did not help me..
    Any other ideas??
    Regards,
    Adarsh

  • Fact Tables without Dimension Tables.

    Hi,
    I am currently working on a project where a data warehouse is being developed using an application database as the source. Since the application database is already normalised, the tables are being loaded in as they are and the consultant involved is creating
    data marts for them. I have asked about why he is not creating separate dimension and fact tables as data marts, and he has told me that he will create a fact table with the dimensions as column names without any dimension tables created as you might expect
    as per star/snowflake schema.
    Is this right, as I always thought you had to have dimension and fact tables, especially if you want to start using SSAS?
    Regards,
    W.

    create a fact table with the dimensions as column names without any dimension tables created as you might expect as per star/snowflake schema.
    This is the trouble with denormalization: where is the stopping point?
    If the data warehouse is the source for SSAS OLAP cubes, it is better to go with the usual star schema.
    Kalman Toth Database & OLAP Architect
    SQL Server 2014 Database Design
    New Book / Kindle: Beginner Database Design & SQL Programming Using Microsoft SQL Server 2014

  • Missing most detailed table for dimension tables

    Hi ,
    I am getting this following error
    Business Model Core:
    [nQSError: 15003] Missing most detailed table for dimension tables: [Dim - Customer,Dim - Account Hierarchy,Dim - Account Region Hierarchy,Fact - Fins - Period Days Count].
    [nQSError: 15001] Could not load navigation space for subject area Core.
    I got this error when I tried to configure # of Elapsed Days and # of Cumulative Elapsed Days by following way-
    1. Using the Administration Tool, open OracleBIAnalyticsApps.rpd.
    Configuration Steps for Controlling Your Data Set
    Configuring Oracle Financial Analytics 5-51
    The OracleBIAnalyticsApps.rpd file is located at:
    ORACLE_INSTANCE\bifoundation\OracleBIServerComponent\coreapplication_
    obisn\repository
    2. In the Business Model and Mapping layer, go the logical table Fact - Fins - Period
    Days Count.
    3. Under Sources, select the Fact_W_DAY_D_PSFT logical table source.
    4. Clear the Disabled option in the General tab and click OK.
    5. Open the other two logical table sources, Fact_W_DAY_D_ORA and Fact_W_
    DAY_D_PSFT, and select the Disabled option.
    6. Add the "Fact - Fins - Period Days Count" and "Dim - Company" logical tables to
    the Business Model Diagram. To do so, right-click the objects and select Business
    Model Diagram, Selected Tables Only.
    7. In the Business Model Diagram, create a new logical join from "Dim - Company"
    to "Fact - Fins - Period Days Count." The direction of the foreign key should be
    from the "Dim - Company" logical table to the "Fact - Fins - Period Days Count"
    table. For example, on a (0,1):N cardinality join, "Dim - Company" will be on the
    (0/1) side and "Fact - Fins - Period Days Count" will be on the N side.
    8. Under the Fact - Fins - Period Days Count logical table, open the "# of Elapsed
    Days" and "# of Cumulative Elapsed Days" metrics, one at a time.
    9. Go to the Levels tab. For the Company dimension, the Logical Level is set to All.
    Click the X button to remove it. Repeat until the Company dimension does not
    have a Logical Level setting.
    10. Make sure to check Global Consistency to ensure there are no errors, and then
    save the RPD file.
    Please help me to resolve.
    Thanks,
    Soumitro

    Could you let me know how you resolved this. I am facing the same.

  • "Missing most detailed table for dimension tables" eror when I run the Global Consistency check

    ERRORS:
    Business Model DAC Measures:
    [nQSError: 15003] Missing most detailed table for dimension tables: [D_DETAILS,D_EXECUTION_PLAN,D_TASK].
    [nQSError: 15001] Could not load navigation space for subject area DAC Measures.
    I am also attaching my Business Model layer for easy understanding. I have a fact table and several Dimension table. I got this error only after creating the following hierarchies:
    Execution Plan -> Tasks -> Details
    Start Date Time Hierarchy
    End Date Time Hierarchy
    Is there a solution for this problem? Thanks in advance!

    Yes ! My Task Hierarchy has 3 dimension tables that form a hierarchy :Execution Plan -> Tasks -> Detail
    All the 3 levels in the hierarchy are 3 different dimension tables.

  • How to update a fact table when a dimension table is reloaded

    We have implemented BI Apps 796. Insertion into W_EMPLOYEE_D table which stores all the employee information had stopped one year back as some company security policy restricted the informatica worklfows to pick up the data. (PER_ALL_PEOPLE_F was a HRMS table and it contained sensitive information line SSN and salary, was inaccessible to the user which informatica uses and the SDE mapping used to return 0 rows).
    Now we have the approval to see those rows and the dimension table is loaded with some 100 new employees who joined in last one year.
    The ROW_WID of W_EMPLOYEE_D is referenced in lot of fact tables and for all those missing employees the WID in the fact table is 0.
    Now that we have all employees, how to make the FACT table point to the correct WID and not store 0. Has anyone faced this problem before?? Writing an update statement will be a tedious task as there are so many fact tables that join to w_employee_d. Also our company uses Sales, Procurement, Finance modules of OB Apps (which constitutes atleast 20 fact tables)
    Any guidance is appreciated. Thanks in advance

    Hello Kostis,
    thank you for your answer. I don't fully understand you. Can you show me short example, please? I create alias table for time dimension on Physical Layer - original table is TimeDayDim and I create aliases TimeDayDim1, TimeDayDim2, TimeDayDim3, TimeDayDim4. Then I create foreign key Fact.Time1 -> TimeDayDim1, Fact.Time2 -> TimeDayDim2, Fact.Time3 -> TimeDayDim3, Fact.Time4 -> TimeDayDim4. And what now? Must I create these table api Bussines Model and create new time dimensions at bussiness model????
    I need in Answers ONE Time dimension. I think I must split my fact table to four tables ... (time1, place1 ...) (time2, place2 ...) (time3 place3...) (time4 place4...) then link those tables to Time dimension (but I dont know where I can split those tables - on Physical Layer or on Bussines Layer).
    I suppose that I will have in Answers one time dimension and four facts tables and I will be able to query them. (for example: Time.Days, Fact1.Place1, Fact3.Speed, Fact4.Count Criteria: Time.Year = 2008)
    Best Regards Vlada

Maybe you are looking for