Best practice of metadata table in data warehouse environment ?

Hi guru's,
In datawarehouse, we have 1. Stage schema 2. DWH(Data warehouse reporting schema). In stageing we have about 300 source tables. In DWH schema, we are creating the tables which are only required from reporting prespective . some of the tables in stageing schema, have been created in DWH schema as well with different table name and column names. The naming convention for these same tables and columns in DWH schema is more based on business names.
In order to keep track of these tables we are creating metadata table in DWH schema say for example
Stage                DWH_schema
Table_1             Table_A         
Table_2             Table_b
Table_3             Table_c
Table_4              Table_DMy question is how do we handle the column names in each of these tables. The stage_1, stage_2 and stage_3 column names have been renamed in DWH_schema which are part of Table_A, Table_B, Table_c.
As said earlier, we have about 300 tables in stage and may be around 200 tables in DWH schema. Lot of the column names have been renamed in DWH schema from stage tables. In some of the tables we have 200 column's
so my concern is how do we handle the column names in metadata table ? Do we need to keep only table names in metadata table not column names ?
Any idea will be greatly appriciated.
Thanks!

hi
seems quite a buzzing question.
In our project we designed a hub and spoke like architecture.
Thus we have 3 layer, L0 is the one closest to the source and L0 table's name are linked to the corresponding sources names by mean of naming standard (like tabA EXT_tabA tabA_OK1 so on based on implementation of load procedures).
At L1 we have the ODS , normalized model , we use business names for table there and standard names for temporary structures and artifacts
Both L0 an L1 keep source's column names as general rule, new columns like calculated one are business driven and metadata are standard driven.
Datamodeler fits perfect for modelling L1 purpose.
L2 is the dimensional schema business names take place for tables and columns eventually rewritten at presentation layer ( front end tool )
hope this helps D.

Similar Messages

  • Best practice on extending the SIEBEL data model

    Can anyone point me to a reference document or provide from their experience a simple best practice on extending the SIEBEL data model for business unique data? Basically I am looking for some simple rules - based on either use case characteristics (need to sort and filter by, need to update frequently, ...) or data characteristics (transient, changes frequently, ...) to tell me if I should extend the tables, leverage the 'x' tables, or do something else.
    Preferably they would be prescriptive and tell me the limits of the different options from a use perspective.
    Thanks

    Accepting the given that Siebel's vanilla data model will always work best, here are some things to keep in mind if you need to add something to meet a process that the business is unwilling to adapt:
    1) Avoid re-using existing business component fields and table columns that you don't need for their original purpose. This is a dangerous practice that is likely to haunt you at upgrade time, or (worse yet) might be linked to some mysterious out-of-the-box automation that you don't know about because it is hidden in class-specific user properties.
    2) Be aware that X tables add a join to your queries, so if you are mapping one business component field to ATTRIB_01 and adding it to your list applets, you are potentially putting an unnecessary load on your database. X tables are best used for fields that are going to be displayed in only one or two places, so the join would not normally be included in your queries.
    3) Always use a prefix (usually X_ ) to denote extension columns when you do create them.
    4) Don't forget to map EIM extensions to the extension columns you create. You do not want to have to go through a schema change and release cycle just because the business wants you to import some data to your extension column.
    5) Consider whether you need a conversion to populate the new column in existing database records, especially if you are configuring a default value in your extension column.
    6) During upgrades, take the time to re-evalute your need for the extension column, taking into account the inevitable enhancements to the vanilla data model. For example, you may find, as we did, that the new version of the S_ADDR_ORG table had an ADDR_LINE_3 column, and our X_ADDR_ADDR3 column was no longer necessary. (Of course, re-configuring all your business components to use the new vanilla column can also be quite an ordeal.)
    Good luck!
    Jim

  • Best practices in Internal table naming convention GT ,  GS , LT  ,  LS  ??

    Hi Gurus,
         Are GT_ ,  GS_ ,  LT_  ,  LS_  --- the  Best practices in Internal table naming convention ????
         I  have  seen this  naming convetions adhered in standard programs .
         What each one  of  the  below  signify
         GT_ ,  GS_ ,  LT_  ,  LS_   ??????? 
    Regards
    Jaman
    Message was edited by:
            ABAP Techie

    Hello
    I use the following naming conventions:
    - G = global variable
    - L = local variable
    - T = internal table
    - S = structure
    - D = field
    That's how the combinations look like:
    - GT_ITAB     = global itab
    - GS_STRUC = global structure
    - GD_FIELD   = global field
    - LT_ITAB     = local itab
    - LS_STRUC = local structure
    - LD_FIELD   = local field
    Function module parameters have to stick to the following rules:
    - I = importing
    - E = exporting
    - [C = changing -> never used]
    - IT_ITAB = imported table type (itab)
    - IS_STRUC = imported structure
    - ID_FIELD   = imported field
    - ET_ITAB = exported table type (itab)
    - ES_STRUC = exported structure
    - ED_FIELD   = exported field
    Depending on their semantics TABLES parameters look like:
    - IT_ITAB = imported data
    - ET_ITAB = exported data
    - XT_ITAB = changing data (import & export)
    Here are the conventions for FORM routine parameters:
    - UT_ITAB = using itab (data are usually treated like constants; no changes will be transfer - although possible - to the calling program)
    - CT_ITAB = changing itab (if it is semantically an exporting itab then one of the very
    first statements in the routine is: REFRESH ct_itab. )
    - US_STRUCT
    - UD_FIELD
    - CS_STRUCT
    - CS_FIELD
    Conventions for class/interface parameters:
    - IT_ITAB = importing table type
    - IS_STRUC = importing structure
    - ID_FIELD = importing field
    - ET_ITAB = exporting table type
    - ES_STRUC = exporting structure
    - ED_FIELD = exporting field
    - RT_ITAB = returning table type
    - RS_STRUC = returning structure
    - RD_FIELD = returning field
    Conventions for class/interface attributes:
    - MT_ITAB = table type
    - MS_STRUC = structure
    - MD_FIELD = field
    - MC_CONST = constant
    <b>Question</b>: Are there any advantages of such elaborated naming conventions?
    My answer to this question is: Yes, definitively.
    I believe that the advantage of semantically differentiating TABLES parameters of function modules is quite obvious:
      CALL FUNCTION 'Z_BAD_NAMING'
        TABLES
           itab1 = ...
           itab2 = ...
           itab3 = ... .
      CALL FUNCTION 'Z_GOOD_NAMING'
        TABLES
           it_itab1 = ...
           et_itab2 = ...
           xt_itab3 = ... .
    I also believe that my naming conventions clearly enhance <b>readability </b>and <b>maintainability </b>of my programs.
    Regards
      Uwe

  • Best practice for Plan and actual data

    Hello, what is the best practice for Plan and actual data?  should they both be in the same app or different?
    Thanks.

    Hi Zack,
    It will be easier for you to maintain the data in a single application. Every application needs to have the category dimension, mandatorily. So, you can use this dimension to maintain the actual and plan data.
    Hope this helps.

  • Best practices in Queue table maintenance

    Hi Fellow AQ Users,
    I am looking to hear from the community about best practices in queue table maintenance.
    I have been mining through metalink about various Oracle recommendations and putting
    together a set of recommendations as a starting point for my DBAs.
    I am looking to answer questions like these --
    How often (in relation to messaging load) would you coalesce and rebuild the indexes?
    How often would you rebuild the table itself to get rid of the high water mark issues ?
    and what procedure would you use to do that?
    Would really love to learn from your experiences in this area. We are using 9.2.0.7
    64 bit DB and have plans to go to 10g over the next year. So, I am looking at 9i related
    stuff and then 10g.
    Thanks
    Vijay

    Hello,
    In general you coalesce once per day ideally during a quiet time to avoid ORA-54 errors as per <Note:271855.1>. Some customers do it more often than that but once per day is a good starting point.
    In terms of shrinking the queue tables you can use the procedure in <Note:304522.1> with a null 3rd parameter. This is an offline procedure so you could only run it during a maintenance window. In 10.2 onwards you can dynamically shrink the queue table and IOTS. Again it depends on exactly what you are doing with your queue tables how often you might need to do this.
    Thanks
    Peter

  • Best Practices to separate voice and Data vlans

    Hello All .
    I am coming to the community to get some advices on a specific subject .
    One of my customer is actually using vlan access-list to isolate it is data  from it is voice vlan traffic .
    As most of us knows VLAN ACLs are very difficult to deploy and manage at an access-port level that is highly mobile. Because of these management issues they have been looking for a replacement solution consisting of firewalls but apparently the price of the solution was too high in the sky .
    Can someone guide me towards security best practices when it comes to data and voice vlan traffic isolation please ?
    thanks
    Regards
    T.

    thomas.fayet wrote:Hi again Collin , May I ask you what type of fw / switches / ios version you are using for this topology ? Also is the media traffic going through your fw if one voice vlan wants to talk to another voice vlan ? rgds
    Access Switches: 3560
    Distro: 4500 or 6500
    FW: ASA5510 or Juniper SSG 140 (phasing out the Junipers)
    It depends. In the drawing above, no voice traffic would leave the voice enclave until it talks to a remote site. If we add other sites to the drawing, at a minimum call-sig would traverse the firewall and depending on the location of the callers, all voice traffic may cross the firewall. All of that depends on how you have your call managers/vm/voice gateways designed and where the callers are.

  • Best Practices for converting SAP HR data (4.7 to ECC)

    Hello Experts ...
    We are going from 4.6 to ECC ... no upgrade ..it will be a new implementation ...
    I am looking for best practices to convert SAP HR data from one sap instance(4.6) to another(ECC) ...
    I am not sure if direct input or LSMW or any other method/tool is the best way ...
    Will really appreciate and award point if I can get good advise or documentation ...
    Let me know if my question is not clear enough.
    Thanks,

    Hi
    You can check SAP Marketplace and there follow the download link. You will require a SAP Marketplace login to download the Best Practices. Fortunately Best practices for ECC 5.00 and 6.00 are available there but they are country specific versions. I know HCM for US is available there.
    Reward points, if helpful.
    Regards
    Waz

  • RMAN crosscheck and expire guidelines for data warehouse environment

    O/S: Windows Server 2008
    DB: Oracle 11gR2
    Are there any guidelines to how often one should do a RMAN crosscheck and set an expiration on archivelogs for data warehouse environments?
    It would seem once a day would be enough for the crosscheck in a data warehouse environment that gets refreshed nightly. Expiration I would expect no less than 1 week.
    Cheers!

    I agree with damorgan
    refer the below links for best practices.
    http://www.oracle.com/technetwork/database/features/availability/311394-132335.pdf
    https://blogs.oracle.com/datawarehousing/entry/data_warehouse_in_archivelog_m
    Hope this helps,
    Regards
    http://www.oracleracepxert.com
    Understand the Power of Oracle RMAN
    http://www.oracleracexpert.com/2011/10/understand-power-of-oracle-rman.html
    Duplicating RAC database using RMAN
    http://www.oracleracexpert.com/2009/12/duplicate-rac-database-using-rman.html

  • Are there any timesten installation for data warehouse environment?

    Hi,
    I wonder if there is a way to install timesten as an in memory database for data warehouse environment?
    The DW today consist of a large Oralcle database and I wonder if and how a timesten implementation can be done.
    what kind of application changes involve with such an implementation and so on?
    I know that the answer is probably complex but if anyone knows about such an implementation and some information about it , it would be great to learn from that experience.
    Thanks,
    Adi

    Adi,
    It depends on what you want to do with the data in the TimesTen database. If you know the "hot" dataset that you want to cache in TimesTen, you can use Cache Connect to Oracle to cache a subset of your Oracle tables into TimesTen. The key is to figure out what queries you want to run and see if the queries are supported in TimesTen.
    Assuming you know the dataset you need to cache and you have control of your application code to change the connection to TimesTen (using ODBC or JDBC), you can give it a try. If you are using a third party tool, you need to see if the tool supports JDBC or ODBC access to the database and change the tool to point to your TimesTen database instead of the Oracle database.
    If you are using the TimesTen Cache Connect to Oracle product option, data synchronization between Oracle and TimesTen is handled automatically by the product.
    Without further details of what you'd like to do, it's difficult to provide more detailed recommendation.
    -scheung

  • Best Practice to fetch SQL Server data and Insert into Oracle Tables

    Hello,
    I want to read sqlserver data everry half an hour and write into oracle tables ( in two different databases). What is the best practice for doing this?
    We do not have any database dblinks from oracle to sqlserver and vice versa.
    Any help is highly appreciable?
    Thanks

    Well, that's easy:
    use a TimerTask to do the following every half an hour:
    - open a connection to sql server
    - open two connections to the oracle databases
    - for each row you read from the sql server, do the inserts into the oracle databases
    - commit
    - close all connections

  • Best practice for having separate clone data for development purposes?

    Hi
    I am on a hosted Apex environment
    I have a workspace containing two instances/ copies of the application: DEV and PROD
    I would like to be able to develop functionality and data in/ with the DEV instance and then insert it into DEV.
    I gather that I can insert pages from DEV to PROD via Create -> New page as copy -> Page in another application
    But I don't know how I can mimic this process with database objects, eg. if I want to create a new table or manipulate the data in an existing table in a DEV environment before implementing in a PROD environment.
    Ideally this would be done in such a way that minimises changing table names etc when elevating pages from DEV to PROD.
    Would it be possible to create a clone schema that could contain the same tables (with the same names) as PROD?
    Any tips, best practices appreciated :)
    Thanks

    Hi,
    ideally you should have a little more separation between your dev and prod environments. At the minimum you should have separate workspaces each addressing separate schemas. Apex can be a little difficult if you want to move individual Apex application objects, such as pages, between applications (a much requested improvement), but this can be overcome by exporting and importing the whole application. You should also have some form of version control/backup of export files.
    As far as database objects go, tables etc, if you have tns access to your hosted environment, then you can use SQL Developer to develop, maintain and synchronize between your development and production schemas and objects in the different environments should have identical names. If you don't have that access, then you can use the Apex SQL Workshop features, but these are a little more cumbersome than a tool like SQL Developer. Once again, scripts for creating and upgrading your database schemas should be kept under some sort of version control.
    All of this is supposing your hosting solution allows more than one workspace and schema, if not you may have to incur the cost of a second environment. One other option would be to do your development locally in an instance of Oracle XE, ensuring you don't have any version conflicts between the different database object features and the Apex version.
    I hope this helps.
    Regards
    Andre

  • Spatial tables in Data warehouse database or separate?

    Hi all,
    I have a question from a customer who's asking what the recommendation is of storage of the spatial tables and the warehouse tables.
    Should they reside on the same database for best performance or doesn't it matter where the spatial tables are located.
    Is there any official recommednation from Oracle on this?
    Best regards,
    Hakan

    Hi Hakan,
    it's not easy to give a quick answer here.
    a) you want to compare warehouse tables with spatial tables
    To answer this very well you should define what is a warehouse table and what is a spatial table. In this forum you will get for spatial tables the definition "A spatial table is a table with one or more columns of type SDO_GEOMETRY". How do you define a spatial table and how do you define a warehouse table?
    b) you ask for performance
    Performance is not depending only on table type. Performance is depending on a lot of steps. For example on the work which has to be done to answer the question, can data read from buffer cache or must they be read from physical devices, execution plan .....
    I don't know if there is a official statement from Oracle which is usefull to answer your query.
    Can you ask a little bit particular please?
    In my experience there it is not adviseable to store tables in different databases (instances) when I have to query it together. If you split the tables in different instances, the execution has much more overhead because you have to talk to 2 instances and it will very hard for the optimizer to find the best execution plan.
    Regards
    U. Martin

  • In the Begining it's Flat Files - Best Practice for Getting Flat File Data

    I probably should have posed this question here before I delved into writing Java to get data for reports, but better late than never.
    Our ERP is written in COBOL. We have a third party ODBC which allows us to access data using a version of SQL. I have several Java sources compiled in my database that access the data and return something relevant. The Java sources are written in a procedural style rather than taking advantage of object oriented programming with attributes and methods.
    Now that I am becoming more comfortable with the Java language, I would greatly appreciate any feedback as to best practices for incorporating Java into my database.
    My guess is that it would be helpful to model the ERP "tables" with Java classes that would have attributes, which correspond to the fields, and methods to return the attributes in an appropriate way. Does that sound reasonable? If so, is there a way to automate the task of modeling the tables? If not reasonable, what would you recommend?
    Thanks,
    Gregory

    Brother wrote:
    I probably should have posed this question here before I delved into writing Java to get data for reports, but better late than never.
    Our ERP is written in COBOL. We have a third party ODBC which allows us to access data using a version of SQL. I have several Java sources compiled in my database that access the data and return something relevant. The Java sources are written in a procedural style rather than taking advantage of object oriented programming with attributes and methods.
    OO is a choice not a mandate. Using Java in a procedural way is certainly not ideal but given that it is existing code I would look more into whether is well written procedural code rather than looking at the lack of OO.
    Now that I am becoming more comfortable with the Java language, I would greatly appreciate any feedback as to best practices for incorporating Java into my database.
    My guess is that it would be helpful to model the ERP "tables" with Java classes that would have attributes, which correspond to the fields, and methods to return the attributes in an appropriate way. Does that sound reasonable? If so, is there a way to automate the task of modeling the tables? If not reasonable, what would you recommend?Normally you create a data model driven by business need. You then implement using whatever means seem expedient in terms of other business constraints to closely model that data model.
    It is often the case that there is a strong correlation between data models and tables but certainly in my experience it is rare when there are not other needs driven by the data model (such as how foreign keys and link tables are implemented and used.)

  • Best practice for including additional DLLs/data files with plug-in

    Hi,
    Let's say I'm writing a plug-in which calls code in additional DLLs, and I want to ship these DLLs as part of the plug-in.  I'd like to know what is considered "best practice" in terms of whether this is ok  (assuming of course that the un-installer is set up to remove them correctly), and if so, where is the best place to put the DLLs.
    Is it considered ok at all to ship additional DLLs, or should I try and statically link everything?
    If it's ok to ship additional DLLs, should I install them in the same folder as the plug-in DLL (e.g. the .8BF or whatever), in a subfolder of the plug-in folder or somewhere else?
    (I have the same question about shipping additional files too, such as data or resource files.)
    Thanks
                             -Matthew

    Brother wrote:
    I probably should have posed this question here before I delved into writing Java to get data for reports, but better late than never.
    Our ERP is written in COBOL. We have a third party ODBC which allows us to access data using a version of SQL. I have several Java sources compiled in my database that access the data and return something relevant. The Java sources are written in a procedural style rather than taking advantage of object oriented programming with attributes and methods.
    OO is a choice not a mandate. Using Java in a procedural way is certainly not ideal but given that it is existing code I would look more into whether is well written procedural code rather than looking at the lack of OO.
    Now that I am becoming more comfortable with the Java language, I would greatly appreciate any feedback as to best practices for incorporating Java into my database.
    My guess is that it would be helpful to model the ERP "tables" with Java classes that would have attributes, which correspond to the fields, and methods to return the attributes in an appropriate way. Does that sound reasonable? If so, is there a way to automate the task of modeling the tables? If not reasonable, what would you recommend?Normally you create a data model driven by business need. You then implement using whatever means seem expedient in terms of other business constraints to closely model that data model.
    It is often the case that there is a strong correlation between data models and tables but certainly in my experience it is rare when there are not other needs driven by the data model (such as how foreign keys and link tables are implemented and used.)

  • OBJECT TABLES in DATA WAREHOUSE

    Can anyone give a high level response of the benefits as well as
    drawbacks of using Nested object tables as a permanent means of
    storage in an ORACLE data warehouse. In particular, is there a
    performance enhancement/degradation, maintenance problems,
    scalability etc in comparison to standard HEAP tables.
    Most books that I have read suggest NOT to use object tables as
    a permanent means of storage.
    Thanks for any help that you can provide.

    Hi Hakan,
    it's not easy to give a quick answer here.
    a) you want to compare warehouse tables with spatial tables
    To answer this very well you should define what is a warehouse table and what is a spatial table. In this forum you will get for spatial tables the definition "A spatial table is a table with one or more columns of type SDO_GEOMETRY". How do you define a spatial table and how do you define a warehouse table?
    b) you ask for performance
    Performance is not depending only on table type. Performance is depending on a lot of steps. For example on the work which has to be done to answer the question, can data read from buffer cache or must they be read from physical devices, execution plan .....
    I don't know if there is a official statement from Oracle which is usefull to answer your query.
    Can you ask a little bit particular please?
    In my experience there it is not adviseable to store tables in different databases (instances) when I have to query it together. If you split the tables in different instances, the execution has much more overhead because you have to talk to 2 instances and it will very hard for the optimizer to find the best execution plan.
    Regards
    U. Martin

Maybe you are looking for