Best practices for initial data loads to MDM

Hi,
   We need to load more than 300000 vendors from SAP into MDM production repository. Import server might take days to load that much if no error occurs.
Are there any best practices for initial loads to MDM available? What considerations must be made while doing the initial loads.
Harsha

Hello Harsh
With SP05 patch1 there is a file aggregation functionality in the import port. Is is supposed to optimize the import performance.
BTW, give me your mail address and I will send you an idoc packaging paper for MDM.
Regards,
Goekhan

Similar Messages

  • Best Practice for initial data

    When installing a database onto a new server, I do not know what is the best solution to populate the database with the initial data that needs to exist before I can start running. I have sql scripts that take care of creating the tables, constraints, and triggers, but what is the best way to populate the database with my data?
    I think that writing sql inserts seem a little complicated and writing a free standing program to interface with the database would take a little longer than I want. Is there a tool available to do my initial population?
    Thanks in advance.

    I have similar situation to yours where I have to install the database with starting data. What I have done to make this process easier is that I have set up a schema that is an exact replica of what is out in production. This schema has all of the required data in which the application needs to order to be functional. I keep this schema updated with all changes. When the time comes to create another database I just export this schema and import it into the new database. The seed data is small so the export is small as well. This approach has saved me a lot of time in the new deployments.

  • Best practice for heirachical data

    First off, I have to say that JMX in Java 6 is terrific stuff. Bundling jconsole in with Java has made JMX adoption so much easier for us.
    Now, to my question. We have read-only hierarchical data (think a DOM tree) that we would like to publish via JMX. What is the best practice? We see two possibilities:
    1. Publish each node of the tree with it's own object name and type. This will allow jconsole to display the information in the tree control.
    2. Publish just the root of the tree with an object name and type and then use CompositeType to describe the nodes of the tree. This means you look at the tree in the "Attribute Value" panel of jconsole.
    Is there any best practices for such data? We have implemented #2 and it works but we are wondering if long term this might lead to unforeseen consequences.
    Thanks in advance.
    --Marty                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

    Path,
    I did go with #1 and it worked out great. Every node in our tree is an ObjectName node. Works very well for us.
    --Marty                                                                                                                                                                                                                                                                   

  • Best practice for sharing data with model window

    Hi team,
    what would the best practice for sharing data with a modal
    window be ? I use a modal window to display record details from a
    record list, but i am not quite sure how to access the data from
    the components in the main application in the modal window.
    Any hints would be welcome
    Best
    Frank

    Pass a reference to the parent into the modal popup. Then you
    can reference anything in the parent scope.
    I haven't done this i 2.0 yet so I can't give you code. I'll
    post if I do.
    Oh, also, you can reference the parent using parentDocument.
    So in the popup you could do:
    parentDocument.myPublicVariable = "whatever";
    Tracy

  • Best practice for initializing objects in a JSF backing bean?

    Hi,
    What is the best practice for initializing some objects in the JSF to-page backing bean before the to-page is displayed for the first time? The initialization would vary and depend upon a command link in the from-page.
    Regards,
    Al Malin

    f:view has two new attributes in 1.2: beforePhase and afterPhase
    which allows you to specify a phase listener method
    which will be called before and after the view is processed.

  • Best Practices on Routine Data Load.

    Can someone please tell me what are the best practices on routine data load from one database to another?
    We have PeopleSoft system where new employees' records are created; however, these new employees are required to take new employee tests that is being tracked by an application outside Peoplesoft on an Oracle db. Therefore, we need to populate the Oracle db with the new employee's information - on a daily basis or as needed. The data we will need to track are new employees or rehires, changes on existing employees - position, title, etc, terminated employees - date of termination, etc.
    What is the best practice to get the employee's information to the Oracle db?
    Any suggestions are appreciated.
    -andy

    Depends on your source and your database version which you didn't mention. What is the easiest way to get them out of your source database?
    Perhaps a database link though that might be a security violation.
    Perhaps as a delimited ASCII file loaded using SQL*Loader or an external table.
    Can you provide more information and database version numbers?

  • Best Practice for Initial Load Data

    Dear Experts,
        I would like to know the best practices or factors to be concerned when performing initial load
    For example,
    1) requirement from business stakeholders for data analysis
    2) age of data to meet tactical reproting
    3) data dependency crossing sap  modules
    4) Is there any best practice for loading master data?

    HI ,
    check this links
    Master Data loading
    http://searchsap.techtarget.com/guide/allInOne/category/0,296296,sid21_tax305408,00.html
    http://datasolutions.searchdatamanagement.com/document;102048/datamgmt-abstract.htm
    Regards,
    Shikha

  • Where to find best practices for tuning data warehouse ETL queries?

    Hi Everybody,
    Where can I find some good educational material on tuning ETL procedures for a data warehouse environment?  Everything I've found on the web regarding query tuning seems to be geared only toward OLTP systems.  (For example, most of our ETL
    queries don't use a WHERE statement, so the vast majority of searches are table scans and index scans, whereas most index tuning sites are striving for index seeks.)
    I have read Microsoft's "Best Practices for Data Warehousing with SQL Server 2008R2," but I was only able to glean a few helpful hints that don't also apply to OLTP systems:
    often better to recompile stored procedure query plans in order to eliminate variances introduced by parameter sniffing (i.e., better to use the right plan than to save a few seconds and use a cached plan SOMETIMES);
    partition tables that are larger than 50 GB;
    use minimal logging to load data precisely where you want it as fast as possible;
    often better to disable non-clustered indexes before inserting a large number of rows and then rebuild them immdiately afterward (sometimes even for clustered indexes, but test first);
    rebuild statistics after every load of a table.
    But I still feel like I'm missing some very crucial concepts for performant ETL development.
    BTW, our office uses SSIS, but only as a glorified stored procedure execution manager, so I'm not looking for SSIS ETL best practices.  Except for a few packages that pull from source systems, the majority of our SSIS packages consist of numerous "Execute
    SQL" tasks.
    Thanks, and any best practices you could include here would be greatly appreciated.
    -Eric

    Online ETL Solutions are really one of the biggest challenging solutions and to do that efficiently , you can read my blogs for online DWH solutions to know at the end how you can configure online DWH Solution for ETL  using Merge command of SQL Server
    2008 and also to know some important concepts related to any DWH solutions such as indexing , de-normalization..etc
    http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
    http://www.sqlserver-performance-tuning.com/apps/blog/show/12927103-data-warehousing-workshop-2-4-
    http://www.sqlserver-performance-tuning.com/apps/blog/show/12927173-data-warehousing-workshop-3-4-
    http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
    Kindly let me know if any further help is needed
    Shehap (DB Consultant/DB Architect) Think More deeply of DB Stress Stabilities

  • Best Practice for Initial Load

    Hello,
    what is the best way of doing the initial load? is there a best practice somwhere that tells you what should be imported first?
    I want to understand the order ex,
    1. load Lookups,
    2. Hierarchies,
    3. taxonomy and attributes
    last the main table
    etc...
    I dont understand the logic.
    Thanks in advance

    Hi Ario,
    If you follow any SAP Standard business content for MDM Repositories like e.g. Material.
    https://websmp130.sap-ag.de/sap/support/notes/1355137
    In the SAP Note attachments, you will get MDM71_Material_Content.pdf
    You will see Import of reference Data(look up table's data) 1st(step6) before import of Master data(step7).
    During Import of Reference Data(look up data), Please follow the Import Sequence by using Processing level 0,1,2 etc.
    Which take care of filling look up flat tables first then filling Hierarchies tables etc.
    After that if you are maintaining Taxonomy, You need to fill taxonomy table in Taxonomy mode of Data Manager, in the sequence (Categories, Attributes, Linkage between Attributes and Categories and lastly Attribute Values)
    After this I mean populating Reference data you need to populate Main table records along with tuples table data since now in MDM 7.1 Tuple has been replaced by Qualified table for most of the Master's but if you are still maintaining Qualified table you can import Mani table data along with Qualified table in a single step. Otherwise for Qualified table you can alos use this approach of populating Non-qualifeirs to Qualified table first before importing main table and then importing Main table data along with Qualifier's field of Qualified table.
    This above entire process for exporting data from SAP R/3 system to MDM. If you are importing data into MDM from legacy system (Non-Sap systems too), Approach should be remain same Populating Lookup tables data and lastly main table data.
    I dont understand the logic.
    The logic is simple in your main table you have fields which are look up to Reference tables( e.g. field in main table which are look up to Lookup flat tables like Countries, Currencies etc, field in main table which is lookup to Hierarchy/Taxonomy table etc). So, if these values are not populated firstly, so during your Main table import you will have incomplete data for all of these fields from main table which are look up to some other tables as values in your lookup table you haven't populated before Main table import.
    Kindly revert if you still have any doubts.
    Regards,
    Mandeep Saini

  • Best practice for extracting data to feed external DW

    We are having a healthy debate with our EDW team about extracting data from SAP.  They want to go directly against ECC tables using Informatica and my SAP team is saying this is not a best practice and could potentially be a performance drain.  We are recommending going against BW at the ODS level.  Does anyone have any recommendations or thoughts on this?

    Hi,
    As you asked for Best Practice, here it is in SAP landscape.
    1. Full Load or Delta Load data from SAP ECC to SAP BI (BW): SAP BI understand the data element structure of SAP ECC, and delta mechanism is the continous process of data load from a SAP ECC (transaction system) to BI (Analytic System).
    2. You can store transaction data in DSOs (granular level), and in InfoCubes (at a summrized level) within SAP BI. You can have master data from SAP ECC coming into SAP BI separately.
    3. Within SAP BI, you SHOULD use OpenHub service to provide SAP BI data to other external system. You must not connect external extractor to fetch data from DSO and InfoCube to target system. OpenHub service is the tool that faciliate data feeding to external system. You can have Informatica to take data from OpenHubs of SAP BI.
    Hope I explain to best of your satisfaction.
    Thanks,
    S

  • Best practice for migrating data tables- please comment.

    I have 5 new tables seeded with data that need to be promoted from a development to a production environment.
    Instead of the DBAs just using a tool to migrate the data they are insistent that I save and provide scripts for every single commit, in proper order, necessary to both build the table and insert the data from ground zero.
    I am very unaccustomed to this kind of environment and it seems much riskier for me to try and rebuild the objects from scratch when I already have a perfect, tested, ready model.
    They also require extensive documentation where every step is recorded in a document and use that for the deployment.
    I believe their rationale is they don't want to rely on backups but instead want to rely on a document that specifies each step to recreate.
    Please comment on your view of this practice. Thanks!

    >
    Please comment on your view of this practice. Thanks!
    >
    Sounds like the DBAs are using best practices to get the job done. Congratulations to them!
    >
    I have 5 new tables seeded with data that need to be promoted from a development to a production environment.
    Instead of the DBAs just using a tool to migrate the data they are insistent that I save and provide scripts for every single commit, in proper order, necessary to both build the table and insert the data from ground zero.
    >
    The process you describe is what I would expect, and require, in any well-run environment.
    >
    I am very unaccustomed to this kind of environment and it seems much riskier for me to try and rebuild the objects from scratch when I already have a perfect, tested, ready model.
    >
    Nobody cares if if is riskier for you. The production environment is sacred. Any and all risk to it must be reduced to a minimum at all cost. In my opinion a DBA should NEVER move ANYTHING from a development environment directly to a production environment. NEVER.
    Development environments are sandboxes. They are often not backed up. You or anyone else could easily modify tables or data with no controls in place. Anything done in a DEV environment is assumed to be incomplete, unsecure, disposable and unvetted.
    If you are doing development and don't have scripts to rebuild your objects from scratch then you are doing it wrong. You should ALWAYS have your own backup copies of DDL in case anything happens (and it does) to the development environment. By 'have your own' I mean there should be copies in a version control system or central repository where your teammates can get their hands on them if you are not available.
    As for data - I agree with what others have said. Further - ALL data in a dev environment is assumed to be dev data and not production data. In all environments I have worked in ALL production data must be validated and approved by the business. That means every piece of data in lookup tables, fact tables, dimension tables, etc. Only computed data, such as might be in a data warehouse system generated by an ETL process might be exempt; but the process that creates that data is not exempt - that process and ultimately the data - must be signed off on by the business.
    And the business generally has no access to, or control of, a development environment. That means using a TEST or QA environment for the business users to test and validate.
    >
    They also require extensive documentation where every step is recorded in a document and use that for the deployment.
    I believe their rationale is they don't want to rely on backups but instead want to rely on a document that specifies each step to recreate.
    >
    Absolutely! That's how professional deployments are performed. Deployment documents are prepared and submitted for sign off by each of the affected groups. Those groups can include security, dba, business user, IT and even legal. The deployment documents always include recovery steps so that is something goes wrong or the deployment can't procede there is a documented procedure of how to restore the system to a valid working state.
    The deployments themselves that I participate in have representatives from the each of those groups in the room or on a conference call as each step of the deployment is performed. Your 5 tables may be used by stored procedures, views or other code that has to be deployed as part of the same process. Each step of the deployment has to be performed in the correct order. If something goes wrong the responsible party is responsible for assisting in the retry or recovery of their component.
    It is absolutely vital to have a known, secure, repeatable process for deployments. There are no shortcuts. I agree, for a simple 5 new table and small amount of data scenario it may seem like overkill.
    But, despite what you say it simply cannot be that easy for one simple reason. Adding 5 tables with data to a production system has no business impact or utility at all unless there is some code, process or application somewhere that accesses those tables and data. Your post didn't mention the part about what changes are being made to actually USE what you are adding.

  • Obiee 11g : Best practice for filtering data allowed to user

    Hi gurus,
    I have a table of the allowed areas for each user.
    I want to show only the data facts associated with these allowed areas.
    For instance my user scott can see France and Italy data.
    I made a variable session. I put this session variable in a filter.
    It works ok but only one value (the first one i think) is taken in account (for instance, with my solution scott will see only france data).
    I need all the possible values.
    I tried with the row wise parameter of the variable session. But it doesn't work (error obiee).
    I've read things on internet about using stragg or valuelistof but neither worked.
    What would be the best practice to achieve this goal of filtering data with conditions by user stored in database ?
    Thanks in advance, Emmanuel

    Check this link
    http://oraclebizint.wordpress.com/2008/06/30/oracle-bi-ee-1013332-row-level-security-and-row-wise-intialized-session-variables/

  • Best Practice for Master Data Reporting

    Dear SAP-Experts,
    We face a challenge at the moment and we are still trying to find the right approach to it:
    Business requirement is to analyze SAP Material-related Master Data with the BEx Analyzer (Master Data Reporting)
    Questions they want to answer here are for example:
    - How many active Materials/SKUs do we have?
    - Which country/Sales Org has adopted certain Materials?
    - How many Series do we have?
    - How many SKUs below to a specific season
    - How many SKUs are in a certain product lifecycle
    - etc.
    The challenge is, that the Master Data is stored in tables with different keys in the R/3.
    The keys in these tables are on various levels (a selection below):
    - Material
    - Material / Sales Org / Distribution Channel
    - Material / Grid Value
    - Material / Grid Value / Sales Org / Distribution Channel
    - Material / Grid Value / Sales Org / Distribution Channel / Season
    - Material / Plant
    - Material / Plant / Category
    - Material / Sales Org / Category
    etc.
    So even though the information is available on different detail  levels, the business requirement is to have one query/report that combines all the information. We are currently struggeling a bit on deciding, what would be the best approach for this requirement. Did anyone face such a requirement before - and what would be the best practice. We already tried to find any information online, but it seems Master data reporting is not very well documented. Thanks a lot for your valuable contribution to this discussion.
    Best regards
    Lukas

    Pass a reference to the parent into the modal popup. Then you
    can reference anything in the parent scope.
    I haven't done this i 2.0 yet so I can't give you code. I'll
    post if I do.
    Oh, also, you can reference the parent using parentDocument.
    So in the popup you could do:
    parentDocument.myPublicVariable = "whatever";
    Tracy

  • Best practice for saving data in SQL server

    Hi all
    Hoping for a little help on this question. 
    If i have a list of fields ex. (name,address,postal,phone etc.). Then i create a webform/task
    to gather some of theese fields (name, postal), then i make another webform/task to gather some other fields (address, phone). 
    What is best practice in the SQL server for storing returning values.
    Is it: 
    1. to make a table with all the fields in the list + taskid. Theese fields could be in
    correct format (number, date etc.). And all answers to all tasks is inserted into this table. 
    2. Make a value table for each field with the correct type + task id. So all name values
    are stored in the "name value table" with the task id.
    How would i select values from a certain task from this kind of setup?
    3. ??
    Best regards
    Bo

    Hi Atul
    Thanks for your reply, can you elaborate a bit further on this, since i am still a little confused. 
    Let me try to explain my scenario at bit more:
    Say instead that it is 50 fields in a table with their own unique ID, maybe an answer table
    would look like this:
    taskid | field_1 | field_2 | field_3 | field 4 | field_n
    So no matter which fields the user fillsout it will can be stored in one table. 
    QUestion is, is this a good way to do it? and how do i select from this table using a join
    As far as i know you cant name columns in a table with just numbers, which would have been
    great, giving the columnnames the field_id.
    OR
    Would you have 50 tables each with a field_id and a value (of correct type) ?
    And could you give me an example of how to bind and select from this kind of structure ?
    Also inserting into 50 tables on a save.... is that the right way to go? :)
    Best regards
    Bo

  • Best Practices for Initial Setup

    Hi, I'm going to be helping a friend setup his new Time Capsule and Airport Express for his home network. I've been following some of the threads here about the nightmare problems some people are having with speed, connectivity, reliability, etc.
    So I was thinking it might be helpful to have a list of steps to take among setting up that might avoid some pitfalls. Feel free to add stuff not mentioned.
    1) Should firmware be updated first? What is current version?
    2) Would doing a '7-Pass Erase' on the TC first potentially avoid some issues (seems like this has helped some folks)?
    3) If you plan on storing other data on the TC yourself (not a TM backup), do you need to partition the drive first? What is the best method for this? Would this have any potential on slowing TM backups?
    4) What is best method for determining best channel to set TC to? (ie, are there ways to test?)
    5) Should you perform initial TM backups from each computer over ethernet before attempting wireless?
    6) What tools are there to accurately check your wifi and/or file transfer performance? Are there listed standards that a good working TC 'should' be to compare against.
    THANKS in advance. I'm trying to help them avoid some of the pitfalls people have encountered. Hopefully this can help others as well.

    1) I would update first Airport Utility is 5.3.2 and firmware 7.3.2
    2) not needed is over kill
    3) partition is a waste of valuable space, Airport Utility is not going to do that; you'll need to pull the drive and do it then you'll run into unusual problem over the long term.
    4) jump by at least 6

Maybe you are looking for