Best option to load data

Hi,
I have a DW table which has around 300K rows with country and state information along with other columns..
My Scenario:
I need to display country and state in drop down in front end for user to select them as search criteria..
currently application populates dropdown of country and state by selecting directly from DW table by doing DISTINCT of country and state as they are repeated in the table and hence taking long time to load the values in dropdown.
Also the selection in dropdown is such that once user selected say United States from country drop down, again the call is made to DW table and distinct states/provinces for United states are populated in state dropdown..
the DW table gets updated although not so frequently...
My query is I need to tune this scenario and hence I am thinking of 2 options as of now..
1) use materialized view which will get refreshed as and when the change is made to DW table as the timeing of changes in DW table is not so fixed...
2) Create a same replica table in my DB and run a batch job which will populate the table by truncating it and repopulating the whole table...this batch job will run once in a day..
Which is the above two solution is more feasible in terms of improving the performance of the above scenario.. or are there any other better alternative then pls suggest..
Thx for your help friends.

1. Yes, fixing the datawarehouse model to include a country & state dimension is the best approach.
2. This doesn't help that much as
SELECT DISTINCT country, state FROM fact_table;will still need to be done but this will now be done with a 'fast full scan' of the index rather than the table.
A better solution would be to create a materialized view with the disctint country & state information in it, and refresh after reloading the fact table, either programmatically or on demand.
Cheers,
Colin

Similar Messages

  • Best Way to Load Data in Hash Partition

    Hi,
    I have partitioning by Hash on a Large Table of 5 TB. We have to load Data say more than 500GB daily on that table from ETL.
    What is the best way to Load data into that Big Table which has hash Partition .
    Regards
    Sahil Soni

    Do you have any specific requirements to match records to lookup tables or it just a straight load - that is an insert?
    Do you have any specific performance requirements?
    The easiest and fastest way to load data into Oracle is via external file and parallel query/parallel insert. Remember that parallel DML is not enabled by default and you have to do so via alter session command. You can leverage multiple CPU cores and direct path operation to perform the load.
    Assuming your database is on a linux/unix server - you could NFS load the file if it is on a remote system, but then you will be most likely limited by network transfer speed.

  • Best way to load data in table from combination of Table and flat file?

    Hi All,
    Could you please share your thoughts on best way of achieving this objective -
    Flat File - 15 Million records (Field A,B,C)
    Table A - 15 Million records ( Field A,D)
    Objective -
    Load Field A,B,C,D in Table B from Flat file and Table A.
    Data can be loaded from flat file in Table B then updated from Table A but this update operation is taking lot of time. (Intermediate Commit, Bulk operations already tried)
    Regards,
    Dark Knight

    Environment -
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    Tables are analyzed.
    Indexes are their.
    Update statement using the index.
    Data is close to 200 MB.
    I am interested in knowing if there are alternate ways of doing this, other than conventional way of loading data then updating it.

  • Best way to load data from External data to ADC

    Hi Guy,
    I am new in BAM. I currently working for a existing system that have all it data in existing system, which i only have the view access to the database/table.
    so i just wonder, what is the best way to load the data so that it can populate to the active data cache?
    thanks.

    Hello,
    Use the EDS.For reference http://docs.oracle.com/cd/E14571_01/integration.1111/e10224/bam_extl_data_sources.htm
    Regards
    Siva Sankar

  • Looking for the best way to load data to the appended ODS

    Hi all,
    I appended ODS with some new infoobjects.
    if there is a way to load data only to these datafields without all data deleting and then all data reloading? 
    thnx

    Hi Andrius,
    If your ODS is set to be overwritten AND the newly appended IOs are not key fields then a full upload without data deletion will works. Because old entries will be overwritten by new ones which have values for the appended IOs.
    On the other hand, if your ODS additive then from my point of view you need to remove all data and then reload again. Or else key figures would be doubled after a full upload.
    Qing

  • Best option to save data populated from a web service?

    I have manage to made a windows form that uses a web service to populate the data into a DataGridView. Then I have a column for adding comments next to each row. So a user could add some comments about this data for each row. Also I have a drop down list
    to select some options for each row.
    Actually everytime when connecting to this web service there are 7 DataGridViews that are populated with data, one for each TabPage added to a TabControl. So 7 tabs then.
    If I close this form and reopen it then it's all gone.
    What is the simpliest way if I want to store the settings (comments and selected option in drop down list). The idea is that I want to store this as a project under a name like "Project1", "Project2" etc. So every project would
    have their own settings. Then when starting the application all over again, instead of populate it from the webservice, I then load it from wherever it's stored. So I can pick the particular project I want for example "Project1".
    So what is the easiest options here? Would it be best to store it in a database or could it be stored in different datasets (one for each project)? Also if I use a dataset every TabPage with it's own unique DataGrid would then be represented as DataTable
    I suppose. But then I don't know if the data in a dataset would still be there the next time - if it only stores temporarily?
    As for now my only intention is to build this for myself, so I don't need to have any end user in mind. 

    Hello,
    Here is one idea which may  be implemented in several ways.
    The data stored locally that is not available in the web service (let's look at the web service as read-only) could be stored in a xml file then read into your project via DataSet.ReadXml and DataSetWriteXml to save changes.
    For this to work you need to utilize primary keys to identify data coming from the service to match up with locally saved data.
    That is the concept which as stated above can be implemented in many ways. I would suggest using a class dedicated to the above and not place the bulk of the code into your forms.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem.

  • Best Practise for loading data into BW CSV vs XML ?

    Hi Everyone,
    I would like to get some of your thoughts on what file format would be best or most efficient to push data into BW. CSV or XML ?
    Also what are the advantages / Disadvantages?
    Appreciate your thoughts.

    XML is used only for small data fields - more like it is easier to do it by XML rather than build an application for the same - provided the usage is less.
    Flat files are used for HUGE data loads ( non SAP ) and definitely the choice of data formats would be flat files.
    Also XML files are transformed into a flat file type format with each tag referring to the field and the size of the XML file grows to a high value depending on the number of fields.
    Arun

  • Best Practices for Loading Data in 0SD_C03

    Hi, Guru, I want to know which is the best practice to have information about Sales, billing, delivery. I know it has this Datasources.
    • Sales Order Item Data - 2LIS_11_VAITM
    • Billing Document Data: Items - 2LIS_13_VDITM
    • Billing Document Header Data - 2LIS_13_VDHDR
    • Sales-Shipping: Allocation Item Data - 2LIS_11_V_ITM
    • Delivery Header Data - 2LIS_12_VCHDR
    • Delivery Item Data - 2LIS_12_VCITM
    • Sales Order Header Data - 2LIS_11_VAHDR
    Do I have to load all this Datasource to Infocube 0SD_C03 or I have to create copy of 0SD_C03 to mach with each Datasources.

    Hi.
        If you just want to statistic the amount or quantity of the sales process,I suppose you to create 3 cubes and then use a multi provider to integrated those 3 cubes you created.for example:
        2LIS_11_VAITM  -> ZSD_C01
        2LIS_12_VCITM -> ZSD_C02
        2LIS_13_VDITM -> ZSD_C03
    In this scenario,you can enhance the 2lis_12_vcitm and 2lis_13_vditm with sales order data,such as request delivery date etc..and then create a Multiprovider such as ZSD_M01.
    Best Regards
    Martin Xie

  • Options for loading data from legacy mainframe to SAP CRM and ECC

    Hello All,
    I am new to SAP.
    As part of data conversion planning from legacy mainframe to SAP CRM and ECC systems, I need to know the different tools available for loading the data and comparative analysis of the tools by showing the features. Please also describe the process to evaluate the tools (prototyping, technical discussions etc) and make the recommendations.
    I would also appreciate any information on testing tools like (eCATT) and any third party tools (like Informatica)
    I know I am asking for lot of things here. But, I really appreciate any information reagrding this.
    Thanks,
    Balu

    Hi
    Data Migration(Conversions) mainly involves the following Major Steps
    Discovery->Extract->Cleanse->Transform->Load
    Discovery-> Involves identifying various source systems available where the actual data is residing.
    Extract->Extract the data from the source systems(legacy systems)
    Cleanse->Cleanse the data by enriching it,De-duplications etc
    Transform->Transform the data into SAP Loadable format
    Load->Upload the data into SAP Using the Load Programs(Eg,BDC,LSMW,Data Transfer Workbench,BAPIs etc)
    Also i would request you to visit http://www.businessobjects.com/solutions/im/data_migration.asp
    Cheers,
    Hakim

  • Best options to recover Data from external?

    Hey guys, I have this person who has a 1tb external WD drive but the drive is not showing up on the desktop of her mac or mac pro, she can see the WD item in the DU. She also tried EaseUS Data Recovery Wizard Trial but her hard drive isn't recognized so she can't select it to run a scan. Since the HDD was not formatted for windows I can take the HDD itself out of it's enclosure to add as a slave on my windows pc to see if I can access it that way, what options would we have here please?

    If all the photos  etc. are on your iPhone, you can back them up after you get your SSD going. If there is more in the backup on your failing HD, you can try copying it manually.
    TimLitch wrote:
    I assume just syncing with the new SSD installed will wipe the phone..?
    Not necessarily. In iTunes Preferences>Devices you can check the box "Prevent iPod, iPhone, iPad etc. from syncing automatically."
    Then you can back up iPhone to your computer or iCloud.
    In Internet Recovery you are given options -- to erase the hd or ssd and install Mavericks, or restore from Time Machine.
    Before you do it you should make a note of your Superduper registration number in case you have to download it again.

  • What's the best way to load FieldPoint measurement data into PI System?

    I am finding the best way to load data collected by NI Field Point (FP2220) into the PI system of our power plant.
    I found pieces of information about FieldPoint OPC server in NI.com. Not sure if it comes with Field Point Hardware, sold by NI as a separate product or it is actually non-standard NI products. Anyway, I know that there exists a thing called FieldPoint OPC server.
    The PI system I mentioned has a OPC client software called PI-OPC interface. It is able to communicate with standard OPC DA server. If that FieldPoint OPC server is a standard OPC DA server provide data collected by Field Point complying to OPC standard, than that's perfect.
    Anyone familar with PI system and NI product, please help if the above is going to work or if there is a better way to put Fieldpoint data into PI.

    Hi Eric,
    This information really helpful, thanks. Regarding to the NI OPC server for NI FieldPoint, I have the other query.
    In my setup, there are two sets of FieldPoint located in two different locations on my ethernet network. They are going to be controlled by a single PC. If I am going to connect both my FieldPoint sets with OPC standard, how many NI OPC server for FieldPoint do I need to connect to? Are there two NI OPC servers each serves one FieldPoint set? Or there is only NI OPC server which serves both FieldPoint sets?
    I am concerning about the number of NI OPC server instances running, because the number of OPC client license I need to purchase depends on how many OPC server I need to connect to. If one NI OPC server serves both my FieldPoint sets, I only need to buy one OPC client license; otherwise, I need to purchase two. In the future, I am going to have another two sets of FieldPoint sets, so the answer of my query determines how many OPC clients I need to purchase eventually - One or four. A huge price difference.
    Looking forward to your reply.
    Regards,
    Roger

  • Load data in R/3 BW delta Queue

    Hi,
    Is there any option to load data in R/3 BW delta Queue. I have done nothing for that. I have found that new record is available in source table. Is there any option to schedule / execute in R/3 end.
    Thanks & regards,
    Goutam

    Hi Goutam,
    What do you mean ...Is it R3 Delta Queue or BW delta Queue???
    As far as I consider it is R3 Delta Queue....For loading the data to R3 delta queue...1st let me know which update method you are using...
    If its Direct update....document posting will be directly written to Delta Queue---RSA7.
    If its V3 unserialised...then the changed document posting wil be captured and will be avaialble in Extraction Queue LBWQ and from there a collective run i.e. control jobs need to be scheduled as per your desired frequency to move it from LBWQ to Delta Queue RSA7.
    If it is queued delta
    u2022 Extraction data is collected for the affected application in an extraction queue
    u2022 Collective run as usual for transferring data into the BW delta queue
    Hope it is clear...
    Let me know in case you need further details...
    Thanks,
    Amit Kr.

  • Loading Data in  to a cube

    Hi,
      I am Loading data from one cube to another cube .The data present in large cube is Larger. So i am loading by Each month.Whether i need to Delete the Indexes after each Load and i want to know what is impact of that.
    Regards
    Arunkumar

    SAP best practice while loading data to IC is to drop the index before the load and create the index after completion of the load.
    Yes you need to delete and create the index for each load.
    If you do not delete the index while loading data to infocube, records that are loaded newly into the table and they need to search for the latest index to which they need to fit in.
    If you drop the index and create the index it will reduce the time for search of index which is latest available for the recent loaded data.
    by doing this loading performance will improve.
    Edited by: prashanthk on Nov 26, 2010 11:25 AM

  • Using Import Man to load Data into Multi Value Fileds in a Qualified Table

    Hi there,
    When using the Import Manager, i can not use the "append" option to load data into my multi value field which is contained within my qualified table.
    Manually it works fine on Data manager, so the field has been set up correctly. Only problem is appending the data during Import Manager Load.
    Any reason why I do not have this option available during Field mapping in Import Manager. The selection options are shown but in gray.
    Would appreciate any sugestions.
    Chris Huggett

    Thanks Sowseel
    Its a good document but doesn't address my problem, maybe My problem isn't clear.
    The structure(part of) that I have currently is as follows.
    Main Table - Material
                           QFTable-  MNF PN
                               LUField - MNF Name(Qualifier Single Value)
                               LUField  - BU ID  (Non Qualifier Multi Value)
                               TField   - P/N- (Non Qualifier)
    I know how to load data to the main and qualified tables, but what I can not do, using Import Manger, is updating the  "LUField  - BU ID  (Non Qualifier Multi Value)" using the append functionality.
    Thanks
    Chris Huggett

  • Load data from File on Client side (via Sqlplus)

    Server OS: RedHat, Oracle 10g rel 2.
    I am trying to load data from OS .txt files to clob field.
    I am able to do this successfully using:
    Oracle DIRECTORY
    BFILE
    DBMS_LOB.loadclobfromfile packageIssue is: this only works if my files and DIR are on database server.
    Is it not possible "load clob from file" from client side, files being on client and exec command via SQLPlus. Is there any other option to load data from client side.
    Thanks for any help.

    Options:
    1) Search for OraDAV
    2) Learn about Application Express.

Maybe you are looking for