Can we load data in chunks using data pump ?

We are loading data using data pump. So I want to clear my understanding.
Please correct me if I am wrong on my understandings -
ODI will fetch all data from source (whether it is INIT or CDC ) in one go and unload into staging area.
If it is true, will performance hamper in case very huge data (50 million records at source) at source as ODI tries to load entire data in one go. I believe it will give better performance if we load in chunks using data pump.
Please confirm and correct.
Also I would like to know how can we configure chunk load using data-pump.
Thanks in Advance.
Regards,
Dinesh.

You may consider usingLKM Oracle to Oracle (datapump)
http://docs.oracle.com/cd/E28280_01/integrate.1111/e12644/oracle_db.htm#r15c1-t2
In 11g ODI reads from source and write to target in parallel. This is the case where you specify select query in source command and insert/update query in the target command. At source side Odi reads records from source and add them to a data queue. At target side a parallel thread reads data from the data queue and writes to the target. So the overall performance would be the slower of the read or write process.
Thanks,

Similar Messages

  • Retrieve LDAP data in Chunks using DBMS_LDAP

    Need help in getting the LDAP data in Chunks using the DBMS_LDAP. The reason being i have over 8000 records in the AD and it is configured that i cannot retrieve more than 1000 at a stretch.
    The AD administrator had given me an option of fetching it based on the pagesize which they use on AD.
    Dim DirSearcher As New DirectorySearcher()
    DirSearcher.SearchRoot = New DirectoryEntry("LDAP://" & System.Environment.UserDomainName)
    DirSearcher.Filter = "(&(objectclass=user)(objectcategory=person))"
    DirSearcher.PageSize = 1000
    Can we have something like this done in the DBMS_LDAP package to fetch the records in AD in chunks of 1000 records?
    version details
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Prod
    PL/SQL Release 10.2.0.4.0 - Production

    if there are more than 1000 entries which starts with a character, how will this solution work?You would have to incorporate one more loop (and so on, in case it still errors out):
      for c in ascii ('a') .. ascii ('z')
      loop
        for d in ascii ('a') .. ascii ('z')
        loop
          l_retval :=
            dbms_ldap.search_s (ld         => ld
                                base       => base,
                                scope      => dbms_ldap.scope_subtree,
                                filter     => '(&(objectCategory=person)(objectClass=user)(sAMAccountName=' || chr (c) || chr(d) || '*))',
                                attrs      => attrs,
                                attronly   => 0,
                                res        => res
        end loop; 
      end loop; 
    ...Not nice I know, but it seems to be the only way to avoid »ORA-31202: DBMS_LDAP: LDAP client/server error: Sizelimit exceeded«.

  • Since loading the latest version of firefox i can no longer print. instead a message comes up "can not load library 'DLBAPRP.DLL' using defaults, error 126. what happened?

    error 126 can not load library "DLBAPRP.DLL" using defaults. this message appears when we try to print from our web bookmarks or any acrobat or pdf files

    That error seems to be new with Firefox 4 - never heard of it with earlier versions of Firefox, but it goes back many years in various Windows programs.
    See this [http://www.google.com/search?client=ubuntu&channel=fs&q=DLBAPRP.Dll&ie=utf-8&oe=utf-8 search].

  • Can't load attachments on aol using safari

    Can't load attachments on aol using safari

    I don't have aol, but I have discovered that some attachment protocols depend on java, and the iPad doesn't play well with java. For example I've discovered that I cannot do certain things with my work's web mail since it's java based and my iPad simply will not handle it.
    You could see if AOL has a mobile app/site that you can access that might not depend on java. Some sites have 'dumbed down' versions of their sites to cater to mobile devices.

  • Since downloading new version, Printer issue "Can't load library DLBAPRP.Dll, Using Defaults Error:126"

    The printer will not work on some printjobs and I get the message shown in my question. Previous to downloading the new version of Firefox, no problem. And I can print these jobs when I use my internet explorer browser.

    That error seems to be new with Firefox 4 - never heard of it with earlier versions of Firefox, but it goes back many years in various Windows programs.
    See this [http://www.google.com/search?client=ubuntu&channel=fs&q=DLBAPRP.Dll&ie=utf-8&oe=utf-8 search].

  • I have OS X 10.5.8.  Can I load Snow Leopard without losing data and then upgrade to Lion?

    I am using OS X 10.5.8.  I have a disk with Snow Leopard on it, but I don't know if I downloaded it.  Can I download this Snow Leopard disk without risking a loss of data on my MacBook Pro?

    If by "download" you mean "install," then you can install Snow Leopard over a Leopard system. But anything can happen, so it would be wise to backup first.
    1. Repair the Hard Drive and Permissions
    Boot from the Leopard Installer disc. After the installer loads select your language and click on the Continue button. When the menu bar appears select Disk Utility from the Utilities menu. After DU loads select your hard drive entry (mfgr.'s ID and drive size) from the the left side list.  In the DU status area you will see an entry for the S.M.A.R.T. status of the hard drive.  If it does not say "Verified" then the hard drive is failing or failed. (SMART status is not reported on external Firewire or USB drives.) If the drive is "Verified" then select your OS X volume from the list on the left (sub-entry below the drive entry,) click on the First Aid tab, then click on the Repair Disk button. If DU reports any errors that have been fixed, then re-run Repair Disk until no errors are reported. If no errors are reported click on the Repair Permissions button. Wait until the operation completes, then quit DU and return to the installer.
    If DU reports errors it cannot fix, then you will need Disk Warrior and/or Tech Tool Pro to repair the drive. If you don't have either of them or if neither of them can fix the drive, then you will need to reformat the drive and reinstall OS X.
    2. Install Snow Leopard
    If the drive is OK then quit DU and return to the installer.  Boot from the Snow Leopard installer DVD.  Follow the onscreen instructions. Note that the Snow Leopard installer will not erase your drive or disturb your files.  After installing a fresh copy of OS X the installer will move your Home folder, third-party applications, support items, and network preferences into the newly installed system.
    Download and install the Mac OS X 10.6.8 Update Combo v1.1.

  • How can I tell which apps are using data

    Perhaps this is a suggestion, just in case someone who is in a position to note this at Apple (not likely?) might read it.
    How about an app included in the OS that allows users to track which apps are transmitting and receiving data through cellular data and/or wireless.
    I keep getting charged by AT&T (it took restraint to not type a bunch of explatives before AT&T) for escessive data usage, even though I'm in wifi 90%+ of each day, and purposely restrict using it when not in wifi. Seriously, I don't now how I could use it less barring not using it at all, and I still get charged for data. I have no idea how to control this since I cannot tell which, if any, apps are transmitting data in the background, or if that's even possible.
    Seriously annoyed at getting charged extra when the 'service' they provide barely works for me the miniscule amount of times I try to use it.

    This is a user to user forum.
    You can leave feedback here
    www.apple.com/feedback/
    You could also try searching for an app that tracks data usage

  • Can you set a default value using Data Mover Scripts?

    Hi,
    We're going through a upgrade of PS Fin 8.9 -> 9.1. We've found some existing tables with new columns that are marked as NOT NULL so we're having trouble migrating the data using DMS.
    Is there a way to Export the data from 8.9 and when importing into 9.1, set a default value for the new column using DMS?
    A specific example would be the PS_SOURCE_TBL where EXCHANGE_RATE_OPTN is new and NOT NULL.
    Thanks for any assistance.

    What you could do is first rename the record to be imported
    IMPORT SOURCE_TBL AS PS_SOURCE_TBL_ORG;
    Then write a insert/select script from PS_SOURCE_TBL_ORG to PS_SOURCE_TBL defaulting column EXCHANGE_RATE_OPTN
    Insert into PS_SOURCE_TBL (COLUMN,COLUMN,COLUMN,COLUMN, EXCHANGE_RATE_OPTN)
    select PS_SOURCE_TBL_ORG (COLUMN,COLUMN,COLUMN,COLUMN, 'defaultvalue');
    And then drop table PS_SOURCE_TBL_ORG to clean up your database.

  • Are there any  best practice templates available to load Customer Master data into ECC using data services?

    Hi,
    As far as I remember there are best practices templates (AIO). I am not able to find the location to download these atl files though.
    Thanks,
    Pramod

    Hi pramod did you reffered this document:
    http://help.sap.com/businessobject/product_guides/sboDS41/en/sbo41_ds_sap_en.pdf
    http://events.asug.com/2011AC/4103_Legacy_Data_Migration_to_SAP_ECC.pdf

  • How can I load and save text using FileReference class

    Let me start by saying I have no intention of using php and I can't wait until flash cs5 comes out with its new read/write capabilities
    I want to load and save XML files (stored on my local hard-drive) in AS3.
    I am using the FileReference class.
    Firstly I have noticed that when using FileReference.save if you replace an existing file instead of writing over the file data is appended to the end of the file. Can I make it so the file is overwritten (as it should be) or make it impossible for the user to save in such a situation.
    Secondly I want to load in text from an external file using FileReference.load I know that somehow you use FileReference.browse first to get it to work but I want to know exactly how to do it.
    I have looked for a tutorial about loading text in this manner and have not found one.
    Any help would be much appreciated especially if you could point me in the direction of a relevant tutorial
    Thanks

    the filereference class is for downloading and uploading files.
    if you want to load xml, use the xml class.
    and, if you want to write to an xml file and don't want to use server-side code, wait.

  • MDM Data Manager Assignments using Dates

    Hello.
    I am trying to create some assignments that default to a certain date and my expressions are getting the error message when I execute them.
    I am currently trying to create an Assignment for the Look-up table: Valid To (which belongs to the QT: Sales Data).
    I want the Valid To Assignment to default at the date 12/31/2999 if the user did not already enter a value.
    My expression looks like this:
    IF (IS_NULL(Sales Data.Valid To), SYSTIME(0) = 12/31/2999)
    I linked Sales Data for my 'Table Field'
    Any Ideas?
    Points awarded promptly for any help!
    Thanks,
    Nichole

    Hi,
    As per my knowledge, writing Assignments on Qualified Table field within the Main table is not possible for both Qualifier as well as Non-Qualifier Field. As if you want to write Assignment on Non-Qualifier fields only then you can do while selecting Qualified table as your Current Table not as Main table.
    For Qualifier field too if you want to asign field using Main table is not Possible as you cant select Property Table Field as Valid to here you could only able to select all Main table fields and fields which is look up to this Qualified table and other Sub tables.
    Don't confuse with other subtables.
    Other assignments that i've made work with just the 'sales data' table field even though there is a lookup table within that Q.T.
    That's why as you could get values for all other subtables in Assignment Expression as lookups tab if you remember like Reconcillation but here you could not find field who look up to your Qualified Table as well as Qualified table. there indirectly u r refering field to this lookup table and you get values here to assign but how could you assign value to Valid to field just think even if this is must requirement for you then you create one more subtable and look up this valid to field to this subtable or just create one more field  Valid to in maintable of type text.
    So from my point of view this is only drawback that we could not Assign Value to Qualified table field. I hope it will help you in understanding. But writing Validations for Qualified Table fields is Possible but not Assignments. May be in MDM 7.1 its possible. This is my knowledge about Writing Expressions for Qualified Table Fields till now. 
    Hope it will Help you,
    Rewards if found Useful.....
    Thanks,
    Mandeep Saini

  • How can I load a picture and use it in ActiveX in run time ??

    I am using an ActiveX object which needs a property "picture" as input.
    I can set the picture property in its property page in design time, but I don't know how to give it a "picture" in run time.
    In VB, I can do:
    dim pic As Picture
    pic = loadpicture("c:\test.bmp")
    control_1.picture = pic
    (("control_1" is the ActiveX object, and picture is the property))
    How can I do the same with labview?
    Is there any example that sets the picture property of an ActiveX in run time?

    sealiker wrote:
    I am using an ActiveX object which needs a property "picture" as input. I can set the picture property in its property page in design time, but I don't know how to give it a "picture" in run time. In VB, I can do:   dim pic As Picture pic = loadpicture("c:\test.bmp") control_1.picture = pic (("control_1" is the ActiveX object, and picture is the property))   How can I do the same with labview? Is there any example that sets the picture property of an ActiveX in run time?
    As
    far as I understand from the MSDN site, it seems that the Picture
    datatype is a specific VB only datatype. Most probably it is a handle
    (uInt32) value as far as other programs including LabVIEW are
    concerned, but there is no way that LabVIEW could deal with this
    datatype directly.
    If you can convert this datatype with some VB fucntions into a HBITMAP
    or HDIB then you could call some Win32 API functions to extract the
    relevant bitmap data from this common Win32 datatype and get somewhere
    from there. OR it maybe that VB itself has other functions to extract
    the bitmap data, color palette and other attributes of the Picture
    datatype. Don't expect LabVIEW to be able to deal with the Picture
    datatype directly though, this is really a proprietary VB thing.
    Rolf Kalbermatter
    Rolf Kalbermatter
    CIT Engineering Netherlands
    a division of Test & Measurement Solutions

  • No garbage collection data/detailed memory use data in Wily?

    Hi All,
    We are running some portals on AIX (1.4.2 SR12). We have installed the wily agent on the portals and the enterprise manager on our solman server. When looking in the introscope workstation the heap usage of the JVMs are shown as it should. However, the J2EE GC Overview 1/2 & 2/2 don't show any detailed data regarding garbage collection and memory usage (e.g. tenured; eden; etc...). At this moment we use IBM PMAT for memory analysis, but this is not real time. We would like to see memory details real time using Wily.
    Does anyone else also encounters this problem?
    Any suggestions?
    Thanks,
    Bart.

    Hello Amit
    Can you please open new thread for your question ?
    People won't check this thread to answer your question because the thread is marked "answered".
    Kind regards
    Tom

  • What is the speed of data transmission when using Data Dashboard?

    Hello, everybody, I've seen that doesn't matter if an iphone or a chinese tablet is used, the speed  of data transmission with Shared Variables and Data Dashboard is almost the same, and is low speed. Does anybody know how could I measure this speed of transmission? Thank you in advance.
    Impossible is nothing
    Solved!
    Go to Solution.

    Hello there;
    The update rate of the Data Dashboard for 1.0 is fixed  1 second for webservices and 0.2 seconds for network shared variables, depending of course on how much information is streaming on the network . On the second version of Data DAshboard you can specify your update rate within a range
    You can find that info in this KB
    Hope this info helps.
    Cheers

  • Jdev 11g, Passing data between pages using data control

    Hi,
    I'm fairly new to Jdev, J2EE, and ADF. I created a project using the tutorial "Build a Web Application with JDeveloper 11g Using EJB, JPA, and JavaServer Faces" (found here: http://www.oracle.com/technology/obe/obe11jdev/11/ejb/ejb.html).
    Data model part seems to work fine. I am able to connect to the database and print out a record using the sample client.
    Now I'm trying to build the web front end for it. I started using a data control and ADF to make the components. I built a search page by dragging and dropping my method from the data control onto a form. I set the action on the form to go to a detail page which I put a database node from my table product_group. My thinking was the user would put in the id and then be taken to the next page that has the detail on it.
    When I ran the page though the search page seemed to execute fine but the detail page came up blank. There were no runtime errors. I think data is not being passed to the next page.
    This is the code for the form on the search page:
    <af:form>
    <af:panelFormLayout>
    <af:inputText value="#{bindings.p_id.inputValue}"
    label="#{bindings.p_id.hints.label}"
    required="#{bindings.p_id.hints.mandatory}"
    columns="#{bindings.p_id.hints.displayWidth}"
    maximumLength="#{bindings.p_id.hints.precision}"
    shortDesc="#{bindings.p_id.hints.tooltip}">
    <f:validator binding="#{bindings.p_id.validator}"/>
    <af:convertNumber type="number"/>
    </af:inputText>
    <af:commandButton actionListener="#{bindings.queryProdGroupFindByID.execute}"
    text="Search ID"
    disabled="#{!bindings.queryProdGrouptestFindByID.enabled}"
    action="search"/>
    </af:panelFormLayout>
    </af:form>
    This is the detail page:
    <af:form>
    <af:panelFormLayout>
    <af:panelLabelAndMessage label="#{bindings.description.hints.label}">
    <af:outputText value="#{bindings.description.inputValue}"/>
    </af:panelLabelAndMessage>
    <af:panelLabelAndMessage label="#{bindings.id.hints.label}">
    <af:outputText value="#{bindings.id.inputValue}">
    <af:convertNumber groupingUsed="false"
    pattern="#{bindings.id.format}"/>
    </af:outputText>
    </af:panelLabelAndMessage>
    <af:panelLabelAndMessage label="#{bindings.status.hints.label}">
    <af:outputText value="#{bindings.status.inputValue}"/>
    </af:panelLabelAndMessage>
    <af:panelLabelAndMessage label="#{bindings.storeId.hints.label}">
    <af:outputText value="#{bindings.storeId.inputValue}"/>
    </af:panelLabelAndMessage>
    <af:panelLabelAndMessage label="#{bindings.type.hints.label}">
    <af:outputText value="#{bindings.type.inputValue}">
    <af:convertNumber groupingUsed="false"
    pattern="#{bindings.type.format}"/>
    </af:outputText>
    </af:panelLabelAndMessage>
    </af:panelFormLayout>
    </af:form>
    I guess what I'm asking is how does data get passed to the detail page using a data control? Also if anyone has an online book about data controls and how they work I'd appreciate that too. Too much magic going on.
    Thanks in advance.
    Edited by: user2936931 on May 3, 2009 5:15 PM

    I went to the bindings tab for my search.jspx page and found in the 'executables' table queryProdGroupFindByIDIterator. I looked in the properties for that and 'CacheResults' is set to '<default>true'. Next page still looks blank. I changed it to straight up 'true' and that didn't work either.
    If I'm in the right place I guess it must be something else?

Maybe you are looking for