Calling Delta Merge in DS after every commit

Hi Folks,
I am using an Delta extraction logic in DS to extract large table from ECC (50 Million rows) to the HANA database. The commits in DS job have been configured fopr every 10,000 records. Three questions
1) Should I disable the delta merge in HANA database for this target table prior to the initial load of table. Once the initial load is complete, manually perform the delta merge in HANA is the right approach or
2) Should I be calling manually performing Delta merge in DS job to make sure the table is merged after every commit? If yes how do I call the Delta merge command in DS jobs and how can I do it per commit?
3) Can I invoke Delta merge in DS as part of Delta extraction logic after the initial load is completed in DS?
Any advise will definately be appreciated.
Thanks,
-Hari

Hi Jim
if your big table requires a merge, AUTOMERGE will pick it up. The mergedog process checks it every 60 seconds, so that should be alright for your requiremen.
If the table doesn't need to be merged, it won't.
Manually handling the delta merge is a fine-tuning action that is most often not required or recommendable.
- Lars

Similar Messages

  • HANA and DataServices - delta merge during big load

    Hi Gurus
    I'll do load oracle table to HANA via DataServices. Main event is big 1 table.
    At that time , I want to execute delta merge during load , like every 100000 records/every 3 mins etc.
    I'm understand about perfoemance risk but ..... I have many times memory shortage problem which caused by forget execute of delta merge.
    But It looks like no setting content about delta merge in Dataservices .... forcely , execute sql after load ?
    I don't want to use subdivide big table and load it and execute merge table after every load method.
    It's just my idea , execute 'MERGE DELTA OF BIGTABLE' every 3 minutes by cron or timer during load , Is it good method?
    Rgds,
    Jim
    Message was edited by: Tom Flanagan

    Hi Jim
    if your big table requires a merge, AUTOMERGE will pick it up. The mergedog process checks it every 60 seconds, so that should be alright for your requiremen.
    If the table doesn't need to be merged, it won't.
    Manually handling the delta merge is a fine-tuning action that is most often not required or recommendable.
    - Lars

  • Commit in procedures after every 100000 records possible?

    Hi All,
    I am using an ODI procedure to insert data into a table.
    I checked that in the ODI procedure there is an option of selecting transaction and setting the commit option as 'Commit after every 1000 records'.
    Since the record count to be inserted is 38489152, I would like to know is this option configurable.
    Can i ensure that commits are made at a logical step of 100000 instead of 1000 records?
    Thank You.
    Prerna

    recently added on this
    http://dwteam.in/commit-interval-in-odi/
    Thanks
    Bhabani
    http://dwteam.in

  • Avoid Commit after every Insert that requires a SELECT

    Hi everybody,
    Here is the problem:
    I have a table of generator alarms which is populated daily. On daily basis there are approximately 50,000 rows to be inserted in it.
    Currently i have one month's data in it ... Approximately 900,000 rows.
    here goes the main problem.
    before each insert command, whole table is checked if the record does not exist already. Two columns "SiteName" and "OccuranceDate" are checked... this means, these two columns are making a unique record when checked together with an AND operation in WHERE clause.
    we have also implemented partition on this table. and it is basically partitioned on the basis of OccuranceDate and each partition has 5 days' data.
    say
    01-Jun to 06 Jun
    07-Jun to 11 Jun
    12-Jun to 16 Jun
    and so on
    26-Jun to 30 Jun
    NOW:
    we have a commit command within the insertion loop, and the each row is committed once inserted, making approximately 50,000 commits daily.
    Question:
    Can we commit data after say each 500 inserted rows, but my real question is can we Query the records using SELECT which are Just Inserted but not yet committed ?
    a friend told me that, you can query the records which are inserted in the same connection session but not yet committed.
    Can any one help ?
    Sorry for the long question but it was to make u understand the real issue. :(
    Khalid Mehmood Awan
    khalidmehmoodawan @ gmail.com
    Edited by: user5394434 on Jun 30, 2009 11:28 PM

    Don't worry about it - I just said that because the experts over there will help you much better. If you post your code details there they will give suggestions on optimizing it.
    Doing a SELECT between every INSERT doesn't seem very natural to me, but it all depends on the details of your code.
    Also, not committing on time may cause loss of the uncommitted changes. Depending on how critical the data is and the dependency of the changes, you have to commit after every INSERT, in between, or at the end.
    Regards,
    K.

  • Commit after every 1000 records

    Hi dears ,
    i have to update or insert arround 1 lakhs records every day incremental basis,
    while doing it , after completing all the records commit happens, In case some problem in between all my processed records are getting rollbacked,
    I need to commit it after every frequency of records say 1000 records.
    Any one know how to do it??
    Thanks in advance
    Regards
    Raja

    Raja,
    There is an option in the configuration of a mapping in which you can set the Commit Frequency. The Commit Frequency only applies to non-bulk mode mappings. Bulk mode mappings commit according the bulk size (which is also an configuration setting of the mapping).
    When you set the Default Operating Mode to row based and Bulk Processing Code to false, Warehouse Builder uses the Commit Frequency parameter when executing the package. Warehouse Builder commits data to the database after processing the number of rows specified in this parameter.
    If you set Bulk Processing Code to true, set the Commit Frequency equal to the Bulk Size. If the two values are different, Bulk Size overrides the commit frequency and Warehouse Builder implicitly performs a commit for every bulk size.
    Regards,
    Ilona

  • COMMIT after every 10000 rows

    I'm getting probelms with the following procedure. Is there any that I can do to commit after every 10,000 rows of deletion? Or is there any other alternative! The DBAs are not willing to increase the undo tablespace value!
    create or replace procedure delete_rows(v_days number)
    is
    l_sql_stmt varchar2(32767) := 'DELETE TABLE_NAME WHERE ROWID IN (SELECT ROWID FROM TABLE_NAME W
    where_cond VARCHAR2(32767);
    begin
       where_cond := 'DATE_THRESHOLD < (sysdate - '|| v_days ||' )) ';
       l_sql_stmt := l_sql_stmt ||where_cond;
       IF v_days IS NOT NULL THEN
           EXECUTE IMMEDIATE l_sql_stmt;
       END IF;
    end;I think I can use cursors and for every 10,000 %ROWCOUNT, I can commit, but even before posting the thread, I feel i will get bounces! ;-)
    Please help me out in this!
    Cheers
    Sarma!

    Hello
    In the event that you can't persuede the DBA to configure the database properly, Why not just use rownum?
    SQL> CREATE TABLE dt_test_delete AS SELECT object_id, object_name, last_ddl_time FROM dba_objects;
    Table created.
    SQL>
    SQL> select count(*) from dt_test_delete WHERE last_ddl_time < SYSDATE - 100;
      COUNT(*)
         35726
    SQL>
    SQL> DECLARE
      2
      3     ln_DelSize                      NUMBER := 10000;
      4     ln_DelCount                     NUMBER;
      5
      6  BEGIN
      7
      8     LOOP
      9
    10             DELETE
    11             FROM
    12                     dt_test_delete
    13             WHERE
    14                     last_ddl_time < SYSDATE - 100
    15             AND
    16                     rownum <= ln_DelSize;
    17
    18             ln_DelCount := SQL%ROWCOUNT;
    19
    20             dbms_output.put_line(ln_DelCount);
    21
    22             EXIT WHEN ln_DelCount = 0;
    23
    24             COMMIT;
    25
    26     END LOOP;
    27
    28  END;
    29  /
    10000
    10000
    10000
    5726
    0
    PL/SQL procedure successfully completed.
    SQL>HTH
    David
    Message was edited by:
    david_tyler

  • Need to commit after every 10 000 records inserted ?

    What would be the best way to Commit after every 10 000 records inserted from one table to the other using the following script :
    DECLARE
    l_max_repa_id x_received_p.repa_id%TYPE;
    l_max_rept_id x_received_p_trans.rept_id%TYPE;
    BEGIN
    SELECT MAX (repa_id)
    INTO l_max_repa_id
    FROM x_received_p
    WHERE repa_modifieddate <= ADD_MONTHS (SYSDATE, -6);
    SELECT MAX (rept_id)
    INTO l_max_rept_id
    FROM x_received_p_trans
    WHERE rept_repa_id = l_max_repa_id;
    INSERT INTO x_p_requests_arch
    SELECT *
    FROM x_p_requests
    WHERE pare_repa_id <= l_max_rept_id;
    DELETE FROM x__requests
    WHERE pare_repa_id <= l_max_rept_id;

    1006377 wrote:
    we are moving between 5 and 10 million records from the one table to the other table and it takes forever.
    Please could you provide me with a script just to commit after every x amount of records ? :)I concur with the other responses.
    Committing every N records will slow down the process, not speed it up.
    The fastest way to move your data (and 10 million rows is nothing, we do those sorts of volumes frequently ourselves) is to use a single SQL statement to do an INSERT ... SELECT ... statement (or a CREATE TABLE ... AS SELECT ... statement as appropriate).
    If those SQL statements are running slowly then you need to look at what's causing the performance issue of the SELECT statement, and tackle that issue, which may be a case of simply getting the database statistics up to date, or applying a new index to a table etc. or re-writing the select statement to tackle the query in a different way.
    So, deal with the cause of the performance issue, don't try and fudge your way around it, which will only create further problems.

  • Dictation capitalizes every word after a comma, no matter what.

    Dictation capitalizes every word after a comma, no matter what.  For instance:  "I am a man, But not an old man."  Any setting to change that could stop this?

    Please read this whole message before doing anything.
    This procedure is a diagnostic test. It won’t solve your problem. Don’t be disappointed when you find that nothing has changed after you complete it.
    The purpose of this test is to determine whether the problem is localized to your user account. Enable guest logins* and log in as Guest. Don't use the Safari-only “Guest User” login created by “Find My Mac.”
    While logged in as Guest, you won’t have access to any of your documents or settings. Applications will behave as if you were running them for the first time. Don’t be alarmed by this behavior; it’s normal. If you need any passwords or other personal data in order to complete the test, memorize, print, or write them down before you begin.
    Test while logged in as Guest. Same problem?
    After testing, log out of the guest account and, in your own account, disable it if you wish. Any files you created in the guest account will be deleted automatically when you log out of it.
    *Note: If you’ve activated “Find My Mac” or FileVault in OS X 10.7 or later, then you can’t enable the Guest account. The "Guest User" login created by "Find My Mac" is not the same. Create a new account in which to test, and delete it, including its home folder, after testing.

  • IPhone 3G freezes after every call after 4.2 update

    There isn't much more to say other than that. As soon as I installed the 4.2 update my phone now freezes after every call. It stays frozen for hours, right now it's been frozen over 12 hours (screen is even still lit up). I'm tired of completely restoring my phone after every phone call!!!

    its nt a  casual evente in many iphone  3G with OS 4.2
    really apple is the responsable.
    we are honest users, and   we  need a  real answer.
    No reset the phone many tiemes, or  change the  phone.
    The  problem is  sorftware and internal politics in apple.
    no more  dirty play ...
    other  companies lets that a user continue  with  her  device, by 5 or more  years.
    an spencive  device  need to be by more time.
    why  apple  continue selling Iphone 3G? ...

  • Is it possible to automatically add a comma after every 6 characters has been typed in a text field?

    Hope someone can help me.
    What I basically need help with is how to make Acrobat add a comma after every 6 characters in a text field:
    XXXXXX,YYYYYY, ZZZZZZ etc

    I'm sorry, but I did not understand that (i'm using Acrobat Pro X)
    Am I supposed to go to:
    Text Field Properties > Format > Custom
    and then use Custom Format Script or Custom Keystroke Script?
    I tried both and it did not work.
    And do the Text Field have to be named "chunkSize"?
    Seems like it works. I had to move to the next formfield in order to see the effect.
    Is it possible to make it happen in real time (as you type the comma is inserted?)

  • Phone crashes after every call

    My Droid crashes after every call.  I have to remove the battery and reboot the phone.  (Am running Droid 2.1).
    Verizon - this isn't even funny any more.  What's up with this?

    I haven't seen this before and it sounds really strange.
    Have you tried restoring you phone to "factory defaults". It is quite possible something that you have installed is conflicting somewhere, but it's really hard to guess what it might be.
    Also, while some of the VZW technical people do hang out here and answer questions, you're probably better off contacting VZW Customer Service with this one. You can reach them by dialing 611 on your mobile (which, in this case I wouldn't recommend) or at (800) 922-0204.
    jason

  • Nokia E6 - Signal Drop after every call

    Hi,
    I have started to face this problem for the last 8-10 days.
    After every call is completed there is a signal drop for about 30-40 seconds. It drops to zero & then comes back to full strength again.
    I have updated the phone software to the latest version and still have the same problem.
    There is no problem if I use my sim card in any other phone.
    Thanks for your solutions in advance.
    Cheers
    KD
    Solved!
    Go to Solution.

    Hey,
    Thanks a ton...working for me so far for the last 3 calls...
    If only you had replied earlier ...it would have saved me from going to the Nokia Service Center, who told me to change my SIM card...which I did as well with no solution. LOL
    I wonder where the nokia moderators are ..not even one reply for almost 3 days now..
    anyways ..thanks a ton buddy
    Cheers
    KD

  • After every call...

    Did anyone notice that after every call ended and when you unlock the phone, it brings up the recent call list automatically instead of the home screen? Am i crazy?
    Message was edited by: actuarius

    I am VERY sure now of my problem. After EVERY incoming call, iPhone brings up the Phone menu upon unlocking. It's very annoying and I dont know how to solve this. Doesnt anyone else have this problem?

  • Call TableCellRenderer after every setValueAt().

    I've made a TableCellRenderer which colours some cells depending on some values. My only problem is, that it only colours the cells after a addRow() or similar and not after every setValueAt().
    I want my table to colour the cells after every change in the table.
    How is this possible?

    Since I don't know how you have implemented your Table Model I can't be sure of the detail but if you have extended DefaultTableModel or AbstractTableModel then in method setvalueAt() you call fireTableCellUpdated() with whatever row and col you are updating. Something like
    public void setValueAt(int row, int col)
        // do your stuff to update the value
        fireTableCellUpdated(row, col);)
    }

  • I am trying to generate an array of 30 random numbers. after every 5 readings a new vi should open and tell the user that 5 readings has been completed. and again carry on with the generation of array.

    since i do not have a transducer now, i am currently generating an array of 30 random numbers. after every 5 readings a warning should be given to the user that 5 readngs are complete. this cycle should repeat. the array size is 30.
    please help me out,  waiting for reply asap.
    once i have the transducer, i will be taking 30 analog samples and then after every 5 smaples that wraning will be displaye din a new VI
    Solved!
    Go to Solution.

    Use a while loop with a time delay representing your sampling period.
    Use the count terminal to check if equals 4, so 4th iteration=5th sample.
    Use a case structure. The true case will only be executed on the 4th iteration.
    In the true case place a subVI  with your desired message in the front panel. Go to the VI properties window and set "open front panel when called".
    The closing condition of the warnign is not giving in your description.
    Consider that rather than usign a subvi for this, you could use the "One/Two/Three button dialog" or "display message" vis at the "dialog and user interface" pallete.
    Please give it a try and send your own VI. Do not expect us to provide a working solution.
    Regards,

Maybe you are looking for