Table Comparisons and Deadlocks

Post Author: Thang Nguyen
CA Forum: Data Integration
Hi ,
Pretty new to this DI stuff, but I've got a dataflow where I'm using a Table Comparison Transform to work out my updates and inserts. My database is SQL server 2000.
When it runs the Table Comparison I get SQL errors regarding deadlock victim and the insert fails. I've ran a trace on SQL Server and the insert statement is being blocked by a select statement, so looks like some sort of issue with the Table Comparison looking for the differences and inserting new rows at the same time.
I've tried to split the operation into two Dataflows using Map Operation where one is doing the updates and the other does in the inserts, but I still get the deadlock issue.
Has anyone else experienced this problem?
Thanks
Thang

Post Author: Thang Nguyen
CA Forum: Data Integration
If anyone is interested the solution I've got from BO is:
"Can you put the following parameter in  your DSConfig   / al_engine section  :  SQLServerReadUncommitted=1 "
Beware that this sets your changes the SQL server Transaction Isolation level to allow dirty reads which isn't ideal

Similar Messages

  • Table Comparion and History Preservation

    Hi all
    I am new in using DI, I have a question about two most common objects in DI (Table Comparison and History Preservation).
    Can we  compare different structure tables in Table comparison or the tables should have the same structure? The same question about History Preservation. Can we connect History Preservation to a different structure table based on our Table comparison?
    Thanks for your help in advance.

    You can as long as your primary key and comparison columns in both tables have the same names.   There are some situations for which a table compare transform could be useful when comparing differently structured tables.   But the History Preserve transform is pretty specific.  I'm can't imagine what you'd want to do with it besides loading a type 2 dimension.

  • I HAVE A SOURCE TABLE WITH 10 RECORDS AND TARGET TABLE 15 RECORDS. MY WUESTION IS USING WITH THE TABLE COMPARISON TRANSFORM I WANT TO DELETE UNMATCHED RECORDS FROM THE TARGET TABLE ??

    I HAVE A SOURCE TABLE WITH 10 RECORDS AND TARGET TABLE 15 RECORDS. MY QUESTION IS USING WITH THE TABLE COMPARISON TRANSFORM .I WANT TO DELETE UNMATCHED RECORDS FROM THE TARGET TABLE ?? HOW IT IS ??

    Hi Kishore,
    First identify deleted records by selecting "Detect deleted rows from comparison table" feature in Table Comparison
    Then Use Map Operation with Input row type as "delete" and output row type as "delete" to delete records from target table.

  • Erratic behaviour map operation after table comparison

    BODI XIR2 11.7.3.6
    We want to detect and store changes in source data and store these changes in the target table.
    After the table comparison the row has got an update operation code flag and goes to a Map operation that converts Update row types to Normal and discards all other row types.
    normal -> discard
    update -> normal
    insert -> discard
    delete -> discard
    The map operation behaviour is erratic: sometimes the update row is mapped to normal and sometimes the update is discarded.

    I would be surprised if that is the case. You could run the dataflow in debug mode, 'cause there you can see the data and the OPCode flag of insert/update/delete after the TC transform.

  • Using table comparison can we use multiple tables as source?

    Using table comparison can we use multiple tables as source?
    Thank you very much for the helpful info.

    Table Comparison
    1) Input Data coming in
    2) Comparison table (table to which the data is compared)
    3) Output (input rows with respective opcodes based on the comparison result of input dataset with the comparison table)
    If your question is whether table comparison can accept union/join of multiple table sources, you can achieve by using merge/query transforms and then feeding to table comparison. Here, you have to be careful about choosing the primary keys inside table comparison

  • Clustered indexes and deadlocks

    Hi,
    I have run into some problem with clustered indexes and deadlocks. I have found some breadcrumbs about this on the web but didn't really understand everything. I am an DBA by accident and mainly BI and DWH developer. The article most relevant to the problem
    seems to be the following:
    SQL Server Deadlocks Caused By Clustered Index Scan .
    The database is running with READ COMMITTED SNAPSHOT Transaction Isolation Level. Problematic seems to be the second query. First a row is inserted into table SubjectRevisionEntity.  Second a row is inserted into table Partner which has a foreign key
    on SubjectRevisionEntity. This foreign key is validated using a Clustered Index Seek.
    Having done some research on the topic using the internet my hypothesis is as follows:
    - Insert from Query 1 in Session 1 locks page in Table SubjectRevisionEntity
    - Now a new session (Session 2) is started. Insert from Query 1 in Session 2 probably locks the same page in Table SubjectRevisionEntity.
    - Insert from Query 2 in Session 1 locks page in Table Partner
       Lock on page in Table SubjectRevisionEntity is necessary to do the Clustered Index Seek. However this page is already locked by Session 2. However, Session 2 needs to lock page in Table Partner which is already locked by Session 1 --> deadlock
    occurs
    Does this make any sense? At the moment I am not having the means to test the hypothesis but I will look after that.
    I am just thinking about countermeasures to undertake. What about configuring the index to avoid page locks? All other queries seem to be fine I suppose as they operate on only one table. My fellows from software engineering favour to replace clustered indexes
    by nonclustered indexes as the already have done in the past. However, I think the disadvantages of nonclustered indexes aka heaps regarding storage (forwarded records) and query performance are much bigger than their use for problems like these.
    Regarding the
    article I did not understand the author's point, that two simultaneous table scans on one table by two sessions won't work. I thought that this is no problem as the sessions would use a shared lock on the table.
    Thank you very much for sharing your expertise in advance!
    Martin

    As you describe this that cannot be the cause of your deadlock.
    After session 1 executes query 1, it will have an IX (Intent Exclusive) lock on the clustered index of table SubjectRevisionEntity (note that a lock on a clustered index is a lock on the table since the table is contained in the clustered index), also an
    IX lock on the page in the index where the new row will be inserted and an X (exclusive) lock on the key that you just inserted.
    When session 2 executes it also needs an IX lock on the clustered index of table SubjectRevisionEntity, this is allowed because multiple sessions can have IX locks on the same resource at the same time, also an IX lock on the page in the index where the
    new row will be inserted (also allowed even if this entry is in the same page as the row inserted by session 1), and an X lock on the key that session 2 inserted.  This is also allowed UNLESS session 2 and session 1 are trying to insert a row with the
    SAME primary key value.  From your description, I gather that session 1 and session 2 are trying to insert different keys.
    Then session 1 attempts to insert a row in Partner which has a foreign key reference to the row in SubjectRevisionEntity.  That means it must check for the existence of the row that session 1 inserted.  But it can do this because all it needs is
    a S (shared) lock on the SubjectRevisionEntity table, and an S lock on the page.  It that get those even though session 2 has an IX lock on those resources because S locks and IX locks are compatible.  It also needs an S lock on the row in SubjectRevisionEntity. 
    That is no problem unless it is trying to reference the row which session 2 just entered.  (Once again, I assume this is not the case in your situation?)  It then inserts the row in Partner getting an IX lock on the Partner table, an IX lock on the
    page and an X lock on the new key in Partner.
    Then session 2 attempts to insert a row in Partner.  That will work unless either it is inserting a row in Partner with the same primary key as the row inserted by session 1 or it is trying to reference the same row in SubjectRevisionEntity that session
    1 inserted.
    So this cannot be the cause of your deadlock unless both sessions are entering the same key values.  The fact that they may both be entering keys on the same page should not be causing you deadlock problems.
    Regarding your question about why 2 table scans of the entire table (or entire clustered index scan if the table has a clustered index), you are correct, the scans do get shared locks and there is no deadlock problem UNLESS both sessions are holding locks
    that are incompatible with S locks.  For example, if you were in read committed mode and session 1 had inserted a row with clustered index key = 47 and session 2 had inserted a row with clustered index key = 23 and and both sessions attempt to do a complete
    clustered index scan (by, for example, by doing something like SELECT <blah blah> FROM <table> WHERE <some nonindexed column> = 0, then both sessions will try to get S locks on every row so session 1 will be stopped at key = 23 and session
    2 will be stopped at key = 47 and that is a deadlock.  BUT
    You are using READ COMMITTED SNAPSHOT, not READ COMMITTED.  In READ COMMITTED SNAPSHOT, writes do not block reads.  So the situation above does not apply to you since neither of the sessions would be blocked because it was attempting to read a
    row which was locked by an update from another session.
    Tom

  • COLLATE Error on Table Comparison

    Hi,
    We have just upgraded to from version 11.7.3 to version 12.2.2 and we are getting this error when running the table comparison transform with sorted input:
    "Expression type int is invalid for COLLATE clause. "
    I asked our DBA to see what BODS was passing through at it is sending this:
    WHERE ( "TCRdr_1"."VEHICLE_ID_NK"  >= @P1  COLLATE Latin1_General_BIN)
    Why is it using COLLATE, and more importantly is there anyway to control this?
    The database is SQL Server 2005.
    Thanks

    this issue is fixed in 12.2.3, there were some issues prior to 12.2.3 related to Access Violation in case of SQL Server 2008 as target table, for DATETIME and DATETIME2 datatypes in target table
    as you mentioned you are using SQL Server 2005, do you get the Access Violation in a particular case, is the issue consistently reproducible ? please file a support case for the Access Violation issue or give me a reproducible scenario so that I can file a bug for that
    if you don't want to apply the 12.2.3 then you will see this issue for all table compare with sorted input and key columns with datatype other than CHAR, VARCHAR or NCHAR or NVARCHAR

  • Table comparisons

    Hi,
    We want to compare the tables between DEV and QA.
    Can you please let me know as what is the quick process we need to follow to compare all the tables between DEV and QA.
    Please help
    Thanks

    Hi Ram,
    You can try either of the below methods:
    1. Tcode SCMP - provide the table name and RFC connection and click on comparison button
    2. Tcode SM31 - provide table name then display. From the Utilities menu select Comparison and provide the RFC connection.
    3. Tcode SCU0 - Manual select then create, provide table name and RFC connection
    Hope it helps!
    Regards,
    Nupur

  • Table Comparison Performance

    Anyone has experience with using table comparison transforming for large volume data, e.g. 10M records.
    How is the performance?
    Any experience and suggestions is welcome.

    Performance will depend on a lot of factors, but the two big ones are:
    1. how many columns you are using in your compare (more columns will be slower)
    2. how you do the compare (sorted input is WAY faster than row comparison)
    Without knowing any details about your data or the process you are trying to implement it's difficult to make any additional recommendations.  But keep in mind that there are other options - especially for really big data sets.
    For example, you can always use a two step process that first deletes existing records matching the incoming set of primary keys, and then do a straight insert of the rows.   This will avoid the whole comparison step - and avoid doing updates which are much slower than inserts.
    However, this only works if you are replacing the existing records.  It wouldn't work if your table compare is part of a type 2 dimension load or something that requires you to track history.

  • TABLES COEJ AND  COEP

    Hi,
    I am trying to develop an account based profitability analysis, with a PLANNED DATA VS ACTUAL DATA comparison.
    To my understanding COEP holds the actual data. And i think COEJ holds planned data.
    COEJ has all numeric fields.
    I was wondering where the CHARACTERISTIC DATA is for PLANNED DATA?
    How do i do a PLANNED DATA VS ACTUAL DATA comparison in BI?
    Thanks

    Hi Sam,
    Just so that I am clear I am planning in using one cube only for both. So data from Table COEJ and COEP will be taken into that cube.I am using the same key figures that is 0AMOUNT. The characteristics would be automatically displayed for planned data also just like it would show for the actual data.
    Also since both datas are going to be in one cube i do not need to bring in characteristic data from a seperate table into that cube.
    Please confirm
    Thanks

  • ABAP Query: FI Tables - BSEG and BKPF

    Dear All,
    I have tried to use SAPQuery - SQ01 to join 2 SAP tables (BSEG and BKPF) by creating an Infoset (SQ01 >> Environment >> Infosets >> Create)
    However, there is always an error message for the table join when I generate the infoset. Tables BSEG and BKPF are joined together by conditions BSEG-BELNR & BKPF-BELNR; BKPF-BURKS & BKPF-BURKS; BSEG-GJAHR and BKPF-GJHAR.
    The error message prompted is, "You cannot use comparisons with column references in pool and cluster". Is the message trying to say that BELNR, NURKS and GJAHR are column references in pool and cluster, thats why we can't join the tables?
    The purpose of the infoset is to extract accounting documents. I will need to extract accounting doc data from DEV, QA and Prodcution Server.  If I use SAP QuickViewer, everytime I go to a differenct server I will need to create the QuickViewer in the system before I can use it. But if i use SAP Query, i will need to create once and transport the SAP Query to another server.
    Kindly advise if there is a way I can extract accountng document by using SAP Query?
    Million Thanks in Advance.

    Hi Mark,
    I appreciate your response.
    Im tyring to extract information for open GL items. Thus the relevant tables would be BSIS. But I find that  some fields such as Purchase Order Number; Purchase Order Line Item, Asset No.,Asset Sub-No which Im interested in do not exist in BSIS. But these fields can be found in Table BSEG?
    If I have to link BSEG and BKPF, how should i go about doing it?
    Kindly advise.
    Thanks in Advance!

  • What is difference between table space and shchema

    what is difference between table space and shchema ?

    784633 wrote:
    so each user has it own space of tables - schema ?yes, but let's clarify a bit ....
    The "schema" is the collection of all objects owned by a particular user. So if user SCOTT creates two tables, EMP and DEPT, and a view EMP_RPT, and a procedure GET_MY_EMP, those objects (tables, views, procedures) collectively make up the SCOTT schema.
    Those objects will be physically stored in a tablespace.
    A tablespace is a named collection of data files. So tablespace USERS will be made up of one or more data files. A specific datafile can belong to one and only one tablespace. If a tablespace has more than one data file, oracle will manage those files as a collection invisible to the application - much like the OS or disk subsystem handles striping across multiple physical disks.
    A specific object in the SCOTT schema can exist in only one tablespace, but not all objects of the schema have to be in the same tablespace. Likewise a tablespace can contain objects from multiple schemas.
    and can one user to access tables of other users?As others have said - FRED can access tables belonging to SCOTT as long has SCOTT has granted that access to FRED.

  • Key fields in tables ce31000 and ce41000

    In transaction KE11 ( change sales and profit plan ), tables CE31000 and CE41000 are used. I have to read those tables in a report but I cannot find the relevant keys to find unique records.  There are some fields I know, but using those fields I get about 300 records where I only should have one.  I guess I should use another table which has the link to the other tables, but I have no idea which.  Does anybody have a clue ?

    As far as I remember there are 4 tables per operating concern, CE1xxxx for tha actuals, CE2xxxx for the planning, CE3xxxx for the summary lines and CE4xxxx for the objects.
    Depending on the defined characteristics there is one PAOBJNR in CE4xxxx for each existing combination. In CE3xxxx you should also find this PAOBJNR. At the moment I have no access to a SAP with COPA, but I hope, this helps you a little bit for the further search.
    Best regards
    André

  • Table Backup and Best Practice

    Hi Guys,
    We have 2 tables, master and child with millions of records in it.These tables gets populated from flat files that we receive from external systems.The major operation on this table are insert/delete(update). The way we do update is, if the record from flat file already exist in table we delete the matching record from master table & child table and re-insert the data from flat files.
    Business has decided to delete (archive) old and inactive data from these 2 tables. This process is run every year. I want to take a backup of this table by creating a similar table in database and start the delete process.
    What would be the best approach to take backup of table,because we run the archive process every year. Archive process here refers to physically deleting a record from a table.
    Any help is greatly appreciated.

    922855 wrote:
    Hi Guys,
    We have 2 tables, master and child with millions of records in it.These tables gets populated from flat files that we receive from external systems.The major operation on this table are insert/delete(update). The way we do update is, if the record from flat file already exist in table we delete the matching record from master table & child table and re-insert the data from flat files.
    Business has decided to delete (archive) old and inactive data from these 2 tables. This process is run every year. I want to take a backup of this table by creating a similar table in database and start the delete process.
    What would be the best approach to take backup of table,because we run the archive process every year. Archive process here refers to physically deleting a record from a table.
    Any help is greatly appreciated.expdp

  • Why table getWidth and setWidth is not working when resize column the table

    hi all,
    i want to know why the setWidth is not working in the following code,
    try to uncomment the code in columnMarginChanged method and run it wont resize the table.
    i cont set width(using setWidth) of the other table column using getWidth of the main table column.
    and i want to resize the right side columns only (you can check when you resize the any column the left and right side columns also resizing)
    any suggestions could be helpful.
    import java.awt.BorderLayout;
    import java.awt.Dimension;
    import java.awt.event.ActionEvent;
    import java.awt.event.ActionListener;
    import javax.swing.Box;
    import javax.swing.BoxLayout;
    import javax.swing.JButton;
    import javax.swing.JFrame;
    import javax.swing.JPanel;
    import javax.swing.JTable;
    import javax.swing.SwingUtilities;
    import javax.swing.event.ChangeEvent;
    import javax.swing.event.ListSelectionEvent;
    import javax.swing.event.TableColumnModelEvent;
    import javax.swing.event.TableColumnModelListener;
    import javax.swing.table.TableColumnModel;
    public class SynTableResize extends JFrame implements TableColumnModelListener, ActionListener
        JTable  table1       = new JTable(5, 5);
        JTable  table2       = new JTable(5, 5);
        JTable  table3       = new JTable(5, 5);
        JButton btn          = new JButton("refresh");
        JPanel  pnlcontainer = new JPanel();
        public SynTableResize()
            setLayout(new BorderLayout());
            pnlcontainer.setLayout(new BoxLayout(pnlcontainer, BoxLayout.Y_AXIS));
            pnlcontainer.add(table1.getTableHeader());
            pnlcontainer.add(Box.createVerticalStrut(5));
            pnlcontainer.add(table2);
            pnlcontainer.add(Box.createVerticalStrut(5));
            pnlcontainer.add(table3);
            table2.setTableHeader(null);
            table3.setTableHeader(null);
            table1.getColumnModel().addColumnModelListener(this);
            table3.setColumnModel(table1.getColumnModel());
            table2.setColumnModel(table1.getColumnModel());
            table2.getColumnModel().addColumnModelListener(table1);
            table3.getColumnModel().addColumnModelListener(table1);
            btn.addActionListener(this);
            getContentPane().add(pnlcontainer, BorderLayout.CENTER);
            getContentPane().add(btn, BorderLayout.SOUTH);
            setSize(new Dimension(400, 400));
            setVisible(true);
            setDefaultCloseOperation(EXIT_ON_CLOSE);
        public static void main(String[] args)
            new SynTableResize();
        public void columnAdded(TableColumnModelEvent e)
        public void columnMarginChanged(ChangeEvent e)
            TableColumnModel tcm = table1.getColumnModel();
            int columns = tcm.getColumnCount();
            for (int i = 0; i < columns; i++)
                table2.getColumnModel().getColumn(i).setPreferredWidth(tcm.getColumn(i).getWidth());
                table3.getColumnModel().getColumn(i).setPreferredWidth(tcm.getColumn(i).getWidth());
                // the following commented code wont work.
    //            table2.getColumnModel().getColumn(i).setPreferredWidth(tcm.getColumn(i).getPreferredWidth());
    //            table3.getColumnModel().getColumn(i).setPreferredWidth(tcm.getColumn(i).getPreferredWidth());
    //            table2.getColumnModel().getColumn(i).setWidth(tcm.getColumn(i).getWidth());
    //            table3.getColumnModel().getColumn(i).setWidth(tcm.getColumn(i).getWidth());
            SwingUtilities.invokeLater(new Runnable()
                public void run()
                    table2.revalidate();
                    table3.revalidate();
        public void columnMoved(TableColumnModelEvent e)
        public void columnRemoved(TableColumnModelEvent e)
        public void columnSelectionChanged(ListSelectionEvent e)
        public void actionPerformed(ActionEvent e)
            JTable table = new JTable(5, 5);
            table.setColumnModel(table1.getColumnModel());
            table.getColumnModel().addColumnModelListener(table1);
            pnlcontainer.add(Box.createVerticalStrut(5));
            pnlcontainer.add(table);
            pnlcontainer.validate();
            pnlcontainer.repaint();
    }thanks
    dayananda b v

    hi,
    thanks for your replay,
    yes i know that, you can check the following code it works fine.
    actually what i want is, when i resize table column it shold not automaticaly reszie when table resized and i dont want horizontal scroll bar, meaning that all table columns should resize with in the table size(say width 300)
    if i make table autoresize off than horizontal scroll bar will appear and the other columns moved and i want scroll to view other columns.
    please suggest me some way doing it, i tried with doLayout() no help,
    doLayout() method only can be used when table resizes. first off all i want to restrict table resizing with in the limited size
    import java.awt.BorderLayout;
    import java.awt.Dimension;
    import javax.swing.Box;
    import javax.swing.BoxLayout;
    import javax.swing.JFrame;
    import javax.swing.JPanel;
    import javax.swing.JScrollPane;
    import javax.swing.JTable;
    import javax.swing.event.ChangeEvent;
    import javax.swing.table.TableColumnModel;
    public class TempSycnTable extends JFrame
        JTable  table1       = new JTable(5, 5);
        MyTable table2       = new MyTable(5, 5);
        MyTable table3       = new MyTable(5, 5);
        JPanel  pnlcontainer = new JPanel();
        public TempSycnTable()
            JScrollPane src2 = new JScrollPane(table2);
            JScrollPane src3 = new JScrollPane(table3);
    //        table1.setAutoResizeMode(JTable.AUTO_RESIZE_OFF);
    //        table2.setAutoResizeMode(JTable.AUTO_RESIZE_OFF);
    //        table3.setAutoResizeMode(JTable.AUTO_RESIZE_OFF);
    //        src2.setHorizontalScrollBarPolicy(JScrollPane.HORIZONTAL_SCROLLBAR_NEVER);
    //        src3.setHorizontalScrollBarPolicy(JScrollPane.HORIZONTAL_SCROLLBAR_NEVER);
            table2.setTableHeader(null);
            table3.setTableHeader(null);
            table3.setColumnModel(table1.getColumnModel());
            table2.setColumnModel(table1.getColumnModel());
            table2.getColumnModel().addColumnModelListener(table1);
            table3.getColumnModel().addColumnModelListener(table1);
            table2.setTableHeader(null);
            table3.setTableHeader(null);
            setLayout(new BorderLayout());
            pnlcontainer.setLayout(new BoxLayout(pnlcontainer, BoxLayout.Y_AXIS));
            pnlcontainer.add(table1.getTableHeader());
            pnlcontainer.add(Box.createVerticalStrut(5));
            pnlcontainer.add(src2);
            pnlcontainer.add(Box.createVerticalStrut(5));
            pnlcontainer.add(src3);
            getContentPane().add(pnlcontainer, BorderLayout.CENTER);
            setSize(new Dimension(300, 300));
            setVisible(true);
            setDefaultCloseOperation(EXIT_ON_CLOSE);
        public static void main(String[] args)
            new TempSycnTable();
        class MyTable extends JTable
            public MyTable()
                super();
            public MyTable(int numRows, int numColumns)
                super(numRows, numColumns);
            public void columnMarginChanged(ChangeEvent event)
                final TableColumnModel eventModel = table1.getColumnModel();
                final TableColumnModel thisModel = getColumnModel();
                final int columnCount = eventModel.getColumnCount();
                for (int i = 0; i < columnCount; i++)
                    thisModel.getColumn(i).setWidth(eventModel.getColumn(i).getWidth());
                repaint();
    }thanks
    daya

Maybe you are looking for