DB need too many locks in single threaded application

Question

Hello,
I`m new to BDB, and search for help of gurus.
I try to create database importing data from csv files. Import procedure is already written and works fine, but sometimes it starts to need too many locks.
Import process read csv file line by line. Check if key is already in the database, if yes it updates data, otherwise create new record. Import procedure is using DBEnv and transaction.
Please, look at 2 cases of data to import.
Case 1: Data in csv files are like:
key1, some_data_1
key1, some_data_2
key2, some_data_3
key2, some_data_4
keyN, some_data_2N-1
keyN, some_data_2N
(key became a key and data became the value in the database)
Case 2: Remove each second line in the csv file, so it became:
key1, some_data_1
key2, some_data_3
keyN, some_data_2N-1
In this case for successful import of the file more locks are needed then in case 1.
This seems very odd to me. Could somebody explain why lock system behave so? Looking forward to hearing any response.
Thanks.
p.s.: maybe my explanation is not very clear, but i don't know which other details are really important. I could provide more details if it could help.
Message was edited by:
user647892

Answer 1

There is nothing wrong.
Oracle by default locks records. In case 1, you insert a record (1 row level lock) and update that same record. Why is a mystery, but the update doesn't require a lock (by you) as you already did lock the record.
In case 2, you have distinct keys for each line, so each line results in a row level lock as expected.
There are as much record locks as you request it to make.
You could consider locking the table explicitly once, prior to the transaction, by a lock table statement.
However, your approach is fundamentally flawed an non-scalable, as it processes each line individually instead of loading the data in a staging table and transferring it to the production table using
insert into production table
select * from staging table
where key not in (select key from product table)
Sybrand Bakker
Senior Oracle DBA