Reliable detect number of occurences (table lock needed or ?)

Question

Hi all,
I got an issue with detecting duplicate messages. Database clients process files and messages and create a hash value that is passed on to the database. The database should return the number of occurences of this hash value in the last (e.g) 14 days.
create table HashHistory ( ID NUMBER,
MESSAGEHASH VARCHAR2 (20),
TS TIMESTAMP);
create sequence SomeSequence;
insert into HashHistory values (SomeSequence.nextval,'first hash', systimestamp);
insert into HashHistory values (SomeSequence.nextval,'second hash', systimestamp);
create or replace procedure DuplDetection  (p_HashIn varchar2,
                                     p_occurences OUT number) AS
l_timestamp timestamp default systimestamp;                          
begin
  -- possible exclusive table lock here... lock table HashHistory in exclusive mode;
  insert into HashHistory values (SomeSequence.nextval, p_HashIn, l_timestamp);
  select count (1)
  into p_occurences
  FROM HashHistory
  where MESSAGEHASH = p_HashIn
  and   TS < l_timestamp
  and   TS > l_timestamp-14;
  commit; --to release the table lock if applicable
end;When this procedure is called by two different machines at the same time with the same new hash value ('third hash'). One session should return 0 while the other should return 1 as number of occurences.... With default Oracle behaviour using row level locks, and executing them in parallel both sessions will not be able to see the others sessions hash values, and both will return 0 occurences. Is an exclusive table lock my only option to enforce this behaviour or can I trust Oracle to handle this correctly?
I expect 10^6 hashes each day and possible up to 10 or 20 clients running at the same time generating and checking these hash values. What are the changes of both sessions returning the same value without an exclusive table lock (as in this example)? What other parameters would you consider?
I am on Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi

Answer 1

Nicosa wrote:
Hi,
Wouldn'that work if you do it as follows ?insert into HashHistory values (SomeSequence.nextval, p_hashIn,...);
commit;
select count (1)
into p_occurences
FROM HashHistory
where MESSAGEHASH = p_HashIn
and   TS > l_timestamp-14
and ID!=SomeSequence.currval;The second session should see that some others existing hash were inserted, and hence an be warned.No, it wouldn't work. Some kind of synchronization is required.
In multi-threaded environment you cannot predict the order of execution of this program.
Let say we have a server with only one processor and 3 users (sessions) connected to that server.
Three users (let say U1, U2 and U3) call this procedure to insert the same hash. The procedure has 3 operations: insert, commit, select,
and assume theoretically that these 3 operation are atomic (each takes only one processor cycle - in reality each of these operations can consume thousands of cycles).
Server can execute these calls in this order:
U1-insert (sequence + 1)
U1-commit
U1-select (returns 0)
U2-insert (sequence + 1)
U2-commit
U2-select (returns 1)
U3-insert (sequence + 1)
U3-commit
U3-select (returns 2)
- in this scenario results will be OK.
But the order might be:
U1-insert (sequence + 1)
U1-commit
- here server decides to switch to another process/thread
U2-insert (sequence + 1)
U2-commit
- server switches to proccess 3
U3-insert (sequence + 1)
U3-commit
- server switches back to U2
U2-Select (user 2 sees record commited by U1 and U3 and returns 2)
- server switches to U3
U3-Select (user 1 sees record commited by U1 and U2 and returns 2)
- server switches to U1
U1 - Select (this also returns 2)
- results are 2,2,2, but should be 0,1,2
If 20 users cal this procedure at the same moment with the same hash value, it is even possible that each user gets 19 as the final result,
but proper results should be 0, 1, 2, 3 .... 19 ;)
Without some kind of synchronization this a lottery.
There is also another trap with the sequence - consider this example:
Session 1
SQL> create sequence abc;
Sequence created.
SQL> select abc.nextval from dual connect by level <=5;
   NEXTVAL
      1
      2
      3
      4
      5
SQL> select abc.currval from dual;
   CURRVAL
      5Session 2
SQL> select abc.nextval from dual;
   NEXTVAL
      6
SQL> select abc.nextval from dual connect by level <=5;
   NEXTVAL
      7
      8
      9
     10
     11
SQL> select abc.currval from dual;
   CURRVAL
     11Back to session 1
SQL> select abc.currval from dual;
   CURRVAL
      5What happens in this scenario:
- user 1 does insert (select nextval from the sequence)
- user 2,3,4,5,6 ..... 100 call the procedure just 3 microseconds after U1 and do insert (increase the sequence)
- user 1 retrieve currval from the sequence ?