AQ on RAC: Callback does not dequeue (sometimes)

Question

Hi,
We've got the following problem on 11.2.0.3:
RAC consists of 2 instances.
The Queue Table is defined with primary instance set to 1.
On instance 1 parameter job_queue_processes is set to 10 - on instance 2 it is set to 0!
Queues defined on this Queue Table are dequeued via callback functions.
Normally every incoming messages should be dequeued automatically by the defined callback functions. But in our setup this does not seem to work: messages are regularly hanging with status READY and need a manual dequeue.
What's wrong with this setup? Why does this not work?
Thanks for any input,
Stephan

Answer 1

Hi,
You need to check if there is any inconsistency.
select count(*) from sys.aq_srvntfn_table n where n.user_data.msg_id in
(select n.user_data.msg_id msgid from sys.aq_srvntfn_table n minus select msgid from APP_SCHEMANAME.APP_QUEUE_TABLENAME)
and n.user_data.queue_name = '"APP_SCHEMANAME"."APP_QUEUENAME"';
Note : in 10.1 onwards, where the schema and queue name can be mixed case, that the query to identify the mismatch needs to reference the n.user_data.queue_name correctly by enclosing the APP_SCHEMANAME and APP_QUEUENAME in double quotes
If above returns any row:
ACTION PLAN:
This shows messages are in the notification queue but not in the user-created application queue.
For those msgids returned, manually dequeue these messages by specifying the message id from the notification queue : AQ_SRVNTFN_TABLE_Q (in versions below 11.2) or AQ_SRVNTFN_TABLE_Q_<N> (in versions starting 11.2). In doing this, we are removing the message in the notification queue which is not in the Application queue. This should allow the notification activity to move on. e.g.
set serveroutput on
declare
enqueue_options dbms_aq.enqueue_options_t;
message_properties dbms_aq.message_properties_t;
dequeue_options dbms_aq.dequeue_options_t;
message_handle raw(16);
mes aq$_srvntfn_message;
begin
  dequeue_options.wait := dbms_aq.no_wait;
  dequeue_options.consumer_name := '<consumer_name>'
  dequeue_options.msgid := '<msg id>';    -- <<< supply mesage id
  dbms_aq.dequeue(queue_name => 'AQ_SRVNTFN_TABLE_Q',   -- <<< as appropriate
  dequeue_options => dequeue_options,
  message_properties => message_properties,
  payload => mes,
  msgid => message_handle);
  dbms_output.put_line('removed: ' || message_handle);
  commit;
end;
It may be required to kill off the register driver jobs to restart the mechanism correctly after the queues have been reconciled and the above query returns 0 rows indicating queues are consistent. It  is also recommended to restart the database after this process has been completed before enqueueing further messages into the user-created application queue.
Note : you may need to specify a consumer_name in the above if the related queue is a multi consumer queue
Thanks,
Reena