Cap at SRC1_DB Prop to TRG1_DB Apply then re-enqueue Prop to TRG2_DB Apply

Question

Hi, I need some assistance.
I am new to Streams and have been doing some research and experimentation. I have successfully setup a Stream from SRC1 db to TRG1 db, using a Propagation and a DML handler procedure.
SRC1............=>...............TRG1
capt_1.......prop...............apply_1
SRC1_Q....str1_to_str2......TRG1_Q
....................................DML_Hndlr_1
Now, I want to do do something more advanced.
SRC1............=>..............TRG1
capt_1.......prop...............apply_1
SRC1_Q....str1_to_str2......TRG1_Q
...................................DML_Hndlr_1
.......................................||..............->..................TRG2
.......................................\/.............prop................apply_2
...................................SRC2_Q.......str2_to_str3........TRG2_Q
...........................................................................DML_Hndlr_2
I want the apply_1 at the TRG1 db to put the LCRs onto a new queue, SRC2_Q. Up to this point, everything works ok. I can see the LCRs on the SRC2_Q.
The problem comes in when I then setup a second propagation. I create a propagation at TRG1, called str2_to_str3. All my scripts to set up the environment run without any errors. I can't find any errors in any of the data dictionaries. I have tested database links. I am stuck.
I have included all the setup scripts that I am using, as well as the log file from my latest installation.
I have also included my scripts to:
1) stop and start all the processes (not sure if these are correct)
2) perform and insert, update and delete on the source table.
3) cleanup the environment.
The scripts are easily configurable via SQLPLUS define statements.
Is what I am trying to do possible?
Please note that I do not want to perform a second capture at the TRG1 database. All the LCRs must be able to be Applied at both TRG1 and TRG2, via seperate apply processes, each using their own DML_Handler.
I see that queue forwarding and apply forwarding do not cater for this scenario above. What are the alternatives? Also comment on performance of the alternatives if possible.
I have all my scripts available and all my log files as well, but I am not sure how to upload them? Do I have to paste them into a response to this question on the forum?
Thanks

Answer 1

Thanks for the response.
There are 3 instances SRC1, TRG1 and TRG2. I want to capture on SRC1, Apply on TRG1, then re-enque and then propagate same LCR (source is SRC1) to TRG2 and Apply on TRG2 (source is still SRC1).
The application of such an architecture would be SRC1= transactional database, TRG1=Audit Trail database, TRG2=Reporting database with denormalised tables.
So, yes, I want to apply LCR for TRG2 also locally on TRG1.
We need to see also the declaration of the apply process on TRG2 as the source DB name now becomes important. CONNECT &db_TRG2_STREAMS_user/&db_TRG2_STREAMS_pass.@&db_TRG2_db
BEGIN
DBMS_APPLY_ADM.SET_DML_HANDLER(
object_name => '&db_SRC1_user..&db_SRC1_table',
object_type => 'TABLE',
operation_name => 'DEFAULT',
error_handler => FALSE,
user_procedure => '&db_TRG2_STREAMS_user..TRG2_dml_handler',
apply_database_link => NULL,
apply_name => '&db_APPLY_2');
END;
All the code is about 1000 lines.
Q)If you like, I can include it all.
If you have 3 test instances, you could run it, it is easy to configure. But for now, I'll include the parts where I think things could be going wrong. Assume that the first transfer is in place and is working. The Apply at TRG1 is using this DML handler (the Apply at TRG2 is identical):
CONNECT &db_TRG1_STREAMS_user/&db_TRG1_STREAMS_pass.@&db_TRG1_db
create or replace PROCEDURE TRG1_dml_handler(in_any IN ANYDATA) IS
lcr SYS.LCR$_ROW_RECORD;
rc PLS_INTEGER;
command VARCHAR2(30);
v_old_trg_ts VARCHAR2(30);
v_new_trg_ts VARCHAR2(30);
v_old_SRC1_ts VARCHAR2(30);
v_new_SRC1_ts VARCHAR2(30);
v_old_name VARCHAR2(30);
v_new_name VARCHAR2(30);
old_values SYS.LCR$_ROW_LIST;
v_temp ANYDATA;
v_temp_int PLS_INTEGER;
BEGIN
rc := in_any.GETOBJECT(lcr);
command := lcr.GET_COMMAND_TYPE();
lcr.SET_OBJECT_NAME('&db_TRG1_table');
     lcr.SET_OBJECT_OWNER('&db_TRG1_user');
     logger.log_it( 'HANDLER_LOG', 'TRG1_dml_handler : '||command||' : '||TO_CHAR(sysdate, 'HH:MI:ss AM : '));
IF command = 'INSERT' THEN
     lcr.SET_VALUE('new', 'TRG_TS', ANYDATA.ConvertVarchar2(TO_CHAR(sysdate, 'HH24:MI:SS AM')));
lcr.EXECUTE(FALSE);
END IF;
IF command = 'UPDATE' OR command = 'DELETE' THEN
v_temp := lcr.GET_VALUE('old', 'NAME', 'Y');
v_temp_int := ANYDATA.GetVarchar2(v_temp, v_old_name);
v_temp := lcr.GET_VALUE('old', 'SRC1_TS', 'Y');
v_temp_int := ANYDATA.GetVarchar2(v_temp, v_old_SRC1_ts);
SELECT TRG_TS INTO v_old_trg_ts FROM &db_TRG1_user..&db_TRG1_table WHERE NAME=v_old_name AND SRC1_TS=v_old_SRC1_ts;
lcr.SET_VALUE('old', 'TRG_TS', ANYDATA.ConvertVarchar2(v_old_trg_ts));
lcr.EXECUTE(FALSE);
END IF;
IF command = 'UPDATE' THEN
v_temp := lcr.GET_VALUE('new', 'NAME', 'Y');
v_temp_int := ANYDATA.GetVarchar2(v_temp, v_new_name);
v_temp := lcr.GET_VALUE('new', 'SRC1_TS', 'Y');
v_temp_int := ANYDATA.GetVarchar2(v_temp, v_new_SRC1_ts);          
     EXECUTE IMMEDIATE 'UPDATE &db_TRG1_user..&db_TRG1_table SET TRG_TS=TO_CHAR(sysdate, ''HH24:MI:SS AM'') WHERE NAME='''||v_new_name||''' AND SRC1_TS='''||v_new_SRC1_ts||''' ';
commit;
END IF;
     IF command = 'DELETE' THEN
lcr.SET_COMMAND_TYPE('INSERT');
lcr.SET_OBJECT_NAME('&db_TRG1_table._DEL');
old_values := lcr.GET_VALUES('old');
lcr.SET_VALUES('new', old_values);
lcr.SET_VALUES('old', NULL);
lcr.SET_VALUE('new', 'TRG_TS', ANYDATA.ConvertVarchar2(TO_CHAR(sysdate, 'HH24:MI:SS AM')));
lcr.EXECUTE(FALSE);
END IF;
END;
This handler works 100% for Applying on TRG1.
Q) What I not sure about is, what exactly gets put on the SRC2_Q. Is it the original SRC1 LCR, or is it the modified one that was Executed by this DML Handler?
define db_SRC2_STREAMS_user=&&db_TRG1_STREAMS_user
CONNECT &db_SRC2_STREAMS_user/&db_SRC2_STREAMS_pass.@&db_SRC2_db
DECLARE
     emp_rule_name_dml VARCHAR2(30);
BEGIN
SELECT RULE_NAME into emp_rule_name_dml FROM DBA_STREAMS_RULES where streams_type='APPLY' and streams_name='&db_APPLY_1' and RULE_OWNER='&db_TRG1_STREAMS_user';
     DBMS_OUTPUT.PUT_LINE('emp_rule_name_dml = '||emp_rule_name_dml);     
DBMS_APPLY_ADM.SET_ENQUEUE_DESTINATION(
rule_name => emp_rule_name_dml,
destination_queue_name => '&db_SRC2_STREAMS_user..&db_SRC2_Q');     
END;
This part seems to be working, the LCRs get into the SRC2_Q, because I can see a record for each LCR, when I query TRG1.SRC2_Q_TABLE.
e.g.
"Q_NAME","MSGID","CORRID","PRIORITY","STATE","DELAY","EXPIRATION","TIME_MANAGER_INFO","LOCAL_ORDER_NO","CHAIN_NO","CSCN","DSCN","ENQ_TIME","ENQ_UID","ENQ_TID","DEQ_TIME","DEQ_UID","DEQ_TID","RETRY_COUNT","EXCEPTION_QSCHEMA","EXCEPTION_QUEUE","STEP_NO","RECIPIENT_KEY","DEQUEUE_MSGID","SENDER_NAME","SENDER_ADDRESS","SENDER_PROTOCOL","USER_DATA","USER_PROP"
"AQ$_SRC2_Q_TABLE_E","B6A77B139CAC426CAFF6DB6A8000745B","","0","3","","","","0","0","7502525","0","06/OCT/10 13:34:49.420000000","SYS"," 17001","","","","0","","SRC2_Q","0","0","","TRG1_STR","","","SYS.ANYDATA",""
What is it declared into the LCR when it arrives on TGR2 and how is your table initialised? Well, at this stage, the LCR is not getting to TRG2. The second propagation is failing.
Any LCR enqueued into TRG2 into SRC2_Q is now an TRG1 LCR and should bear the meta data of TGR1. Perhaps this answers my quesion above. I will give it a quick try now, making the neccessary changes assuming that the LCR is now the modified LCR with TRG1 as it's source.
Q)Is there a way to get the Apply process to re-enqueue the original unmodified LCR?
Watch out the error if you duplicated the LCR into DML_hndlr_1, you still need to replace the meta data and the SCN of SRC1 by new one of TRG1 . What your are doing is not a downstreams capture.Q)I dont think that I duplicated the LCR in my DML Hanlder?
Thanks