Needs tips for tuning Streams 10g propagation

Question

I setup bi-directional streams replication in Oracle 10.2.0.3 non-RAC between a pair of Solaris 10 servers on the same subnet with 100 Mbps fiber connection. Setup was done using MAINTAIN_SCHEMAS and it uses queue-to-queue propagation. Everything works well in both databases, but performance bottlenecks occur with propagation. Please advise on tuning options for Streams propagation.
Example, while importing 100 million rows into the first DB we see the count continually rise in both databases at a ratio of 5:1. That is, the local db grows five times faster than the remote db, so by the time the first database had 20 million rows loaded the second db only received 4 million rows through replication. As time went on the ratio got a bit worse, all the way down to 10:1. Both DBs have a 3 GB SGA and 1 GB pga. The AWR/ADDM reports very good statistics.
There are 4 capture processes in each db, and I raised the number of apply processes in each db to 6 with no impact detected after the change. All of these apply and capture processes seem to be starved for work. Capture continually goes into the state of "paused for flow control" while waiting for things to propagate out.
From what I have read in the Oracle Streams documentation, propagation is not parallelized between a given source + destination pair. What options do I have?

Answer 1

AQ_TM_PROCESSES sure makes life interesting. I agree with the recommendation to completely omit this parameter from your spfile and let Oracle auto-manage it, but that does not mean Streams will always work right. Sometimes flow control will get stuck in the on position, and the way you kick Oracle in the pants is to manually set AQ_TM_PROCESSES to some value and then remove the parameter later to get back to the recommended configuration.
I have read a few things about creating more propagation jobs. That is, simulate parallelism through multiple jobs because propagation parallelism is not supported. Right now my database only has one schema that owns one dblink to a single remote DB, and there is one propagation job configured for q2q over that link. This was all setup by the Oracle scripts when I ran maintain_schemas to setup replication. I'd like more documentation from Oracle on how else it could be configured or modified to have more propagation jobs to simulate parallelism.
Another angst ... the Oracle docs suggest when importing large tables to set commit=y so that if the process dies you can re-run the same import command and let Oracle ignore errors about existing rows. Yes, commit=y is great for importing into Streams because things will propagate and apply immediately with lower impact on flow / buffers / undo. But, wow, massive penalty if you follow their advice completely. For example, I killed the import after 3 million rows (6 hours) and restarted it. During the second attempt of importing data each of the 3 million rows generated an error but import kept going. It took 4 days to get past the errors! Recall it only took 6 hours to load the 3 million rows, but it took 96 hours to resume the import and get back to that same point. It would have been much better to truncate the table and let import reload things rather than generate errors.