Replication agent doesn´t free memory after expensive operation

Question

Our environment is made up of master node and 17 subscriber.
We have issued a delete action implying 1000000 rows. During the deletion, normal CPU, memory and swap usage was observed. Later, CPU usage has recovered while memory and swap has not. The most memory consumming process was timestenrep with 9Gb.
Deletion has been correctly replicated to all nodes. However, the replication agent needed to be restarted manually in order to recover memory/swap.
Why the replication agent did not recover on its own and free memory?
Definition in sys.odbc.ini
Datastore=/data/db/xxxx
PermSize=4000
tempSize=250
DurableCommits=0
TypeMode=1
LogFileSize=64
LogBuffSize=65536
MemoryLock=3
LockWait=10
PrivateCommands=1
Authenticate=0
DatabaseCharacterSet=WE8ISO8859P15
ConnectionCharacterSet=WE8ISO8859P15
CkptFrequency=300
CkptLogVolume=128
Connections=90
TimesTen Version installed (ttVersion command)
TimesTen Release 7.0.5.0.0 (64 bit Solaris) (ivpnges2tt70:17000) 2008-04-04T01:19:41Z
Instance admin: root
Instance home directory: /opt/TimesTen/ivpnges2tt70
Daemon home directory: /var/TimesTen/ivpnges2tt70
Regards,
Matú

Answer 1

In a replicated environment, executing very large transactions (such as deleting 1M rows in a single transaction) is a very hazardous thing to do and is something you should avoid at all costs. It can cause a number of serious problems. Transactions should be limited to no more than a few thousand operations. If you need to empty a table, use TRUNCATE instead. This will execute far more quickly, generate less log and will not cause problems for replication. If you cannot use TRUNCATE then you should break the DELETE up into multiple smaller transactions. For example if you wish to delete all row where 'startdate' is less than some value and this is likely to result in a large number of deletions then you could repeatedly execute the following sequence:
DELETE FIRST 1000 FROM sometable WHERE startdate < :someval;
COMMIT;
and just keep doing this until the 'rows affected' value is zero.
The same applies to large inserts and updates.
The memory usage behaviour of the replication daemon, while perhaps not ideal, is correct and as expected. When you delete 1M rows in a single transaction the replication daemon has to allocate memory (via malloc) for 17x copies (in your case) of the transaction data. That is probably a lot of memory. Once the transaction has been replicated to all 17 subscribers. This memory is moved to an internal 'free' list for re-use but is not returned to the O/S until the replication agent is terminated. Since it such a bad idea to execute these large transactions this behaviour is generally not a problem.
Chris