Memory leak when transferring files Lion

Question

hi all, help   I'm experiencing a memory leak since upgrading to Lion when transferring shared files between my macbook pro and imac. This occures through ethernet (wireless/wired) and firewire 800. No other apps are open except activity montior and finder. I am trying to transfer about 100g of itunes music / video files and pictures. Time machine backups to my TC do not have this affect.
When transfer starts the free memory slowly decreases whilst the inactive memory increases. Eventually the free memory drops below 10mg, active memory is approx 5-6gb and wired/active is about 2gb. Bascially as the free memory decreases the inactive memory increases however does not get realocated so the mac grinds to a halt and dies with lots of page outs etc.
this only happens when trying to transfer files. I have tried whole folders and just small individual ones and noticed the same event. I am struggling to find the cuase of the problem - any ideas?
mac os x 10.7.1
processor 2.53 GHz Intel Core i5
Memory 8 GB 1067 MHz DDR3
cheers, Kevin

Answer 1

I dont think you have a memory leak. I think the problem is the xml file is 1000000 records long and takes up too much memory. Even if you find a way to increase memory size, you are loading down the server too much for other people's applications to run. I suggest instead reading up on xml and learning how to read in only a few records at a time, processing it, and getting the next set of records to process. There are two methods to parse an xml file using an xml parser, one is to parse it all and put it in memory, the other is to process one record at a time (an xml book explains it better).
However, I question why you have reports that are 1000000 records long. end-users cannot effectively use such records (you cant scroll through 1000000 records). I suggest finding a way to greatly decrease the number of records in each file such as by providing just the records a particular user needs to do his job and not all records. For instance, put a textfield on his screen to let him only fetch records within a certain date range.
Lastly, I suggest putting your code in a try/catch/finally block where the finally block actually closes the objects. Example:
finally{
if(conn!=null)
conn.close();
}