When do different scopes get removed from memory?

Question

Hi,
Here's what I've got.  I have a cfc that processes complex xml documents for product catalogs.  We've got functions in that cfc to process various parts of each document, with parts being passed to the function (product object struct, option struct, etc) in loops.
It would work fine for small docs, but as the documents got bigger, the process would slow, and as we approached 3-6 mb it would grind to a halt and after 4-9 mb, we'd get heap space errors.  Memory was just being consumed violently. 
All that was happening though is we do an xml search to get an array of products/options/media etc.  We'd loop that array passing in the option or whatever.  The function it was passed to just loaded the orm objects up and saved them, flushing and clearing the session at the end.
One thing I noticed trying to figure out what was going wrong was that even though is if we force kill the server half way through the for loop, we get a hundred or so errors that there's variables, but no ORM.  Makes me think that even though we had items being passed through arguments and then all variables in the functions being var scoped, variables are still hanging in memory attached to the sessions. 
Which makes me ask the question, when do variables in each scope get marked for garbage collection and will var scoped/argument scoped variables hang through the entire request?
I found the best/only solution to my problem was to create a variable scoped argument to use as an accessor for the product/option/media object between the loop and the method doing the work.  When I changed to not passing it as an argument, my memory consumption went up initially, but then held at the same level and my logic started to proceed at a constant rate.
Thanks,
Joel

Answer 1

I wish I could go that simple.  But we're using Coldbox, so there's a lot more being put into variables scope than what I'm using.  Also, it seems to be narrowing down to orm calls. 
We've got a pretty basic bit of logic, as it loops through an xml file over say elements that represent options for a product, we use the ormExecute to find if the object exists, if it doesn't, create a new one, set some more variables, save and flush the orm.  That's really all it comes down to. 
I'd turned on logging the gc to console and we can watch as each method runs to push an element into the orm the memory used grows.  Even with doing a forced gc, we see it keep growing.  Even tried commenting out all code except for a single ormExecute to get the object if it exists and that part still kept making the memory grow. 
Which makes me wonder if there's something in CF orm that caches, but doesn't get cleared on a flush.  We don't have the orm cache turned on though, so it isn't making a lot of sense yet.