How is PeopleSoft Memory Managed?

Question

I'm running a custom app engine with one PeopleCode section which is very brief and simple calls an app package. This app package calls another app package that is using a CI. The process is failing after processing approximately 10,000 out of 20,000 rows (the actual number of rows varies from run to run, same data), with a simple out of memory error. I've had our SysAdmin increase the memory for Java without any change. So, my question is how does PeopleSoft manage it's memory. Is it best to cancel the CI after each row or keep one instance and do a new get/create (I believe that's doable)? Should I should I re-instantiate the app package class objects after each row, etc. Does anyone have any clue as to how the memory is managed?

Answer 1

Jim - As to fine tuning, I probably won't get around to it. I suspect that you are right about somewhere there's a tipping point. But the CI, is CRM's Consumer Component, so there's just a lot to it, which is why it takes a while to process 20,000+ rows. To answer your restart question, it's probably worth describing the higher levels of how it all fits together.
The way we use CRM, requires that we are frequently required to load large amounts of data from spreadsheets into various components - Users, Workers, Consumers, configuration tables, etc. Often times there's some complex logic associated with processing the raw data. So, to accommodate that need we started out by building an almost abstract packages/class to handle all the common needs of opening, reading and manipulating an input file, trap errors, write to a custom log, etc. We then extend this class to meet the specific needs of a given load. At the same time since we were frequently using the same code for a given CI, we started creating custom classes for each CI. So that all this would be reusable whether we were doing a load or needing to use it somewhere else. Somewhere along the ling PeopleSoft has probably caught up to what we've been doing, but we're too invested in our own methodology to take a look. But, since we were always loading from files, our users wanted to see the progress each process was making. We incorporated logging each row processed out into our logs. (I know this is not very efficient, but when users were watching very long running processes, they wanted to be able to know it was making progress, so we forced the app engine/classes to write to the log by opening and closing the file.) I guess this a very wordy way to say we don't use restart. We just remove the processed rows from the file and start again. I'm sure we can find more efficient ways to do all this, given the added features of tools over the years, but we can very quickly turn out new CI loading processes, and we don't have the time to rework what we're currently using.
Thanks for the input.
I should also tell you that your original blogs on 64 Base encoding enabled us to figure out how to get all our attachments into our document app.
Thanks,
Russ
P.S. I forgot to answer your question. Yes basically you psuedocode is what we're doing. But, instead of ci = null, we use ci.cancel(). I assume that has the same affect?
Edited by: user4414415 on Apr 11, 2012 12:51 PM