Specific question about an OutOfMemoryError.... with code!

Question

i have the need to move tons of small chunks of byte[] data. We're talking 100-450 Million chunks of 750-2000 bytes of data.
I am trying to streamline some of my algorithms. Because of the checks and edits in place I need to use a LinkedList to hold these in until they are rewritten to disk.
The LinkedList generally holds 1 or 2 of these chunks at a time before passing it on to be written to disk.
Anyways, i was trying to gauge what kind of time is consumed in this process alone. I constructed the following FOR loop:
               LinkedList ll = new LinkedList();
               byte data[] = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX".getBytes(); //about 700 bytes of data
               byte claimNew[];
               for(int i=0;i<10000000;i++){
                    ll.add(data);
                    claimNew = (byte[])ll.get(0);
                    if(i%100000==0){
                         System.out.println("Processed: " + i);
               }i wanted to run it and increase the number of iterations to see approximately how much time is consumed in this process. NOTE: I know this isn't a very effificent
way to get this information, but it's fast and easily toyed with. I'm not writing a thesis, i just wanted some ballpark ideas.
Anyways, when this loop runs to about 2.7 million it dies with an OutOfMemoryError. I used only the default heap settings. However, this got me curious:
1-What is filling up the heap? is it the space allocated for the byte buffer in the class cast?
2-Shouldn't the byte buffer be marked for the garbage collector once the for loop reaches the end of a certain iteration? Shouldn't that free up heap space?
I am curious about the answer, it seems like this error shouldn't have happened in the first place.
I appreciate any info!
NEVERMIND! I'm a total dork. i didn't notice that i never put that little part in to ... ahem ... REMOVE the data chunk once i've read it from the LinkedList! SORRY!
Edited by: pnandrus on Mar 11, 2010 9:33 AM

Answer 1

Now however you do have the advantage now of knowing roughly how many your can process at once without running out of memory.