OutOfMemoryError: Java heap space when using List String

Question

Hey folks,
i stumbled about the following problem: I want to read IDs from a textfile (around 100.000 lines) an store them in a List or HashMap for later use. When I store them as Integers there is no problem, but when I store them as Strings I quickly run out of memory. The IDs are 6 chars long, so an Integer would use 2 Bytes and an unicode String 12. When I run the following code the memeory usage is ca. 13 MB, but when nbot commenting out the add-command for the string list, it qucikly raises up to ca. 70 MB and then throws the OutOfMemoryError. I tried a run with -mx1024M then the programm succesfully terminates but it uses a peak heap of 300 MB! Can anyone explain these differences?
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
public class ReadTheFile {
     public static void main(String[] args) {
          try {
               // read file
             BufferedReader in =
                  new BufferedReader(
                       new InputStreamReader(
                            new FileInputStream(args[0]), "UTF-8"));
             ArrayList<Integer> list1 = new ArrayList<Integer>();
             ArrayList<String> list2 = new ArrayList<String>();
             String line;
             while ((line = in.readLine()) != null) {
                 String str = line.substring(10, 16);
                 list1.add(Integer.valueOf(str));
                 //list2.add(str);
          } catch (Exception e) {
               e.printStackTrace();
}Any help would be appreciated. Thanks.
schneidexe
P.S.: I also tried HashMaps instead of List, same failure! :-(
Edited by: schneidexe on Feb 7, 2008 4:15 PM

Answer 1

kajbj wrote:
kajbj wrote:
What happens if you replace the line
String str = line.substring(10, 16);
with
String str = new String(line.substring(10, 16));
?I should probably explain why I want this tested.
A call to substring creates a substring that chares the character data with the original string (in this case the whole line), so all of your substrings are still forcing the whole lines to be kept in memory. Calling new String() on the substrings will create a new string that doesn't share the character data, so the char[] will be truncated and the string that is referencing the whole line can be GC:ed
KajThat did it! Thank you very much. I did not test this, because I generated the substring in both cases. But I think in the first case (w/o storing in in the List) the GC frees the memeory because the generated substring isn't used anymore after it's creation.
So thanx again!
schneidexe
P.S.: I'm using Java 1.6.0-b105