Unable to read big files into string object and java.lang.OutOfMemory Prob

Question

Hi All,
I have an application that uses applet and servlet communication. On the client side I am reading an large xml file of 12MB size (using JFileChooser) and converting the file to an string object using below code. But I am getting java.lang.OutOfMemory on the client side . But the same below code works fine for small xml files which are less than 4MB sizes:
BufferedReader in = new BufferedReader(new InputStreamReader(new FileInputStream(file),"UTF8"), 1024*12);
String s, s2 = new String();
while((s = in.readLine())!= null)
     s2 += s + "\n";
I even tried below code but still java.lang.OutOfMemory is coming:
while (true)
     int i = in.read();
     if (i == -1)
          break;
     sb.append(i);
Please let me know what am I doing wrong here ...
Thanks & Regards,
Sony.

Answer 1

Using a String is bad for the following reason:
When you initially create the String, it has a certain memory size (allocated length if you will). As you keep appending to this String, then memory reallocation will occur over and over, slowing your program down dramatically (ive seen with a 16k x 8 Char file taking 30 secs to read into memory using Strings in this way)
A Better way would be if you knew the number of characters in the XML file (Using some File size method for example) Then you can use a StringBuffer, which will pre allocate enough space (or try to, it may just be that you cannot create a string as large as you need). You can use toString() method to get the resultant in a String Object (the extra allocated space at the end of the Buffer will be removed)
StringBuffer strBuf = new StringBuffer(xxxx);
Where xx is the length (int). Assuming that you are only allowed to enter an int to the constructor then (platform depedant) an int is 2^31 at maximum (or whatever) which allows 2.14e9 characters, therefore an xml file being totally filed would allow a size of ~2048 MB to be read in.
Try it and see.