Loading large files in Java Swing GUI

Question

Hello Everyone!
I am trying to load large files(more then 70 MB of xml text) in a Java Swing GUI. I tried several approaches,
1)Byte based loading whith a loop similar to
pane.setText("");
             InputStream file_reader = new BufferedInputStream(new FileInputStream
                       (file));
             int BUFFER_SIZE = 4096;
             byte[] buffer = new byte[BUFFER_SIZE];
             int bytesRead;
             String line;
             while ((bytesRead = file_reader.read(buffer, 0, BUFFER_SIZE)) != -1)
                  line = new String(buffer, 0, bytesRead);
                  pane.append(line);
             }But this is gives me unacceptable response times for large files and runs out of Java Heap memory.
2) I read in several places that I could load only small chunks of the file at a time and when the user scrolls upwards or downwards the next/previous chunk is loaded , to achieve this I am guessing extensive manipulation for the ScrollBar in the JScrollPane will be needed or adding an external JScrollBar perhaps? Can anyone provide sample code for that approach? (Putting in mind that I am writting code for an editor so I will be needing to interact via clicks and mouse wheel roatation and keyboard buttons and so on...)
If anyone can help me, post sample code or point me to useful links that deal with this issue or with writting code for editors in general I would be very grateful.
Thank you in advance.

Answer 1

Hi,
I'm replying to your question from another thread.
To handle large files I used the new IO libary. I'm trying to remember off the top of my head but the classes involved were the RandomAccessFile, FileChannel and MappedByteBuffer. The MappedByteBuffer was the best way for me to read and write to the file.
When opening the file I had to scan through the contents of the file using a swing worker thread and progress monitor. Whilst doing this I indexed the file into managable chunks. I also created a cache to further optimise file access.
In all it worked really well and I was suprised by the performance of the new IO libraries. I remember loading 1GB files and whilst having to wait a few seconds to perform the indexing you wouldn't know that the data for the JList was being retrieved from a file whilst the application was running.
Good Luck,
Martin.