Out of memory error when writing large file

I have the piece of code below which works fine for writing small files, but when it encounters much larger files (>80M), the jvm throws an out of memory error.
I believe it has something to do with the Stream classes. If I were to replace my PrintStream reference with the System.out object (which is commented out below), then it runs fine.
Anyone else encountered this before?
     print = new PrintStream(new FileOutputStream(new File(a_persistDir, getCacheFilename()),
                                                            false));
//      print = System.out;
          for(Iterator strings = m_lookupTable.keySet().iterator(); strings.hasNext(); ) {
               StringBuffer sb = new StringBuffer();
               String string = (String) strings.next();
               String id = string;
               sb.append(string).append(KEY_VALUE_SEPARATOR);
               Collection ids = (Collection) m_lookupTable.get(id);
               for(Iterator idsIter = ids.iterator(); idsIter.hasNext();) {
                    IBlockingResult blockingResult = (IBlockingResult) idsIter.next();
                    sb.append(blockingResult.getId()).append(VALUE_SEPARATOR);
               print.println(sb.toString());
               print.flush();
} catch (IOException e) {
} finally {
     if( print != null )
          print.close();
}

Yes, my first code would just print the strings as I got them. But it was running out of memory then as well. I thought of constructing a StringBuffer first because I was afraid that the PrintStream wasn't allocating the memory correctly.
I've also tried flushing the PrintStream after every line is written but I still run into trouble.

Similar Messages

  • Out of Memory Error when writing from mySQL to JPanel

    Hi,
    I have a JPanel with BoxLayout to which I am adding JCheckBoxes, which are the result of an Select query to mySQL.
    My select query is giving me 15000 lines of result, in which case my program is screwed up, and I am getting OutofMemory Error.
    Does somebody know how to fix this problem?
    This is my code for the JPanel :
    srchRP = new JPanel();
         BoxLayout bl = new BoxLayout(srchRP, BoxLayout.Y_AXIS);
         srchRP.setLayout(bl);
         srchSP = new JScrollPane(srchRP);
         srchSP.setPreferredSize(new Dimension(300,470));

    If I understand correctly, your JVM simply cannot deal with 15,000 objects (the result set generated by your query). You can either make more memory available to the JVM as described above, or you can consider redesigning your application.
    It seems what you are trying to do is get every record in the database table, and display those records in a JPanel, allowing the end user to select which records he wants to perform an action on (such as delete).
    Instead of using a "select * from table" query, you could do a "select id, title from table" type of query, in which case, your data object will be a lot smaller and still give the user enough information to decide whether to delete the record or not.
    Alternatively, you could write a utility program to get the information you need from a text file generated as a result of a database dump and load that information upon startup, i.e., cache the master list of records.
    But, it seems to me, if I were your customer, I would not want to scroll through a list of 15,000 entries and deciding which ones I wanted to check off. If you figure out an easier way to manage 15,000+ records, such as allowing the user to narrow down his criteria, then you go a long way towards optimizing your code to reduce memory requirements.

  • Out of memory error when creating large matrix

    hello,
    i have to create a matrix of 100000 cols and 10000 rows for store numeric values, 0 and 1, but when the program is executing , the error "outofmemory no stack trace available" appear.
    �how can I resolve this? Is neccesary to modify the JVM?
    Urgent !!!!!!!!!!!!!!!
    Thanks in advance.

    Which is 125 MiB of memory, minimum.Doesn't follow:
    http://en.wikipedia.org/wiki/Sparse_matrix
    If this were for sparse data a simple list of Integers would represent the same data just as effectively as a matrix.
    If the data is not sparse, a one-dimensional byte array would be my choice.

  • Out of Memory error when opening pdf file.  Windows XP Reader v 10.0.1

    I am getting this message when trying to open some of
    my saved pdf files.  It happens on some but not others.

    yeah its working fine. Thank you very much..
    But it would be better if we provide the chunklength rather than using a zero to upload the file fastly.

  • Out of memory error when sending messages

    Hello!
    I have OS X 10.4.10 and I have an email account setup (IMAP)in mail.
    Lately when I go to send a message, I get an "Out of Memory" error that it says is coming from the mail server, but I know for a fact that it's not a server issue (mail account works on a PC fine). If I try to send it again it goes through just fine.
    Any ideas?

    You are building an HTML table with 1500 rows? That is definitely a lot. And I assume you are also putting some styling on that table...
    These days programmers resource to [url en.wikipedia.org/wiki/AJAX]AJAX techniques for things like that.
    You can, for example, send the table data as a list of comma separated values, and then use Javascript on the browser to create a partial table with options to browse through the different pages of data.
    If you need to show all that data in one single page, you can still try sending just the data as CSV or XML and then building the table on the browser via a Javascript loop. Use CSS for the styling so you don't overload the HTML code with styling information.
    Marcos Broc
    Hi,
    I have a bit of a problem. I?m trying to generate
    quite a large JSP file (a table with about 1500
    rows). The problem is that I get an out of memory
    error when the JSP is being generated. And it is
    definitely that the JSP is too large, or rather the
    HTML being generated from it. Now I was under the
    impression that the JSP page was flush now and again,
    but I tried to set all the different flush options
    and I have also tried to manually flush the page.
    Nothing of this made any difference except for
    displaying a white page instead of an error page
    since the header was already sent. Now this suggests
    to me that the page is being cached internally by the
    server and then sent to the client. So is there
    anything I can do about this?
    Regards
    Hertz

  • Out of Memory Error when generating JSP

    Hi,
    I have a bit of a problem. I?m trying to generate quite a large JSP file (a table with about 1500 rows). The problem is that I get an out of memory error when the JSP is being generated. And it is definitely that the JSP is too large, or rather the HTML being generated from it. Now I was under the impression that the JSP page was flush now and again, but I tried to set all the different flush options and I have also tried to manually flush the page. Nothing of this made any difference except for displaying a white page instead of an error page since the header was already sent. Now this suggests to me that the page is being cached internally by the server and then sent to the client. So is there anything I can do about this?
    Regards
    Hertz

    You are building an HTML table with 1500 rows? That is definitely a lot. And I assume you are also putting some styling on that table...
    These days programmers resource to [url en.wikipedia.org/wiki/AJAX]AJAX techniques for things like that.
    You can, for example, send the table data as a list of comma separated values, and then use Javascript on the browser to create a partial table with options to browse through the different pages of data.
    If you need to show all that data in one single page, you can still try sending just the data as CSV or XML and then building the table on the browser via a Javascript loop. Use CSS for the styling so you don't overload the HTML code with styling information.
    Marcos Broc
    Hi,
    I have a bit of a problem. I?m trying to generate
    quite a large JSP file (a table with about 1500
    rows). The problem is that I get an out of memory
    error when the JSP is being generated. And it is
    definitely that the JSP is too large, or rather the
    HTML being generated from it. Now I was under the
    impression that the JSP page was flush now and again,
    but I tried to set all the different flush options
    and I have also tried to manually flush the page.
    Nothing of this made any difference except for
    displaying a white page instead of an error page
    since the header was already sent. Now this suggests
    to me that the page is being cached internally by the
    server and then sent to the client. So is there
    anything I can do about this?
    Regards
    Hertz

  • [N95 8GB]Out of Memory error when opening 15MB pdf...

    What is the file size limit when opening a pdf file? I have a 15MB pdf file that cause 'Out of memory' error when I opened it with the PDF Reader that comes with the phone.
    Thanks.

    Hi,
    It seems that problem is file specific, so could you please share a file with me. I sending you a private message.
    Regards,
    Anoop

  • Indesign cs5 'out of memory' error when using preflight

    I have been regulary getting an 'out of memory' error when i choose to use my bespoke preflight profile.
    I have 4gig of ram and run Indesign CS5 on OS 10.6.8.
    Does anyone know a work around?
    As soon as I select from the basic default profile, i get the beach ball from hell for 10mins, then it kindly lets me know that I am out of memory, sends a crash report to Adobe and then asks if I want to relauch. I'm stuck in a vicious circle. I must of sent my 4th crash report now and no feedback from anyone at Adobe.

    I have replaced my preferences, but still the problem persists. I have tried switching my view from typical display to fast display before i selected a profile. I thought this may give me the extra memory I needed to avoid the enevitable crash. I learnt that 2 files were indeed rgb instead of cmyk before it crashed again. So I switched them to cmyk and tried again, selected my bespoke profile, but yet again it crashed. I think the problem lies with the file, not Indesign, as i have tried the same profile on a different file and the program doesn't crash and runs as it should. So if in future I need to use said crashing file again, firstly i will need to try Peter's isolate fix method. Otherwise i'll never be able to progress to successful a pdf.

  • Getting 'Out of memory' error while opening the file. I have tried several versions of Adobe 7.0,9.0,X1. It is creating issue to convert PDF into TIFF. Please provide the solution ASAP

    Hello All,
    I am getting 'Out of memory' error while opening the file. I have tried several versions of Adobe 7.0,9.0,X1.
    Also, it is creating issue to convert PDF into TIFF. Please provide the solution ASAP.

    I am using Adobe reader XI. When i open PDF it gives "OUT of memory" error after scrolling PDF gives another alert "Insufficient data for an image". after clicking both alerts it loads full data of PDF. It is not happening with all PDFs. couple of PDFs are facing this issue. Because of this error my software is not able to print these PDFS into TIFF. My OS in window7*64. I tried it on win2012R2 and XP. Same issue is generating there.
    It has become critical issue for my production.

  • Out of Memory Error when Saving SequenceFile generated using API

    I am currently developing an application that creates a SequenceFile using .NET 4.0, C# (2010), Windows 7 Pro x64, Dual Core 2.5 GHz, 3.0Gb Memory.  Using the API EngineClass, I create a complete SequenceFile based on user/file inputs and then save it to disk.  When I'm done, I save using SequenceFile.Save() and then release using ReleaseSequenceFileEx().
    For smaller files, there is no problem.  For larger files, I sporadically get the following error:
    System.Runtime.InteropServices.COMException (0xFFFFBD98): Out of memory.
    When the file does save, it is ~2.8Mb.  Breaking the file up is not an option.  I am not looping, this is one shot.  I've tried shutting down VS2010 and restarting the computer.  It is still inconsistent on saving.
    I also had to set the Embed Interop Types to False per guidance from NI.  They said that it is unstable in 4.0.
    Questions
    1. Is there a file limit size to the Save() API function?
    2. Is there a workaround to using the Save()?
    Best Regards
    Solved!
    Go to Solution.

    I am using C#.NET.  I called them in my save method immediately prior to the actual test save sequence.
    GC.Collect();
    GC.WaitForPendingFinalizers();
    GC.Collect();
    GC.WaitForPendingFinalizers();
    try
           CurrentSequenceFile.Save(outputPath);
           catch (Exceptionerror)
           FireLogEvent(ParserLogType.Parsing_Error, string.Format("{0}", error));
    Factory.TSEngine.ReleaseSequenceFileEx(CurrentSequenceFile, ReleaseSeqFileOptions.ReleaseSeqFile_UnloadFile);
    Hope this helps

  • "Out of memory" error when receiving "Bytes" via Webservices.

    When I receive Byte content via Webservices over 3.5MB, I get Out of memory error.
    Does Everybody know how to solve this problem?
    I'm using TomCat 4.1 [and 5.0.28 too] with Axis.

    Then you are not using them correctly. If I set up my stream to read into 2K buffers, then my memory overhead should not significantly exceed 2K when reading the file. My bet is you are reading everything into memory and then writing it.
    Post some code. Not the whole application or even the whole class, just the part performing I/O. And please use the 'code' formatting button above when doing so. Then, perhaps, we can help diagnose your issue.
    - Saish

  • Out of Memory Errors when Exporting Slideshows

    Hi -
    I've been using LR for sometime now. Today, for the first time ever, I attempted to export a slideshow as a .pdf file, 90% quality setting, 48 slides total. LR created a 41MB file. However, when I played it back, there were many slides which only showed an empty frame with the message "out of memory error" upside down in the lower left corner of the frame. The same slides displayed the message each time I played it, so I'm sure the problem stemmed from creating the file, not showing it.
    I have a core 2 duo machine with 4GB Ram and no other applications running at the time I exported the slideshow.
    Any thoughts on this? BTW, I'm running version 1.2
    Thanks!
    E-

    Try restarting and reexporting. OOM errors have little to do with how much memory you have or how much is available.

  • Out of memory errors when trying to sync local and remote sites

    We cannot get our remote and local sites synchronized for the first time. We have a huge site and we get out or memory errors or the synch just doesn't work. Any solutions or insights?
    Our site has 9,000+ HTML files and accompanying images, pdfs, etc. When we try to synch the remote to the local for the first time, it just will not happen. Every once in a while someone gets lucky, but for the most part, we either get an "out of memory" error or the synch just doesn't work, and it doesn't tell us anything. It just stops responding. HELP!

    Hi dmooresatx,
    I am not aware of the limitations of the file sizes that are allowed for a successful sync operation. If you are using a purchased version of Dreamweaver CC, send me your Adobe ID along with your contact details (phone, email number). Click on my picture and use the message option. If you are using a team license, get these details from your administrator.
    Thanks,
    Preran

  • Get out of memory error when linking Frame doc to RoboHelp project

    I just upgraded to TC Suite 5.
    Now I'm trying to link FrameMaker files to my RoboHelp project. The FrameMaker files are from a document that was done in Frame 9 (TC Suite 3.5) and now has been converted to Frame 11. It's working fine in Frame 11. No errors when opening any of the files in Frame 11.
    I created a new project in RH 11 and tried linking the FM files to it. Some work, but some bring on an Out of Memory message. Interestingly, one file that is 68MB worked fine, one file that's only 58MB brings that message.
    I'm going to try using an Externalize Graphics addon that worked well with the old Frame--I can only hope it works with FM 11 now.
    Does anyone have any ideas why this is happening, and any solutions besides externalizing the graphics and trying to reduce the file size?
    Thanks.

    I was finally able to get the files to work by opening and saving in FrameMaker numerous times.
    Re: your questions. . . I did not try importing directly. I linked and then generated. Yes, the linking did work with RH9 from the TCS 3.5.

  • "Out of memory" message when rendering - MP4 files imported

    I imported MP4 files into FCP7 and I get an "out of memory" message when I try to render a sequence. I don't think it's the size of the file because I often work with much larger files without any problem. I even saved the MP4 to MOV files with QT and imported those, and still it gives me the  "out of memory" message. VERY ANNOYING.  There seems to be something in those original MP4 files that FCP doesn't like. Any idea how to get around this?

    Not to be rude or insult intelligence, but What format is the timeline in. Not the video files (It's great that you put them to PR422, but if your timeline's native format is something other than PR422, Final cut is going to try to render files in that format... )
    And Again please define "Huge"... How big of a hard drive are you using, and how much RAM space do you have? I checked your profile for any further information, but found none. these are key things to helping figure out your problem.

Maybe you are looking for

  • "No bootable device" -- insert book disk and press any key

    Hello, I am having a little problem with my new iMac. I search in google and I don't seem to get an answer...or maybe I just don't know which one I should do... This is what happen. When I turn on my iMac it goes to a white screen, then to a black sc

  • Checkbox macros not working in Microsoft Word 2003

    Using Visual Basic Editor, I made an Employee Performance Review form that automatically calculates an employee's total score as check boxes are checked. Everything worked perfectly when I made it, but since saving and closing the program, the check

  • Adding a mask in a datagrid

    The first code block is an item renderer that is used to place a button into a datagrid. The second code block is a class that adds some interactivity to the button. The second code block also contains two commented out lines where I have attempted t

  • Archivelink - details connection

    Hi, We have automatically uploaded attachments with ArchiveLink to an Case Management object. The attachments can be seen with the generic object services. All works fine. I would like to have the details of the attachment. In particular i would like

  • Trying to upgrade from 10.5.8 to snow leopard. It says I cannot use my current version of Installer

    It says I cannot use my current version of Installer. How do I get around this. Auto updates are current.