ObjectInputStream with large files?

I write two arrays to their own separate files using:
FileOutputStream fos = new FileOutputStream("C:/location");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(array);
oos.close(); One file is 2MB and the other is 600MB.
When attempting to read them using:
InputStream fis = classname.class.getResourceAsStream("resources/filename");
ObjectInputStream ois = new ObjectInputStream(fis);
array = (double[][][]) ois.readObject();
ois.close(); The smaller array is loaded normally, but the larger one results in a java.io.EOFException.
The code for reading and writing is exactly the same for each array with the exception of the array name.
Why is this exception being thrown, and how might I correct this?

It seems to work fine. Some streams detect end-of-file just by catching EOFException. Try to look here.
Edited by: elOpalo on Dec 23, 2009 2:19 PM

Similar Messages

Photoshop CS6 keeps freezing when I work with large files

I've had problems with Photoshop CS6 freezing on me and giving me RAM and Scratch Disk alerts/warnings ever since I upgraded to Windows 8. This usually only happens when I work with large files, however once I work with a large file, I can't seem to work with any file at all that day. Today however I have received my first error in which Photoshop says that it has stopped working. I thought that if I post this event info about the error, it might be of some help to someone to try to help me. The log info is as follows:
General info
Faulting application name: Photoshop.exe, version: 13.1.2.0, time stamp: 0x50e86403
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16451, time stamp: 0x50988950
Exception code: 0xe06d7363
Fault offset: 0x00014b32
Faulting process id: 0x1834
Faulting application start time: 0x01ce6664ee6acc59
Faulting application path: C:\Program Files (x86)\Adobe\Adobe Photoshop CS6\Photoshop.exe
Faulting module path: C:\Windows\SYSTEM32\KERNELBASE.dll
Report Id: 2e5de768-d259-11e2-be86-742f68828cd0
Faulting package full name:
Faulting package-relative application ID:
I really hope to hear from someone soon, my job requires me to work with Photoshop every day and I run into errors and bugs almost constantly and all of the help I've received so far from people in my office doesn't seem to make much difference at all. I'll be checking in regularly, so if you need any further details or need me to elaborate on anything, I should be able to get back to you fairly quickly.
Thank you.

Here you go Conroy. These are probably a mess after various attempts at getting help.

Wpg_docload fails with "large" files

Hi people,
I have an application that allows the user to query and download files stored in an external application server that exposes its functionality via webservices. There's a lot of overhead involved:
1. The user queries the file from the application and gets a link that allows her to download the file. She clicks on it.
2. Oracle submits a request to the webservice and gets a XML response back. One of the elements of the XML response is an embedded XML document itself, and one of its elements is the file, encoded in base64.
3. The embedded XML document is extracted from the response, and the contents of the file are stored into a CLOB.
4. The CLOB is converted into a BLOB.
5. The BLOB is pushed to the client.
Problem is, it only works with "small" files, less than 50 KB. With "large" files (more than 50 KB), the user clicks on the download link and about one second later, gets a
The requested URL /apex/SCHEMA.GET_FILE was not found on this serverWhen I run the webservice outside Oracle, it works fine. I suppose it has to do with PGA/SGA tuning.
It looks a lot like the problem described at this Ask Tom question.
Here's my slightly modified code (XMLRPC_API is based on Jason Straub's excellent [Flexible Web Service API|http://jastraub.blogspot.com/2008/06/flexible-web-service-api.html]):
CREATE OR REPLACE PROCEDURE get_file ( p_file_id IN NUMBER )
IS
    l_url                  VARCHAR2( 255 );
    l_envelope             CLOB;
    l_xml                  XMLTYPE;
    l_xml_cooked           XMLTYPE;
    l_val                  CLOB;
    l_length               NUMBER;
    l_filename             VARCHAR2( 2000 );
    l_filename_with_path   VARCHAR2( 2000 );
    l_file_blob            BLOB;
BEGIN
    SELECT FILENAME, FILENAME_WITH_PATH
      INTO l_filename, l_filename_with_path
      FROM MY_FILES
     WHERE FILE_ID = p_file_id;
    l_envelope := q'!<?xml version="1.0"?>!';
    l_envelope := l_envelope || '<methodCall>';
    l_envelope := l_envelope || '<methodName>getfile</methodName>';
    l_envelope := l_envelope || '<params>';
    l_envelope := l_envelope || '<param>';
    l_envelope := l_envelope || '<value><string>' || l_filename_with_path || '</string></value>';
    l_envelope := l_envelope || '</param>';
    l_envelope := l_envelope || '</params>';
    l_envelope := l_envelope || '</methodCall>';
    l_url := 'http://127.0.0.1/ws/xmlrpc_server.php';
    -- Download XML response from webservice. The file content is in an embedded XML document encoded in base64
    l_xml := XMLRPC_API.make_request( p_url      => l_url,
                                      p_envelope => l_envelope );
    -- Extract the embedded XML document from the XML response into a CLOB
    l_val := DBMS_XMLGEN.convert( l_xml.extract('/methodResponse/params/param/value/string/text()').getclobval(), 1 );
    -- Make a XML document out of the extracted CLOB
    l_xml := xmltype.createxml( l_val );
    -- Get the actual content of the file from the XML
    l_val := DBMS_XMLGEN.convert( l_xml.extract('/downloadResult/contents/text()').getclobval(), 1 );
    -- Convert from CLOB to BLOB
    l_file_blob := XMLRPC_API.clobbase642blob( l_val );
    -- Figure out how big the file is
    l_length    := DBMS_LOB.getlength( l_file_blob );
    -- Push the file to the client
    owa_util.mime_header( 'application/octet', FALSE );
    htp.p( 'Content-length: ' || l_length );
    htp.p( 'Content-Disposition: attachment;filename="' || l_filename || '"' );
    owa_util.http_header_close;
    wpg_docload.download_file( l_file_blob );
END get_file;
/I'm running XE, PGA is 200 MB, SGA is 800 MB. Any ideas?
Regards,
Georger

Script: http://www.indesignsecrets.com/downloads/MultiPageImporter2.5JJB.jsx.zip
It works great for files upto ~400 pages, when have more pages than that, is when I get the crash at around page 332 .
Thanks

Working with Large files in Photoshop 10

I am taking pictures with a 4X5 large format film camera and scanning them at 3,000 DPI, which is creating extremely large files. My goal is to take them into Photoshop Elements 10 to cleanup, edit, merge photos together and so on. The cleanup tools don't seem to work that well on large files. My end result is to be able to send these pictures out to be printed at large sizes up to 40X60. How can I work in this environment and get the best print results?

You will need to work with 8bit files to get the benefit of all the editing tools in Elements.
I would suggest resizing at resolution of 300ppi although you can use much lower resolutions for really large prints that will be viewed from a distance e.g. hung on a gallery wall.
That should give you an image size of 12,000 x 18,000 pixels if the original aspect ratio is 2:3
Use the top menu:
Image >> Resize >> Image Size

Probs with large files in adapter engine

Hi all,
in a scenario I'm currently working on, I have to process files (from file adapter). These files have 1KB up to 15MB. So I set up the tuning parameters, that files with more than 2MB are queued in this special queue for large messages. This works fine. small messages are processed directly and the larger messages are queued and processed in sequence by the integration engine.
My problem is now on the point where the integration engine sends this large files to the adapter engine. There are always 4 messages in parallel sent to the adapter engine and this slows down the whole system due to an additional self-developed module.
So my question is: Can I restrict the sending of the messages to only 1 at a time from integration engine to adapter engine?
The time of processing is not important. I think I can handle this with EOIO, but is there an other way? Perhaps depending on the queue?
Thanks
Olli

Hi William,
thx for your reply.
Yes I think it is the easiest way. But the problem is, if a message runs to a failure the others will not be processed until the failure is fixed or processing is canceled.
So I hoped to find a solution where I can restrict the sending to the adapter engine to one message at a time in order to not affect the processing of the other messages as in EOIO.
Regards
Olli

Bug report - Finder crashes in Cover Flow mode with large files

I just came back from the Apple store and was told that I have discovered another bug in Leopard. When in Cover Flow view in Finder and trying to browse directories with large ( multiple GB) files, Finder continually crashes and reboots, oftentimes with 2 new FInder windows.
I created a new user on my MBP to remove the effect of any preferences and the problem repeated itself.
Come up Apple... get on top of these bugs and issue some patches...
Not the kind of new OS I expected from a top notch company like Apple.

Ah... that'll be it then, they are 512 x 512. well i guess that bugs been hanging around for quite some time now, anyone got any ideas if anyone's trying to fix it? i guess making 128 x 128 icons wouldn't be the end of the world but it does seem like a step backwards.
one thing that still confuses me... an icon file contains icons of all sizes, why does cover flow not select the right size icon to display?
thanks for that info V.K., much obliged.
regards, Pablo.

Dealing with large files, again

Ok, so I've looked into using BufferedReaders and can't get my head round them; or more specifically, I can't work out how to apply them to my code.
I have inserted a section of my code below, and want to change it so that I can read in large files (of over 5 million lines of text). I am reading the data into different arrays and then processing them. Obvioulsy, when reading in such large files, my arrays are filling up and failing.
Can anyone suggest how to read the file into a buffer, deal with a set amount of data, process it, empty the arrays, then read in the next lot?
Any ideas?
void readV2(){
     String line;
     int i=0,lineNo=0;
        try {
          //Create input stream
          FileReader fr = new FileReader(inputFile);
             BufferedReader buff = new BufferedReader(fr);
            while((line = buff.readLine()) != null) {
          if(line.substring(0,2).equals("V2")){
                 lineNo = lineNo+1;
          IL[i] = Integer.parseInt(line.substring(8,15).trim());
                //Other processing here
                 NoOfPairs = NoOfPairs+1;
                 }//end if
                 else{
                  break;
        }//end while
        buff.close();
        fr.close();
        }//end try
        catch (IOException e) {
        log.append("IOException error in readESSOV2XY" + e + newline);
        proceed=false;
        }//end catch IOException
        catch (ArrayIndexOutOfBoundsException e) {
               arrayIndexOutOfBoundsError(lineNo);
     }//end catch ArrayIndexOutOfBoundsException
     catch (StringIndexOutOfBoundsException e) {
          stringIndexOutOfBoundsError(e.getMessage(),lineNo);
}//end V2Many thanks for any help!
Tim

Yeah, ok, so that seems simple enough.
But once I have read part of the file into my program,
I need to call another method to deal with the data I
have read in and write it out to an output file.
How do I get my file reader to "remember" where I am
up to in the file I'm reading?
An obvious way, but possibly not too good technically,
would be to set a counter and when I go back to the
fiel reader, skip that number of lines in the inpuit
file.
This just doesn't seem too efficient, which is
critical when it comes to dealing with such large
files (i.e. several million lines long)I think you might need to change the way you are thinking about streams. The objective of a stream is to read and process data at the same time.
I would recommend that you re-think your algorithm : instead of reading the whole file, then doing your processing - think about how you could read a line and process a line, then read the next line, etc...
By working on just the pieces of data that you have just read, you can process huge files with almost no memory requirements.
As a rule of thumb, if you ever find yourself creating huge arrays to hold data from a file, chances are pretty good that there is a better way. Sometimes you need to buffer things, but very rarely do you need to buffer such huge pieces.
- K

Mounting CIFS on MAC with large file support

Dear All,
We are having issues copying large files ( > 3.5 GB) from MAC to a CIFS Share (smb mounted on MAC) whereby the copy fails if files are larger than 3.5 GB in size and hence I was wondering if there is any special way to mount CIFS Shares (special option in the mount_smbfs command perhaps ) to support large file transfer?
Currently we mount the share using the command below
mount_smbfs //user@server/<share> /destinationdir_onMAC

If you haven't already, I would suggest trying an evaluation of DAVE from Thursby Software. The eval is free, fully functional, and supported.
DAVE is able to handle large file transfer without interruption or data loss when connecting to WIndows shared folders. If it turns out that it doesn't work as well as you like, you can easily remove it with the uninstaller.
(And yes, I work for Thursby, and have supported DAVE since 1998)

Adobe Illustrator crashes with large files

when i open illustrator to view or edit a larger file i get
once i hit ok, this is what i get
this particular file is 588MB and it is an iAI file
the computer im running this on is spec out to the following
Win7 Ult 64bit
I7-3770K cpu
32 gb ram
WD 500 GB Veloci Raptor
samsung ssd 840 evo 120 gb scratch drive
paging file set up as min 33068 max 98304
i tried with and with out extra scratch disk, with paging file set and with paging file set to system managed.
i have uninstalled ai and re installed same response

A 588 MB AI File? From where? CS5 is 32bit and expanding such a large file may simply go way beyond the measly 3GB you have in AI plus if it was generated in another program there may be al lsorts of otehr issues at play. This may be a hopeless cause and require you to at least instal lCS6 or CC 64bit as a trial...
Mylenium

DW MX 2004 Slow with large files?

I'm using DW MX 2004 with a static website with a few
thousand files.
It's very slow when opening multiple files at the same time
or
opening a large html file.
But DW4 is fine, nice and quick compared with DW MX 2004.
Is there any way to help DW MX 2004 work better with these
files?
Many thanks, Craig.

Many thanks.
But already running the 7.0.1 update.
"Randy Edmunds" <[email protected]> wrote in
message
news:eggcl0$6ke$[email protected]..
> Be sure that you've installed the 7.0.1 updater. There
were a few
> performance fixes, especially on the Mac, that may help
your workflow.
>
> HTH,
> Randy
>
>
>> I'm using DW MX 2004 with a static website with a
few thousand files.
>> It's very slow when opening multiple files at the
same time or
>> opening a large html file.
>> But DW4 is fine, nice and quick compared with DW MX
2004.
>>
>> Is there any way to help DW MX 2004 work better with
these files?

IdcApache2Auth.so Compiled With Large File Support

Hi, I'm installing UCM 10g on solaris 64 Bit plattform and Apache 2.0.63 , everything went fine until I update configuration in the httpd.conf file. When I query server status it seems to be ok:
+./idcserver_query+
Success checking Content Server idc status. Status: Running
but in the apache error_log and I found the next error description:
Content Server Apache filter detected a bad request_rec structure. This is possibly a problem with LFS (large file support). Bad request_rec: uri=NULL;
Sizing information:
sizeof(*r): 392
+[int]sizeof(r->chunked): 4+
+[apr_off_t]sizeof(r->clength): 4+
+[unsigned]sizeof(r->expecting_100): 4+
If the above size for r->clength is equal to 4, then this module
was compiled without LFS, which is the default on Apache 1.3 and 2.0.
Most likely, Apache was compiled with LFS, this has been seen with some
stock builds of Apache. Please contact Support to obtain an alternate
build of this module.
When I search at My Oracle Support for suggestions about how to solve my problem I found a thread which basically says that Oracle ECM support team could give me a copy IdcApache2Auth.so compiled with LFS.
What do you suggest me?
Should I ask for ECM support team help? (If yes please tell me How can I do it)
or should I update the apache web server to version 2.2 and use IdcApache22Auth.so wich is compiled with LFS?
Thanks in advance, I hope you can help me.

Hi ,
Easiest approach would be to use Apache2.2 and the corresponding IdcApache22Auth.so file .
Thanks
Srinath

Read parameter error -50 with larger file - please help

I have this line of code: (read sfRef from SourcePosition as data)
It works fine with these 811.2MB files but when I try to read from a 1.62GB file I get parameter error -50. Did I miss something about larger files?

found it!
You can't read larger than 1gb. I had it grab the data in multiple pieces and it works now.

Canon ip5000 freezes with large files while printing via Airport Express

I've discovered a fairly consistent problem when I print from my Powerbook G4 17" to my Canon ip5000 printer. My printer is connected to an Airport Express, which is set as a print server.
When I print large files - say, a full-quality photo, or a web page with a lot of images - the printer freezes about 20% of the way through the document. It will hang, and the print queue just says its still printing, with the status bar at the 20% - 30% range and holding.
If I leave things alone, eventually, the printer will spit out the unfinished print-out. The queue will continue to read "in progress" until I manually delete the print job.
I've been able to repeat this problem with about 80% consistently, although 20% of the time, the print-out works fine (even when printing the same image that failed previously).
Any ideas / assistance is apprecaited.
- David

Same problem here with my Canon ip4200. Still searching for a solution.
I've discovered a fairly consistent problem when I
print from my Powerbook G4 17" to my Canon ip5000
printer. My printer is connected to an Airport
Express, which is set as a print server.
When I print large files - say, a full-quality photo,
or a web page with a lot of images - the printer
freezes about 20% of the way through the document. It
will hang, and the print queue just says its still
printing, with the status bar at the 20% - 30% range
and holding.
If I leave things alone, eventually, the printer will
spit out the unfinished print-out. The queue will
continue to read "in progress" until I manually
delete the print job.
I've been able to repeat this problem with about 80%
consistently, although 20% of the time, the print-out
works fine (even when printing the same image that
failed previously).
Any ideas / assistance is apprecaited.
- David
Mac OS X (10.4.7)

My iPhone 4s will not forward emails with large files, pictures etc attached. How can I fix this?

How do I get my iPhone to forward or send large files? Such as pictures.

Check Settings>Messages>MMS Messaging to see if it's set to "On". Check with your carrier to see that your account is properly provision. Check to see if your carrier is a supported iPhone carrier. Not all features work or work properly on unsupported carriers.
Best of luck.

Two work-arounds for Siena when working with large files

MonaTech pointed out that Project Siena will crash when switching to the desktop or another app. I've also noticed this happening even when the monitor goes to sleep with inactivity. To be clear, this is *developing* in Siena, not after the app
you're working on has been compiled and installed.
The 'non-technical' work-around that has been successful for me is to split the screen before I switch to another app. It's not perfect but at least I can switch to another app (browser, desktop app, etc.) without losing where I'm at.
A more technical work-around is proposed in this thread:http://social.technet.microsoft.com/Forums/en-US/c768be8f-3c85-444e-bb44-6f29abdecee7/project-siena-app-memory-issues-with-large-project-?forum=projectsiena
If you have a version of Visual Studio (that's not Express) it may do the trick for you.
FYI - In the post you'll see that Olivier responded and indicates that this is a known issue.
Thor

Wow - that's what you get for multi-tasking! :)
The link is now fixed in my original post and posted here for convenience:
http://social.technet.microsoft.com/Forums/en-US/c768be8f-3c85-444e-bb44-6f29abdecee7/project-siena-app-memory-issues-with-large-project-?forum=projectsiena
Thor

ObjectInputStream with large files?

Similar Messages

Maybe you are looking for