Uploading large files from applet to servlet throws out of memory error

Question

I have a java applet that needs to upload files from a client machine
to a web server using a servlet. the problem i am having is that in
the current scheme, files larger than 17-20MB throw an out of memory
error. is there any way we can get around this problem? i will post
the client and server side code for reference.
Client Side Code:
import java.io.*;
import java.net.*;
// this class is a client that enables transfer of files from client
// to server. This client connects to a servlet running on the server
// and transmits the file.
public class fileTransferClient
private static final String FILENAME_HEADER = "fileName";
private static final String FILELASTMOD_HEADER = "fileLastMod";
// this method transfers the prescribed file to the server.
// if the destination directory is "", it transfers the file to
"d:\\".
//11-21-02 Changes : This method now has a new parameter that
references the item
//that is being transferred in the import list.
public static String transferFile(String srcFileName, String
destFileName,
String destDir, int itemID)
if (destDir.equals(""))
destDir = "E:\\FTP\\incoming\\";
// get the fully qualified filename and the mere filename.
String fqfn = srcFileName;
String fname =
fqfn.substring(fqfn.lastIndexOf(File.separator)+1);
try
//importTable importer = jbInit.getImportTable();
// create the file to be uploaded and a connection to
servlet.
File fileToUpload = new File(fqfn);
long fileSize = fileToUpload.length();
// get last mod of this file.
// The last mod is sent to the servlet as a header.
long lastMod = fileToUpload.lastModified();
String strLastMod = String.valueOf(lastMod);
URL serverURL = new URL(webadminApplet.strServletURL);
URLConnection serverCon = serverURL.openConnection();
// a bunch of connection setup related things.
serverCon.setDoInput(true);
serverCon.setDoOutput(true);
// Don't use a cached version of URL connection.
serverCon.setUseCaches (false);
serverCon.setDefaultUseCaches (false);
// set headers and their values.
serverCon.setRequestProperty("Content-Type",
"application/octet-stream");
serverCon.setRequestProperty("Content-Length",
Long.toString(fileToUpload.length()));
serverCon.setRequestProperty(FILENAME_HEADER, destDir +
destFileName);
serverCon.setRequestProperty(FILELASTMOD_HEADER, strLastMod);
if (webadminApplet.DEBUG) System.out.println("Connection with
FTP server established");
// create file stream and write stream to write file data.
FileInputStream fis = new FileInputStream(fileToUpload);
OutputStream os = serverCon.getOutputStream();
try
// transfer the file in 4K chunks.
byte[] buffer = new byte[4096];
long byteCnt = 0;
//long percent = 0;
int newPercent = 0;
int oldPercent = 0;
while (true)
int bytes = fis.read(buffer);
byteCnt += bytes;
//11-21-02 :
//If itemID is greater than -1 this is an import file
transfer
//otherwise this is a header graphic file transfer.
if (itemID > -1)
newPercent = (int) ((double) byteCnt/ (double)
fileSize * 100.0);
int diff = newPercent - oldPercent;
if (newPercent == 0 || diff >= 20)
oldPercent = newPercent;
jbInit.getImportTable().displayFileTransferStatus
(itemID,
newPercent);
if (bytes < 0) break;
os.write(buffer, 0, bytes);
os.flush();
if (webadminApplet.DEBUG) System.out.println("No of bytes
sent: " + byteCnt);
finally
// close related streams.
os.close();
fis.close();
if (webadminApplet.DEBUG) System.out.println("File
Transmission complete");
// find out what the servlet has got to say in response.
BufferedReader reader = new BufferedReader(
new
InputStreamReader(serverCon.getInputStream()));
try
String line;
while ((line = reader.readLine()) != null)
if (webadminApplet.DEBUG) System.out.println(line);
finally
// close the reader stream from servlet.
reader.close();
} // end of the big try block.
catch (Exception e)
System.out.println("Exception during file transfer:\n" + e);
e.printStackTrace();
return("FTP failed. See Java Console for Errors.");
} // end of catch block.
return("File: " + fname + " successfully transferred.");
} // end of method transferFile().
} // end of class fileTransferClient
Server side code:
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
import java.util.*;
import java.net.*;
// This servlet class acts as an FTP server to enable transfer of
files
// from client side.
public class FtpServerServlet extends HttpServlet
String ftpDir = "D:\\pub\\FTP\\";
private static final String FILENAME_HEADER = "fileName";
private static final String FILELASTMOD_HEADER = "fileLastMod";
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException,
IOException
doPost(req, resp);
public void doPost(HttpServletRequest req, HttpServletResponse
resp)
throws ServletException,
IOException
// ### for now enable overwrite by default.
boolean overwrite = true;
// get the fileName for this transmission.
String fileName = req.getHeader(FILENAME_HEADER);
// also get the last mod of this file.
String strLastMod = req.getHeader(FILELASTMOD_HEADER);
String message = "Filename: " + fileName + " saved
successfully.";
int status = HttpServletResponse.SC_OK;
System.out.println("fileName from client: " + fileName);
// if filename is not specified, complain.
if (fileName == null)
message = "Filename not specified";
status = HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
else
// open the file stream for the file about to be transferred.
File uploadedFile = new File(fileName);
// check if file already exists - and overwrite if necessary.
if (uploadedFile.exists())
if (overwrite)
// delete the file.
uploadedFile.delete();
// ensure the directory is writable - and a new file may be
created.
if (!uploadedFile.createNewFile())
message = "Unable to create file on server. FTP failed.";
status = HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
else
// get the necessary streams for file creation.
FileOutputStream fos = new FileOutputStream(uploadedFile);
InputStream is = req.getInputStream();
try
// create a buffer. 4K!
byte[] buffer = new byte[4096];
// read from input stream and write to file stream.
int byteCnt = 0;
while (true)
int bytes = is.read(buffer);
if (bytes < 0) break;
byteCnt += bytes;
// System.out.println(buffer);
fos.write(buffer, 0, bytes);
// flush the stream.
fos.flush();
} // end of try block.
finally
is.close();
fos.close();
// set last mod date for this file.
uploadedFile.setLastModified((new
Long(strLastMod)).longValue());
} // end of finally block.
} // end - the new file may be created on server.
} // end - we have a valid filename.
// set response headers.
resp.setContentType("text/plain");
resp.setStatus(status);
if (status != HttpServletResponse.SC_OK)
getServletContext().log("ERROR: " + message);
// get output stream.
PrintWriter out = resp.getWriter();
out.println(message);
} // end of doPost().
} // end of class FtpServerServlet

Answer 1

OK - the problem you describe is definitely what's giving you grief.
The workaround is to use a socket connection and send your own request headers, with the content length filled in. You may have to multi-part mime encode the stream on its way out as well (I'm not about that...).
You can use the following:
http://porsche.cis.udel.edu:8080/cis479/lectures/slides-04/slide-02.html
on your server to get a feel for the format that the request headers need to take.
- Kevin
I get the out of Memory Error on the client side. I
was told that this might be a bug in the URLConnection
class implementation that basically it wont know the
content length until all the data has been written to
the output stream, so it uses an in memory buffer to
store the data which basically causes memory issues..
do you think there might be a workaround of any kind..
or maybe a way that the buffer might be flushed after
a certain size of file has been uploaded.. ?? do you
have any ideas?