How to copy large files - fast

Question

Hi, here is an I/O challenge: I need to copy large binary files under windows in one of my apps. File size can be above 2 GB. The copying should be as fast as possible. What would be the fastest way to do this?
Here is what I've tried so far:
1) Use FileInputStream/FileOutputStream with a byte buffer (see copy1 below)
2) Use BufferedInputStream/BufferedOutputStream (see copy2 below)
3) Use java.nio.FileChannel#transferTo (see copy3 below)
4) Use a shell copy command (see copy4 below)
Testing the four methods with a 500 MB file I get
1) 525271525 Byte in 47141 ms (11142 kByte/s)
2) 525271525 Byte in 68969 ms (7616 kByte/s)
3) 525271525 Byte in 38765 ms (13550 kByte/s)
4) 525271525 Byte in 37984 ms (13828 kByte/s)
This is on a Pentium 4, 2.4 GHz, 512 MB RAM, Windows XP with JDK 1.4.2.
Hence, my favorite method would be 3), since it is more portable than 4).
However, with java.nio.FileChannel#transferTo I get exceptions for files larger than 2 GB. The exception is
java.io.IOException: Falscher Parameter
  at sun.nio.ch.FileChannelImpl.map0(Native Method)
  at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:673)
  at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:421)
  at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:490)leading me to suspect that the long values used by Java to specify the file length are not supported by the native code, and that they may be downcast to a 32-bit signed number. (If anyone could shed some more light on this, I'd be grateful!) So, is there a better way than using 4), a shell copy command?
Following is the code I used to for my test:import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.FileChannel;
/** Class for testing different ways to copy files */
public class TestCopy
  /** i/o buffer size */
  public static final int BUF_SIZE = 1024 * 1024;
  /** Copy method, using FileInputStream and byte buffer */
  void copy1(File source, File target) throws IOException
    FileInputStream in = new FileInputStream(source);
    FileOutputStream out = new FileOutputStream(target);
    byte[] buf = new byte[BUF_SIZE];
    int i;
    while ((i = in.read(buf, 0, BUF_SIZE)) != -1)
      out.write(buf, 0, i);
    in.close();
    out.close();
  /** Copy method, using BufferedInputStream */
  void copy2(File source, File target) throws IOException
    BufferedInputStream in =
      new BufferedInputStream(new FileInputStream(source), BUF_SIZE);
    BufferedOutputStream out =
      new BufferedOutputStream(new FileOutputStream(target), BUF_SIZE);
    int i;
    while ((i = in.read()) != -1)
      out.write(i);
    in.close();
    out.close();
  /** Copy method, using FileChannel#transferTo */
  void copy3(File source, File target) throws IOException
    FileChannel in = new FileInputStream(source).getChannel();
    FileChannel out = new FileOutputStream(target).getChannel();
    in.transferTo(0, in.size(), out);
    in.close();
    out.close();
  /** Copy method, using command shell copy */
  void copy4(File source, File target) throws IOException
    Process p =
      Runtime.getRuntime().exec(
        new String[] {
          "cmd",
          "/C",
          "copy",
          "/Y",
          "/B",
          source.getAbsolutePath(),
          target.getAbsolutePath()});
    int j;
    try
      j = p.waitFor();
      if (j != 0)
        throw new IOException("copy returned exit code " + j);
    catch (InterruptedException e1)
  public static void main(String[] args)
    if (args.length != 2)
      System.out.println("usage: java TestCopy <sourceFile> <targetFile>");
      return;
    File source = new File(args[0]);
    File target = new File(args[1]);
    if (!source.exists())
      System.out.println("File not found: " + args[0]);
      return;
    TestCopy tc = new TestCopy();
    try
      long time = System.currentTimeMillis();
      tc.copy1(source, target);
      time = System.currentTimeMillis() - time;
      System.out.println(
        "Copy1: "
          + source.length()
          + " Byte in "
          + time
          + " ms ("
          + source.length() / time
          + " kByte/s)");
      time = System.currentTimeMillis();
      tc.copy2(source, target);
      time = System.currentTimeMillis() - time;
      System.out.println(
        "Copy2: "
          + source.length()
          + " Byte in "
          + time
          + " ms("
          + source.length() / time
          + " kByte/s)");
      time = System.currentTimeMillis();
      tc.copy3(source, target);
      time = System.currentTimeMillis() - time;
      System.out.println(
        "Copy3: "
          + source.length()
          + " Byte in "
          + time
          + " ms("
          + source.length() / time
          + " kByte/s)");
      time = System.currentTimeMillis();
      tc.copy4(source, target);
      time = System.currentTimeMillis() - time;
      System.out.println(
        "Copy4: "
          + source.length()
          + " Byte in "
          + time
          + " ms("
          + source.length() / time
          + " kByte/s)");
    catch (IOException e)
      e.printStackTrace();
}Cheers, HJK

Answer 1

Hmm, I guess I have to agree with jschell. Runtime.exec("cmd /c copy <file1> <file2>"); is the way to go.
The FileChannell#transferTo method in a loop, as asjf suggested, is not much slower, but it's a workaround. Using JNI to call the system command, is probably not worth the effort. I can't see how it will be faster than calling the shell copy command.
Thank you all for your help! Cheers, HJK