File splitter won't accept some text files

Question

sorry, this is in the wrong place is there any way i can move it to general programming?
i'm making a file splitter to split large text files into smaller parts. so far everything works as far as test files that i make, however, sometimes when i copy and paste text off the internet, for example, the splitter does not import any of the text as if it can't recognize it. what might i be able to do to get it to recognize this kind of text. the files were saved as ANSI encoding, if it matters.
import java.util.*;
import java.lang.*;
import java.io.*;
public class FileSplitter
    private File inFile;
    private Scanner scan;
    private String path;
    private String filename;
    private String ext;
    private File outFile;
    private String allData = "";
    //max chars in the file
    private final int FILE_LIMIT = 3500;
    public FileSplitter(String path, String name)
        try
            this.path = path;
            int dot = name.lastIndexOf('.');
            this.filename = name.substring(0,dot);
            this.ext = name.substring(dot,name.length());
            this.inFile = new File(path+name);
            this.scan = new Scanner(this.inFile);     
        catch(FileNotFoundException e)
            e.printStackTrace();
            System.out.println("File not found at:");
            System.out.println(path+filename);
    public void split()
        while(scan.hasNextLine())
                allData = allData.concat(scan.nextLine());
                //maybe input an "allData = allData.concat(\n);"
        int x = 0; // file portion sequence number
        int currIndex = 0;  // current index in "scan"
        String writeToFile = "";
        System.out.println(allData.length());
        while(currIndex <= allData.length())
            x++;
            System.out.println("Making file "+x);
            try
                BufferedWriter out = new BufferedWriter(new FileWriter(path + filename + "-" + x + ext));
                //System.out.println(currIndex);
                //System.out.println(allData.length());
                if(currIndex+FILE_LIMIT > allData.length())
                    writeToFile = allData.substring(currIndex, allData.length());
                else
                    writeToFile = allData.substring(currIndex, currIndex+FILE_LIMIT);
                out.write(writeToFile);
                out.close();
            catch (IOException e)
                e.printStackTrace();
                System.out.println("File write error.");
            currIndex = currIndex + FILE_LIMIT;
    public static void main(String[] args)
        String path = "C:\\split\\";
        String filename = "file.txt";
        FileSplitter x = new FileSplitter(path, filename);
        x.split();
        System.out.println("Done.");
}the program works perfectly for most files, but certain ones, and i'm not sure how they differ, do not get read at all. Is there a kind of text that will not get read? all files were saved with notepad.
no error is thrown on these, but the length() function returns 0.
i've also tried using scan.next() and nextByte() as well, to no success, with appropriate hasNext...() calls
if anyone's wondering, this is to make files easily accessible to ipods with 4kb text file limits on what they can show.
i am now thinking of accepting web pages as raw source and parsing out <...>

Answer 1

Instead of using a Scanner why not use a BufferedReader which will handle files that use either '\n' or '\r' as line terminators. After all, all you are doing is reading the file in line by line and storing into a string with out the newline character.