Tell me Logic for search for duplicate words(or strings) in a large file.

Question

Search for duplicate words (or strings) in a text file containing one word per line. For each word that occurs more than once in the flat file output should be as follows
<word> <number of occurrences> <line numbers in the file where the word occurs>
For example, if the word Hello occurs thrice in a file at lines 100, 178 and 3456 the output should read
Hello, 3, [100, 178, 3456]

Answer 1

Incidentally i wrote similar code some days back. You need to do some modifications to get the exact output you want, but i hope it will be of some help.
One more thing its written using JAVA5
public class Test
     private static final String COLLECTIONS_TEXT = "C:\\Documents and Settings\\amrainder\\Desktop\\Collections.txt";
     public static void main(String[] args) throws IOException
          findDuplicateWords();
     private static void findDuplicateWords() throws IOException
          Collection<String> words = new LinkedHashSet<String>();
          File file = new File(COLLECTIONS_TEXT);
          StreamTokenizer streamTokenizer = new StreamTokenizer(new FileReader(file));
          int token = streamTokenizer.nextToken();
          while(token != StreamTokenizer.TT_EOF)
               if(token == StreamTokenizer.TT_WORD)
                    words.add(streamTokenizer.sval);
               token = streamTokenizer.nextToken();
          System.out.println(words);
}Cheers,
Amrainder