Thesaurus

Question

Dear All,
I have a program that do the following
1. User enters key word(s)
2. The program surfs the website (s) to bring documents that are more relevant.
3.Another program will extract all the important words from those documents. Calculate the term frequency and also the document frequency.
I need to do the following
From those extracted words I need to create a thesaurus table in the following form
Index : Term
0 : Computer
0 : Computers
1 : Data
2 : Database
2 : Databases
Note: Similar Terms should have same index.
Can I use the term frequency/ document frequency to come up with the index value??
Looking for your smart ideas!!!
Cheers,
--Vj

Answer 1

here's a lightweight class to give you a basic thesaurus...
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.util.HashMap;
public class Thesaurus {
  private Map indexWords = new HashMap();
  private Map wordIndexes = new HashMap();
  public void addWord(int index, String word) {
    // Make sure word is not already in Thesaurus
    if(!wordIndexes.containsKey(word)) {
      Integer indexObject = new Integer(index);
      // Place word in word list
      wordIndexes.put(word,indexObject);
      // maintain index of matching words
      List words;
      if(indexWords.containsKey(indexObject)) {
        words = (List) indexWords.get(indexObject);
      else {
        words = new ArrayList();
        indexWords.put(indexObject,words);
      words.add(word);
  public List getWordMatches(String word) {
    // Check if word is in Thesaurus
    if(wordIndexes.containsKey(word)) {
      Integer indexObject = (Integer) wordIndexes.get(word);
      List words = (List) indexWords.get(indexObject);
      List result = new ArrayList(words);
      result.remove(word);
      return result;
    else return new ArrayList();
  public static void main(String[] args) {
    Thesaurus t = new Thesaurus();
    t.addWord(0,"Computer");
    t.addWord(0,"Computers");
    t.addWord(1,"Data");
    t.addWord(2,"Database");
    t.addWord(2,"Databases");
    System.out.println(t.indexWords);
    System.out.println(t.wordIndexes);
    List list = t.getWordMatches("Computer");
    System.out.println(list);
}Hope that helps...