Search Algorithm in Java

Hey Guys,
I have written a crude search algorithm which extracts 4 fields from a database table into the String type of Java. Then this String object is tokenized and then within these tokens I search for the text the user has used.
Once, I faced this problem that one of the String pulled from the DB was so huge that I got the OutOfMemory exception. Simple stating it, the text was so big that my system could not store all that text in one String object.
I resolved this by adding more heap to java by doing:
java -Xmx512M MySearchClass
Now I am afraid that if there is another HUGE text in the DB , I can still run out of memory.
Do I just need to get MORE RAM to my system and allot really huge heap space, such as 4 GB or so , so that if need be, the JVM can use it ?
Suppose I get a server machine with 4 GB RAM, at some stage I'd still have an upper limit on the amount of text I can extract from the DB, right ?
Is there any other way to stream the text from DB and then search ? Remember that I am tokenizing the text in the algorithm, so I think I can't use a buffer, or can I ?
thanks a ton !
-AZ

These are SQL queries. I use SQL queries to pull out the text in the DB and then store it in a String object. The problem is that the text in DB can get so huge that the String object cannot hold it anymore due to memory limits. The JVM gets assigned a heap size and the size of the String object can get really big to exceed the heap size.
I used : java -Xmx512M MySearchClass
That assigned 512MB to the JVM so I haven't encountered the OutOfMemory exception so far. But my guess is that for some other data set, the String can be so big that even 512MB might not be enough.
This search algorithm will be deployed on a server class machine with about 4 GB RAM. So we can give JVM 4GB heap space to use. But its a limitation again , right ?
These text in DB are extracted from PDF files.

Similar Messages

  • Java library for local search algorithms?

    Hi everybody,
    Could anyone please help me with the following?
    I am looking for a Java library with already implemented local search algorithms. Does such a thing exist? Would anyone recommend one?
    I am testing several AI programs and need to select the best parameters for each one (in a relatively large number of experiments). I could write my own search algorithm, of course, but it would be so much easier to use several already implemented algorithms and choose the best one.
    Thanks in advance for any pointers.
    Anna

    Here is an interesting question. Say you build the central control system of a rocket ship using Java, should you then be able to ask questions about rocket ships in Java forums?
    Well you can try, but I wouldn't hold your breath to find another rocket scientist in such a place. You may want to go to a forum where rocket scientists congregate in stead. Just a friendly tip.

  • Creating A Binary search algorithm !!!! URGENT HELP

    hi ..
    i;m currently tryin to create a binary search algorithm ..
    the user should be able to input the size of the array
    and also the key that he would like to find.
    it also has to have to ability to measure the run time of the algorithm.. it how long it too the algorithm to search through the array and find they key..
    i have created 3 classes
    the first class is the binary search class
    which is the mathamatical side of things
    the second class is the Array
    this creates an array selection a random first number
    and then incrementing from there, so that its a sorted array
    the third class is the binary search class
    which is my main class.
    this class should take the users input
    and pass it to the array
    and the binary seach accordingly
    it should also measure the running time, from when it passes the array
    to the binary search class
    i am having a really hard time creating this last class.
    i have created the other 2 successfully
    the codes for the binary search class is as follows
    public class BinarySearch
         static int binSearch(int[] array, int val)
             // setting the start and the end of the array
              int low = 0, high = array.length;
              //While loop
              while(low <= high) {
              // How to find the mid point      
                  int mid = (low + high)/2;
                   // if the mid point is the value return the value
                  if(array[mid] == val) {
                        return mid;
                   // if the value is smaller than the mid point
                   // go search the left half
                   } else if(array[mid] > val) {
                        high = mid - 1;
                   //if the value is greater then the mid point
                   // go search the right half
                   } else if(array[mid] < val) {
                        low = mid + 1;
              // if value is not found return nothing
              return -1;
    }and the code for the Array class is as follows
    import java.util.Random;
    public class RandomSortedArray
        public int[] createArray(int length)
            // construct array of given length
            int[] ary = new int[length];
            // create random number generator
            Random r = new Random();
            // current element of the array; used in the loop below.  Starts at
            // -1 so that the first element of the array CAN be a 0
            int val = -1;
           for( int i = 0; i < length; i++)
                val += 1 + r.nextInt(10);
                ary[i] = val;
            return ary;
    }can some pne please help me create my binarysearchTest class.
    as i mentioned before
    it has to take the users input for the array size
    and the users input for the value that they want to find
    also needs to measure the running time
    thanks for all ur help in advance

    import java.util.*;
    public class AlgorithmTest
         public static void main(String args[])
             long StartTime, EndTime, ElapsedTime;
             System.out.println ("Testing algorithm");
             // Save the time before the algorithm run
             StartTime = System.nanoTime();
             // Run the algorithm
             SelectionSortTest1();
             // Save the time after the run
             EndTime = System.nanoTime();
             // Calculate the difference
             ElapsedTime = EndTime- StartTime;
             // Print it out
             System.out.println("The algorithm took " + ElapsedTime + "nanoseconds to run.");
        }this is the code i managed to work up for measuring the time..
    how would i include it into the main BinarysearchTest Class

  • Data Structures and Algorithms in java book

    Hi guys,
    I want to know a good book which is good for Data Structures and Algorithms in java. I am good at Core java but a beginner for Data Structures in Java. I am a little poor in Data Structures concepts.
    Following are the books I have found on the net. Could you help me the choose the best outta them.
    1. Data Structures and Algorithms in Java - Mitchell Waite
    2. Data Structures in Java - Sandra Anderson
    3. Fundamentals of OOP and Data Structures in Java - Richard Weiner & Lewis J. Pinson
    4. Object Oriented Data Structures Using Java - Nell Dale, Daniel T. Joyce, Chip Weems

    lieni wrote:
    I good data structures book doesn't have to be language-specific.Thx DrLazlo, my speachYes.
    The OP wrote:
    I have access to these books and dont know which one to start with.What I meant is that you shouldn't narrow your search to insist that the book you choose have "Java" in the title.

  • Database Searching Algorithms

    Hi..
    I have to do some research about database searching algorithms.
    Would someone here help me and give a list of that algorithms and which one is the best.
    What is the name of this algorithms which when u type in a key such "a" then it display a list of information that started with "a".

    I guess I have more time than I thought... :)
    Here is a full working example all in one class...
    import javax.swing.*;
    import java.awt.*;
    import java.awt.event.*;
    public class MAIN
         DefaultListModel model_my_list = new DefaultListModel();
        JList my_list = new JList(model_my_list);
         DefaultListModel model_list_of_finds = new DefaultListModel();
        JList list_of_finds = new JList(model_list_of_finds);
         JTextField search_box = new JTextField(20);
         JTextField add_box = new JTextField(20);
         public MAIN()
              JFrame frame = new JFrame( "Search test" );          
              frame.setSize( 500, 500 );
              Container container = frame.getContentPane();
             GridBagLayout gbl = new GridBagLayout();             
             container.setLayout(gbl);
             GridBagConstraints gbc = new GridBagConstraints();
             Action actionAdd = new AbstractAction("ADD")
                 public void actionPerformed(ActionEvent evt)
                      model_my_list.add( model_my_list.size(), add_box.getText() );
                      add_box.setText("");
             JButton buttonAdd = new JButton(actionAdd);
             buttonAdd.setPreferredSize( new Dimension( 100, 30 ) );
             gbc.gridx = 2;
             gbc.gridy = 2;
             gbl.setConstraints(buttonAdd, gbc);
             container.add( buttonAdd );
             Action actionSearch = new AbstractAction("SEARCH")
                 public void actionPerformed(ActionEvent evt)
                      search( search_box.getText() );
             JButton buttonSearch = new JButton(actionSearch);
             buttonSearch.setPreferredSize( new Dimension( 100, 30 ) );
             gbc.gridx = 2;
             gbc.gridy = 4;
             gbl.setConstraints(buttonSearch, gbc);
             container.add( buttonSearch );         
             JLabel add_label = new JLabel( "Add element" );
             gbc.gridx = 1;
             gbc.gridy = 1;
             gbl.setConstraints(add_label, gbc);
             container.add( add_label );
              gbc.gridx = 1;
             gbc.gridy = 2;
             gbl.setConstraints(add_box, gbc);
             container.add( add_box );
             JLabel search_label = new JLabel( "Search keyword" );
             gbc.gridx = 1;
             gbc.gridy = 3;
             gbl.setConstraints(search_label, gbc);
             container.add( search_label );
              gbc.gridx = 1;
             gbc.gridy = 4;
             gbl.setConstraints(search_box, gbc);
             container.add( search_box );
              JLabel all_label = new JLabel( "All elements" );
             gbc.gridx = 1;
             gbc.gridy = 5;
             gbl.setConstraints(all_label, gbc);
             container.add( all_label );
             JScrollPane all_scroll = new JScrollPane( my_list );
              gbc.gridx = 1;
             gbc.gridy = 6;
             gbl.setConstraints(all_scroll, gbc);
             container.add( all_scroll );
              JLabel found_label = new JLabel( "Searched elements" );
             gbc.gridx = 2;
             gbc.gridy = 5;
             gbl.setConstraints(found_label, gbc);
             container.add( found_label );
             JScrollPane found_scroll = new JScrollPane( list_of_finds );
              gbc.gridx = 2;
             gbc.gridy = 6;
             gbl.setConstraints(found_scroll, gbc);
             container.add( found_scroll );
              frame.pack();
              frame.setResizable( false );
              frame.setVisible( true );          
         public void search( String searchWord )
              model_list_of_finds.clear(); //empty
              for ( int i = 0; i < model_my_list.size(); i++ )
                   String text = (String)model_my_list.get(i);
                   if ( text.startsWith( searchWord ) )
                        model_list_of_finds.add( model_list_of_finds.size(), text );
                        continue;
                   if ( text.endsWith( searchWord ) )
                        model_list_of_finds.add( model_list_of_finds.size(), text );
                        continue;
                   String keyword[]  = text.split( searchWord );
                   if (keyword.length > 1 ) //if there was a split... there was a find
                     model_list_of_finds.add( model_list_of_finds.size(), text );
                     continue;
         public static void main ( String args[] )
              new MAIN();
    }

  • Announce: Generic Algorithms for Java, 0.3 release

    The latest release of jga: Generic Algorithms for Java is now available at
    http://jga.sf.net/
    Included in this release are a collection of algorithms adapted from STL that operate over generic collections and iterations. What's included are the algorithms that do not attempt to modify the collection/iteration that they're given: in other words, all of the find/search/count algorithms. Also included are the complete set of functors and predicates that were present in the last release, with a few modifications for easier use.
    The library is released under the LGPL, and includes the source for the library, the test framework, and a non-genericized version of the test framework that verifies that the library may be used without having the generic environment available.

    A (similar, it seems) parameterized collections library for JSR-14 Java.I had looked at JUtil some time ago, and I'm not sure of the similarity: JUtil looks to provide a number of interesting (if somewhat specialized) collections. JGA provides functors & predicates, and STL-ish algorithms that operate on collections and iterators. Looks to me to be more complimentary than similar.
    Mr. Ananian/Astaire's site is also the source of SinjDoc, the javadoc tool that I used to generate the javadoc at jga.sf.net, and with which I replaced the munge4doc.pl and mungedoclist.pl scripts that I had previously used and posted on this forum.

  • In search of simple java editor

    Hello,
    I'm currenty searching for a java component that can display images.
    Preferably it has:
    - zooming capabilities
    - pluggable commands
    I have allready tried using DisplayJAI in combination with imagefilters. The only thing missing is a simple zooming (currently, I have quality loss when zooming in and out succesively).
    Does anyone know of a simple imagewidget that has these capabilities (preferably open-source)?

    You could just extend JComponent and make your own class...

  • Classify and cluster documents with algorithm in java

    Hello everybody,
    I want to classify and cluster, let's say, 100 documents which are on the Intranet.
    I was thinking of including an algorithm in java who can do this.
    But i have no clue where to start of where to begin.
    Please give me some advice or maybe an example ;),
    Greetings,
    Thijs Leusink
    [email protected]

    I was thinking of including an algorithm in java who
    can do this.
    But i have no clue where to start of where to begin.Two steps:
    1. Find an algorithm that can do what you want.
    2. Write it in Java.
    Start with the first step: if you haven't done that yet, I don't think we can help you here. Possibly you could use Google to find an algorithm.
    Then go on to the next step: if you don't even know how to begin writing a Java program then there is a collection of introductory tutorials here:
    http://java.sun.com/docs/books/tutorial/
    If you are having problems after that, come back here and ask a specific question.

  • Quasi Newton Optimization Algorithm in Java

    I need to implement Quasi Newton Optimization Algorithm in Java, I�ve found it in C. Can anyone help me to find this in Java?

    Plagiarise in British English. Plagiarize in American English.
    These forums are used internationally, so either is reasonable.
    Don't correct other people's spelling unless you're really sure of your ground.

  • Duplicating wincrypt algorithm in JAVA

    Here is the background on this project: I am creating an automated licensing system for a company that sells telco products. Until now, they have been manually generating the activation keys using an exe program developed in house.
    I have been hired to remove some of the manual interaction required from the employees, one of which is to automatically generate these activation keys. I emailed the developer of their exe about the algorithm he was using to generate the activation keys (based on serial number and hardware ID) because I need to duplicate that server-side. He said it probably wouldn't be possible because his program relies on wincrypt.
    So my question is this: Is there any way at all to duplicate the wincrypt algorithm in JAVA? because i dont need that system to based on WINCRYPT, and using wincrypt.h will bound the user to have that file. I need to duplicate the wincrypt functionality exactly. Being off by even a little bit would render the script useless as it would generate invalid keys...
    If anyone has any insight, I would love to hear it.
    Sarwat
    Edited by: Curgol on Jun 25, 2009 6:54 AM

    Curgol wrote:
    So you are saying that by using JCA/JCE i can have the same functionality that is being provided by wincrypt ?? i can implement the same encryption and decryption algorithm without being dependent on wincrypt.h and associated files ??
    I know JCA/JCE packages provides encryption/decryption tools, but the problem is that i dont have any idea how wincrypt encrypt files, have it's cpp files though, but can the wincrypt functionality impolemented in JAVA ??.
    Wincrypt from Microsoft is a library of hundreds of encryption related functions. The JCE defines an interface that providers provide implementations for. Just about everything in Wincrypt is available though one or more of the JCE providers.
    There is no one encryption method provided by Wincrypt. If you are trying to match in Java an existing cryptographic process implemented using Wincrypt then you need to know exactly which of the Wincrypt algorithms (cipher, block mode, padding, key generation etc etc etc) is being used. Without this knowledge you are stuffed.

  • Different algorithms in Java

    What are the different algorithms in Java? Can someone list me each and its uses?

    Genetic Algorithm Method to Count Elephants:
    Have lots of babies. Wait till they're sixteen and
    throw them into Africa telling them to count
    elephants. After 1 year, have a family reunion and
    ask them to tell you how many they saw. Kill all the
    ones that didn't count as many as some threshold.
    Have the surviors have lots of kids, with each other.
    Have them repeat the process. Keep it up and it'll
    l converge to the right number, as long as they don't
    die from a horrible genetic disease.
    A. I don't think I can have babies. No uterus found. Pls advise.
    B. Your fitness function doesn't guarantee that your babies are counting unique elephants. Its not much help if your kids count the same elephant 1 billion times.

  • Search Algorithms in Oracle

    What are the search algorithms that Oracle supports?

    You use SQL to select from a database. You can find the Oracle SQL syntax documentation on tahiti.oracle.com
    If you want to get down lower and find how the sql syntax is implemented, then some of this is documented on metalink.oracle.com in notes, and some is in books, and some is unknown ... (you could research your whole life ....)
    If you need to ask this question, I think you should start with the basics of blocks, memory, processes, undo, redo etc.

  • El Gamal Algorithm and Java problem

    I was trying to implement the El Gamal algorithm in java, but the problem is with the last part. I'm not getting the value of Message but according to algorithm i should get the value of message. Can anyone please tell me what did i do wrong?
    theory says, msg = x * (z^a)modInverse(p);
    import java.math.BigInteger;
    import java.math.*;
    import java.util.*;
    class ElGamel
         public static void main(String[] args)
              BigInteger p = new BigInteger("7747");
              BigInteger q = new BigInteger("3853");
              BigInteger a = new BigInteger("773");
              BigInteger k = new BigInteger("1193");
              BigInteger msg = new BigInteger("39");
              BigInteger y = new BigInteger("0");
              BigInteger x = new BigInteger("0");
              BigInteger z = new BigInteger("0");
              BigInteger temp = new BigInteger("0");
              BigInteger t2 = new BigInteger("0");
              int t1 = 0;
              y = q.modPow(a,p);
              System.out.println("Y is "+y);
              x = q.modPow(k,p);
              System.out.println("X is "+x);
              t1 = k.intValue();
              z = (msg.multiply(y.pow(t1))).mod(p);
              System.out.println("Z is "+z);
              t1 = a.intValue();
              x = x.pow(t1);
              temp = x.modInverse(p);
              msg = z.multiply(temp);
              System.out.println("Message is "+msg);
    }

    Reposting your code with markup. In future please use &#91;code] tags, as you can see the effect it has on legibility: import java.math.BigInteger;
    import java.math.*;
    import java.util.*;
    class ElGamel
        public static void main(String[] args)
            BigInteger p = new BigInteger("7747");
            BigInteger q = new BigInteger("3853");
            BigInteger a = new BigInteger("773");
            BigInteger k = new BigInteger("1193");
            BigInteger msg = new BigInteger("39");
            BigInteger y = new BigInteger("0");
            BigInteger x = new BigInteger("0");
            BigInteger z = new BigInteger("0");
            BigInteger temp = new BigInteger("0");
            BigInteger t2 = new BigInteger("0");
            int t1 = 0;
            y = q.modPow(a,p);
            System.out.println("Y is "+y);
            x = q.modPow(k,p);
            System.out.println("X is "+x);
            t1 = k.intValue();
            z = (msg.multiply(y.pow(t1))).mod(p);
            System.out.println("Z is "+z);
            t1 = a.intValue();
            x = x.pow(t1);
            temp = x.modInverse(p);
            msg = z.multiply(temp);
            System.out.println("Message is "+msg);
    }

  • Standard Algorithms in Java

    Hello everone,
    does someone of you know a collection of standard algorithms programmed in Java where you can download the examples
    (a kind of library or documentation)
    We are trying to standardize our code (e.g. complex loops and conditions) and this would be a great help!
    thanks a lot
    Julian

    hi,
    sorry for the misty question.
    I searched of the solution of quite common programming
    problems .
    meant to find something like that but it should be a little
    bit more complex:
    http://www.acknowledge.co.uk/snippets/java/docs/
    thank you
    Julian

  • How do I search the downloaded Java? Platform, Standard Edition 6 API?

    I am a newbie to Java (I had done some limited programming in C ++ years ago). I have downloaded the Java? Platform, Standard Edition 6 API Specification, and can view this in my Firefox browser. How can I search to certain classes in this documentation easily? For instance, if I want to ding out all of the operations available for String or System.out, etc, , is there a way that I can type in System.out, or String somewhere, and get a list of what is available?
    Thanks

    Brian_Rohan wrote:
    Here is the code that I compiled and it worked:Yeah, "args" there isn't part of the API. It's the name of the variable that is the first parameter to the main method defined there.
    The stuff between the parentheses is a list of declarations of local variables that form the parameter list to that method, to put it technically.
    The thing that's part of the API is the class String. In "String[] args", the bit on the left is the type of the variable, and the bit on the right is name of the variable. A name of something you define like that wouldn't be in the API. However, the name of the class String is.
    If you look at the import statements, you'll see if the name "String" is actually shorthand for a class defined elsewhere. There's this:
    import java.util.*;So you can know that "String" is either defined in the java.util package, in the "helloworld" package (which also wouldn't be in the API because you're defining in there using the package keyword), or in java.lang. (The "java.lang" package is automatically imported.)
    You could look in the API docs for java.util.String or java.lang.String to find the class definition. But I'll save you the time by letting you know that it's actually java.lang.String.
    Again I am new at this, what would args.length be?You'll see that the type of the "args" variable is "String[]". That means that it's an array of String objects. (The [] means arrays.) Arrays all have a field called "length". This also is part of the language, not the API, so it would be in a language guide, not the API.
    The value of args.length would depend on how you ran your program. When you run a program on the command line like this:
    java HelloWorldor this:
    java HelloWorld foo bar bazThe java virtual machine takes all the things after the class name and sticks them in an array of String. Then it looks for a static method named "main" in the HelloWorld class, and passes it that array. So in the first case above, args.length would be zero, and in the second case it would be three.
    By the way, when you post code, please wrap it in code tags. Highlight it, then click the "CODE" button above the text input box.
    Edited by: paulcw on Nov 4, 2009 6:35 PM

Maybe you are looking for