Reading Lots of Objects into Memory

I need to read thousands of fairly large objects from an object database to the point that the JMV memory runs out.
A solution is to: read, use and discard as opposed to loading all the objects into memory at once -pretty simple but I want to do it right. I have a general idea on how to implement the approach but I need your help in locating some reading materials, code snipplets and/or best practices.
Thanks!

If you're using thousands of objects you might look into the Flyweight Pattern (depending on what these objects are and what state they have). You might also look into increasing the amount of memory available to the JVM via that -Xms switch.
Aside from that, if you reuse the same reference name (variable name)(s) you'll allow the Garbage Collector to recover that memory.
Good Luck
Lee

Similar Messages

  • Reading entire txt file into memory?

    When you are using BufferedReader to read info into a buffer, that means you are reading the file into memory, correct? (Is that what buffer means?)
    I want to look for pattern matches in text files (about 1000 of them) using the regex utils. But I don't want to read and examine the text files line by line. I want to read in the entire text file into memory first and then look for the pattern matches. The text files generally don't exceed about 15K in size. I'm only going one file at a time, too, so this won't give me any out of memory errors, will it?
    And more importantly, how do I do it? I mean the "reading in the file" part only. I have my RegEx, I have my array of files to examine already. I just can't figure out the right code to use to read each file into memory before I look for pattern matches.
    Could someone help, please?

    When you are using BufferedReader to read info into a
    buffer, that means you are reading the file into
    memory, correct? (Is that what buffer means?)Yes.
    I want to look for pattern matches in text files
    (about 1000 of them) using the regex utils. But I
    don't want to read and examine the text files line by
    line.Why not?
    I want to read in the entire text file into
    memory first and then look for the pattern matches.Why?
    The text files generally don't exceed about 15K in
    size. I'm only going one file at a time, too, so
    this won't give me any out of memory errors, will
    it?Depends on how much memory you've given the VM and how much of that it's using already at the time you read the files, but in general, probably not a problem.
    And more importantly, how do I do it? I mean the
    "reading in the file" part only.Use BufferedReader to read line by line and then append each line (plus a newline, since BR.readLine() strips those off) to a StringBuilder.
    Or use a BufferedInputStream and and array that's as big as the file, and in a loop, try to read as much as is left into that array at an offset equal to how much has been read so far.
    I still think this is probably not a good approach though.

  • Is DocumentBuilder.parse(..) reading the whole XML into memory?

    Hello,
    I have a quick question about the following code snippet:
    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);
    DocumentBuilder builder = factory.newDocumentBuilder();
    Document doc = builder.parse("books.xml");Please have a look at the last line "builder.parse("books.xml");". I would like to use XPath to retrieve some informations of a xml file without having to deserialize the whole file (since the XML file rather big and I need to go through several of them in a loop).
    What exactly does the "builder.parse(...)" method? Is it reading the XML file and writing all the content to memory? Or does it only analyze and read the structure of the XML code? Or in other words, is the text of an element "//books/title" in memory after parsing, or not until I really access the "title" node of that XML?
    Thanks a lot for your help
    Best Regards,
    Ben

    Hi ,
    I did'nt knew how to start a new thread , so i am asking the question here:
    ===========================================================
    I am trying to read a xml file but i am not able to do so properly. I am new to xml .
    am trying to read a xml file of the following format in the staf and stax job xml file:
    <?xml version="1.0" encoding="utf-8"?>
    <operating_system>
    <unix_80sp1>
    <tests type="quick_sanity_test">
    <prerequisitescript>preparequicksanityscript</prerequisitescript>
    <acbuildpath>acbuildpath</acbuildpath>
    <testsuitscript>test quick sanity script</testsuitscript>
    <testdir>quick sanity dir</testdir>
    </tests>
    <machine_name>u80sp1_L004</machine_name>
    <machine_name>u80sp1_L005</machine_name>
    <machine_name>xyz.pxy.dxe.cde</machine_name>
    <vmware id="155.35.3.55">144.35.3.90</vmware>
    <vmware id="155.35.3.56">144.35.3.91</vmware>
    </unix_80sp1>
    </operating_system>
    Here machine_name tag can be n number. I need to read all the machine_name tags in an array , so that i can use them later.
    Also i need to read all the vmware tags in an array (both value and attributes) so that i can use them later. The number of vmware tags can again be n numbers.
    I am trying to use the following code :
    <!-- ******************************************************************* -->
    <!-- Following function is used to parse an XML file and return the DOM -->
    <!-- document object -->
    <!-- ******************************************************************* -->
    <function name="parseXML" scope="local">
    <function-list-args>
    <function-required-arg name="xmlFileName">
    Name of file containing XML to be parsed
    </function-required-arg>
    </function-list-args>
    <sequence>
    <!-- Parse the XML -->
    <script>
    factory = DocumentBuilderFactory.newInstance();
    factory.setValidating(1)
    factory.setIgnoringElementContentWhitespace(0)
    builder = factory.newDocumentBuilder()
    document = builder.parse(xmlFileName)
    </script>
    <script>
    vmware = None
    machine_name = None
    # Get the text value for the element with tag name "vmware"
    nodeList = document.getElementsByTagName("vmware")
    for i in range(nodeList.getLength()):
    node = nodeList.item(i)
    if node.getNodeType() == Node.ELEMENT_NODE:
    children = node.getChildNodes()
    for j in range(children.getLength()):
    thisChild = children.item(j)
    if (thisChild.getNodeType() == Node.TEXT_NODE):
    machine_name = thisChild.getNodeValue()
    # Get the text value for the element with tag name "machine_name"
    nodeList = document.getElementsByTagName("vmware")
    for i in range(nodeList.getLength()):
    node = nodeList.item(i)
    if node.getNodeType() == Node.ELEMENT_NODE:
    children = node.getChildNodes()
    for j in range(children.getLength()):
    thisChild = children.item(j)
    if (thisChild.getNodeType() == Node.TEXT_NODE):
    machine_name = thisChild.getNodeValue()
    </script>
    <return>[machine_name,vmware]</return>
    </sequence>
    </function>
    I am able to read only one value for machine_name (u80sp1_L005) and also one value for vmware
    i.e 144.35.3.91.
    What i want is read all the machine_value tags into a list , so that i can use it later
    Also i want to read all the vmware tags attributes into a list and all corresponding values into another list.
    what i mean is considering the following:_
    <machine_name>u80sp1_L004</machine_name>
    <machine_name>u80sp1_L005</machine_name>
    <machine_name>xyz.pxy.dxe.cde</machine_name>
    <vmware id="155.35.3.55">144.35.3.90</vmware>
    <vmware id="155.35.3.56">144.35.3.91</vmware>
    read all machine_name tag values into one list (u80sp1_L004,u80sp1_L005,xyz.pxy.dxe.cde)
    also all vmware attributes into one list *(155.35.3.55,155.35.3.56).....*and all the vmware values into one list *(144.35.3.90,144.35.3.91)*
    Also a good link to understand parsing using doumentbilder factory would be good help.
    Regards
    Sangram
    mail: [email protected]

  • Reading a text data file into memory

    hi,
    I have a text file which contains data, The text file is parsed and objects are created. The problem is the text file is quite huge measuring upto 1.8~2 Mb. The format of the text file is as follows
    Units: METRIC (atm, m3)
    * Step: 1 Time: 0.00
    * Average Field Pressure : 204.14
    * Region 1 Pressure : 204.14
    Well GROUP Layer Blk_Pressure BHP ResRate OilRate WaterRate GasRate KhProd Windex PindeWELLTYPE
    1 FIELD 1 204.14 49.33 6601.22 6568.10 37.14 538.07 99999.00 260.35 99999.00 P
    1 FIELD 2 204.14 50.34 6558.13 6525.23 36.90 534.56 99999.00 260.35 99999.00 P
    1 FIELD 3 204.14 51.35 6515.04 6482.36 36.65 531.04 99999.00 260.35 99999.00 P
    1 FIELD Tot 204.14 50.34 19674.40 19575.69 110.69 1603.67 99999.00 99999.00 99999.00 P
    2 FIELD 1 204.14 377.66 7573.96 0.00 7403.68 0.00 99999.00 260.35 99999.00 I
    2 FIELD 2 204.14 378.40 7606.33 0.00 7435.32 0.00 99999.00 260.35 99999.00 I
    2 FIELD 3 204.14 379.14 7638.70 0.00 7466.96 0.00 99999.00 260.35 99999.00 I
    2 FIELD Tot 204.14 378.40 22818.99 0.00 22305.95 0.00 99999.00 99999.00 99999.00 I
    * Step: 2 Time: 20.23
    * Average Field Pressure : 300.11
    * Region 1 Pressure : 300.11
    Well GROUP Layer Blk_Pressure BHP ResRate OilRate WaterRate GasRate KhProd Windex PindeWELLTYPE
    1 FIELD 1 194.20 49.33 858.83 853.40 5.36 68.22 99999.00 260.35 99999.00 P
    1 FIELD 2 194.48 50.34 871.71 866.22 5.42 69.35 99999.00 260.35 99999.00 P
    1 FIELD 3 194.76 51.35 884.86 879.29 5.48 70.49 99999.00 260.35 99999.00 P
    1 FIELD Tot 194.48 50.34 2615.40 2598.91 16.26 208.06 99999.00 99999.00 99999.00 P
    2 FIELD 1 370.40 377.66 912.25 0.00 891.74 0.00 99999.00 260.35 99999.00 I
    2 FIELD 2 371.26 378.40 895.75 0.00 875.61 0.00 99999.00 260.35 99999.00 I
    2 FIELD 3 372.12 379.14 879.29 0.00 859.52 0.00 99999.00 260.35 99999.00 I
    2 FIELD Tot 371.26 378.40 2687.28 0.00 2626.86 0.00 99999.00 99999.00 99999.00 I
    The Step goes on till like 3000, I am creating an object for each step which inturn has nested object for each well and the well in turn for each layer. In the above case of step 3 the object would be
    class Step 2{
    inner class well
    {     //for well 1
    inner class layer { // for layer 1 }
    inner class layer {/ for layer 2  }
    inner class layer {/ for layer 3  }
    inner class well
    {     //for well 2
    inner class layer { // for layer 1 }
    inner class layer {/ for layer 2  }
    inner class layer {/ for layer 3  }
    This architecture of mine is proving to be heavy as I would end up with around 9000 Java objects in memory, though my classes only have have int, float, String data items. I am using this data to plot graphs, so I guess it wouldnt be optimal to read data from text file for each plot.
    So in short the problem is can anyone tell a better way to read the data into memory ? given that there could be 3000 steps in the format given above.
    Thanks
    AM

    I have implemented and it takes around 30-45 sec to parse and also the GUI has become very slow. I query the objects for multiple combinations of graphs.
    The data from the objects is being used to feed the graphs on my GUI. I have a number of options on my GUI for different kinds of graphs, for each combination chosen the Objects are queried for the data. The GUI is written using Swing.
    So is there anyway I can fine tune the application, any tips about the object architecture or how to improve the speed. I am also explicitly running the garbage collector a few times in my program. Also how can I make JVM occupy lesser memory so that my program can have more memory.
    Thanks
    am

  • Read a file without loading it into memory

    I have a VERY large file filled with words. I want to determine if a String in my program matches one of them. Loading the entire file into memory would take a long time and would use way too much memory. Is there some way to load just the part I need each time or make the comparison directly from the file on the hard disk?

    The file is static, so your plan seems to have merit.
    I've only used sequential files before, but I'm
    assuming this would require a random access file since
    you have to begin reading at a specified offset. Am I
    correct?Yes.
    In addition, if this file is fixed-length "records" so that you could programmatically determine the word offsets without having to keep a map of indexes in memory, you could just employ a search algorithm (I forget what this one is called) - you could start by defining 2 endpoints, initially at offset 0 and the offset of the last word.
    Iteratively read the word halfway between the 2 endpoints, adjusting the endpoints until a match is found or the endpoints converge.

  • IB Error: Unable to read symbols from "UIKit" (not yet mapped into memory)

    I created a simple view program for iPhone, using IB to add a button and a label, once the button is clicked, the label text is changed.
    I am able to compile it fine, and ran it. The program/view showed up. But when I clicked on the button, the console showed this message:
    This GDB was configured as "i386-apple-darwin".warning: Unable to read symbols for "/System/Library/Frameworks/UIKit.framework/UIKit" (file not found).
    warning: Unable to read symbols from "UIKit" (not yet mapped into memory).
    I checked UIKit info, it's physically (full path) at:
    /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.0.sdk/System/Lib rary/Frameworks/UIKit.framework
    The relative path to SKD is:
    System/Library/Frameworks/UIKit.framework
    I'm running the latest SDK beta 8, OS X 10.5.4 latest update.
    Anyone had this bizarre problem? I was able to run all Apple samples fine.
    Please help!
    THanks!

    The answer is mentioned in a few other posts, but I thought I'd repeat it here, so that it's easier to find this solution in the future.
    I got the same problem trying to build an iPhone-Simulator application using another build process -- in this case, Boost Build. When I tried to run the program, it gave me the same nonsensical dynamic-library error.
    The problem is that the application you built for the iPhone Simulator is NOT a real application, even though it looks like one. You need to run it from within Xcode.
    In my case, I got around the problem by building the GLPaint sample using Xcode, copying over the executable it generated with the one I generated via Boost Build, then running GLPaint (actually my program now) from within Xcode. Then it worked.
    So far, it seems that Macintosh and iPhone are very unfriendly platforms to someone that's trying to port an existing project. Any attempt to do something, without completely buying into the Apple way of doing everything, leads to a torrent of cryptic undocumented errors. Even an old Unix guru like me can't keep up.

  • Read a folder and contents into memory

    Hello All,
    I want to put an entire folder (and it's contents) into memory.
    Use Case :
    1) Computer 1 has a folder called "foo" with 5 files in it. I want to run some program here which will serialize down into some file call it "fooserialized"
    2) Computer 2 deserializes "fooserialized" and can then create folder "foo" with all of it's original contents.
    Any suggestions? I browsed through the java.util.zip package with no avail.
    Thanks!
    TheCypher

    I want to do this simply as possible.tar cfj archive.tar.bz2 <files>
    <send to other computer>
    tar xfj archive.tar.bz2

  • Limitation of BitmapData loaded into memory through Loader.load()

    I am creating a photo gallery using Loader.load() to load
    pictures into Flash Player 9. I plan to make them preloaded in
    advance so when the users turn the page they can see the pic on the
    new page right away without waiting. So the pics are loaded one
    after another automatically on backend. When users request a new
    page, the bitmap object are simply added onto the page’s
    display list.
    I publish the code and test on IE 6 and Firefox 1.5. Only
    about 50 pics can be preloaded on FF and about 70 pics on IE. I
    read some articles talking about memory problem of Loader in Flash
    8.5 alpha. So is it still a problem for "Flash 9 Actionscript 3
    Preview Alpha" ? Check the article -
    http://www.jessewarden.com/archives/..._battlefi.html
    Please see attached my code for preloading job. No error is
    thrown.
    Please advise. Thank you very much

    Hi,
    if it's a IA report the limit of columns you can show at the same time is 100,
    check http://docs.oracle.com/cd/E17556_01/doc/user.40/e15517/limits.htm for more limnitations.
    Regards
    Bas

  • Missing/empty symbols, "Could not load scene into memory. Your document may be damaged." CS6

    Hey folks,
    we've been getting this problem a lot. We had it in Flash CS5.5 and still have it CS6.
    We're animating a series in Flash and this problem keeps cropping up way to often, but seems totally random.
    Basically I'll be working on a .fla saving regular versions through out the day, no problems. I'll close the file down. Then I (or someone else on a different computer) will open it up and I'll get the error msg "Could not load scene into memory. Your document may be damaged." Everything will be fine except for a few missing graphic symbols. Sometimes just the one, sometimes a whole bunch. So where the symbol should be on the stage, instead you get a small white square, but it still contains the animation information. So it will still move about the stage. If I bring in the same symbol from an old scene I can swap them out and the problem is fixed, (until it does it again on a totally different graphic symbol).
    If i select the symbol in the library the preview is white/blank and i can't go into it to edit. When I select the symbol on the stage, the "instance of:" gives me a blank "_ _ _" in the properties tab. How ever if i right click show in library it does show me the correct (although broken as in blank) symbol in the library.
    I havent been able to reproduce this problem on purpose but I'd say it happens in roughly one scene out of 10. Its seems to happen at random, I know there must be something in common but i havent figured out what. Its going on in heavy scenes (large library multipule characters), lite scenes (just the one character). Its not like it happens to the same symbol, different ones each time. Everything is local to the scene, all elements are created in flash. We are working across a network but as everything is local to the scene i don't see how that would be a problem. Also we don't have duplicate named symbols. It also won't necessarily affect something that has been changed since the last version. could be a background element thats been there, untouched since the first version of the scene.
    One point on how we set up our scenes which may be relevant. Start with empty scene, copy and paste symbols from other scenes to populate. Save.
    The problem could appear on say version 4 or version 36. Older versions are usually fine.
    So to sum up. Flash file is fine when we save it, open it up again later and a symbol will have disappeared. It's happening far too often.
    I know how to fix the problem, thats not what I'm asking. I need to know WHY its doing it so we can stop this from happening in the first place.
    Phew! Thanks for reading this far. Any help would be highly appreciated.
    Sander/

    Can you elaborate on how you use the network? Are you working from FLA/XFLs stored on a machine across a network (and saving them across the network)? I had plenty of those issues and Adobe has always warned not to work across a network. I just fell into the habit of copying over what I need to work on locally, then updating the file servers at the end of the day. Nothing was corrupt after that.
    Also lately in CS5.5 (not CS6 yet) I had noticed that I could change some graphics assets, close the document in OS X and open in Windows only to find freshly updated graphics reverted back. What's even more odd is if the OS X machine that made the changes opens it the changes are still made. This happens vice versa as well. My only solution on that was not to work cross-platform with other machines on a network unless absolutely necessary.

  • Pasting multiple objects into frame

    Greetings everyone
    I was wondering if someone could shed some light on how to best paste multiple objects into a single frame.
    Mainly, I'm trying to nest objects that form a part of a newspaper article header; since it's a design element that keeps repeating with each single article I don't want to spend time with manually positioning the elements each time--it's time consuming and prone to sloppiness.
    Unfortunately, the base frame will have about three objects that need to be nested within it. Once I paste one object into a frame, the next time I do `Paste into`--the content (previously pasted object in this case) just gets overridden.
    I've read that, to paste multiple objects into a single frame, one should group the objects first. Now this works pretty well for getting them into the frame, but leaves me with no options to position them relative to the frame. I can't move or position an object individually and I have no option to ungroup them once they've been pasted.
    Here's a small mock-up to depict my peculiar predicament:
    Ideally, I'm aiming to have the main header frame to fit the content of the nested objects and for it to be flexible enough that I can quickly span it across multiple columns as per article requirements.

    !kRON wrote:
    For items that are part of the group, their (X,Y) location coordinates are relative to the document--neither to the group nor the parent frame. The group itself has coordinates that are relative to the parent frame (X+ offset, Y+ offset). I can select the group and the parent frame and use the align to selection options. For individual items, I cannot make a selection of both them and the frame, only between items that are part of a group. I loose options to align an individual item relative to the parent frame, which means I have to drag them around and rely on smart guides.
    I've tinkered around some more and it'd appear it's not possible to lay out the elements as I imagined. So I have to resort to grouping the items that would form a frame and pasting them into a frame. If multiple frames have to be members of a single frame I'd have to group the frames and paste them into the single frame and so on.
    Based on my layout mock-up, I'd end up with 3 frames within the header frame:
    Article header
    Heading
    Article type
    Separator
    Article heading
    Title
    Footer
    Author
    Shape
    Based on experience, should I go with this or stick with the old fashined moving around of loose elements? Will it be a lot of work to maintain resizing and repositioning of the various items and their frames once I start changing the width and height of the header frame or will it save me some time?
    I'm not a working designer or production worker, so anything new takes me longer than I think it should, because I don't have a rich reservoir of quick methods at my fingertips.
    Your description above interested me, because I've been wrestling with the annoying limitations of manipulating and adding compound objects in anchored frames. For many situations, it's probably simplest to assemble each iteration of a compound object, that introduces a new element, outside the anchored frame, then group the elements and paste the grouped object into the anchored frame.
    However, the arrangement of elements in your example made me think that a table might be a workable solution. The answer I arrived at is "Yes, it's a solution, but it takes a while to set up." If it's reusable easily, then it might be worthwhile investing in a few experiments.
    I've attached a zipped idml interchange version of the file, and also a PDF.
    Here are some notes:
    * I created a table with three rows and three columns, no header or footer rows. Three columns because I thought I'd paste a vertical line into an inline anchored frame in the center cell of the top row, for the separator line. Later, I realized that defining the vertical cell rule as paper color would suffice, so I merged the two right cells on the top row to create a two-column row, with text in each cell and the vertical separator between them.
    * I merged all three cells on the second row into a single cell.
    * I merged the two right cells on the bottom row, but later, when I moved the cell rule to position the triangular graphic in the inline anchored frame in the right cell, it moved the vertical cell rule in the first row. So, I had to unmerge (or split) the cells, and cut/paste the triangular graphic's anchored frame into the rightmost cell. To position the anchored frame, I moved the vertical cell rule with Shift+Drag, to keep the table width unchanged.
    * I sampled the purple color with the eyedropper tool while cells were selected and the swatch panel was set to fill the selected object.
    * I didn't spend much time matching the type, as you can see. To position the text, I adjusted the cell margins in some cases, and left/right indention in some cases; I'm not sure which is more efficient in situations where there will be different amounts of text to fit. I didn't create cell styles or paragraph styles (BAH! Bad, bad, bad process) because I won't be repeating this stuff, but you probably will benefit greatly from defining styles for your repeating needs.
    * The triangular cut into the bottom row made it easy to manage creating and positioning the graphic. A non-geometrical object might be trickier. The triangle is a text frame, set to negative space above, so it overlaps the cell above it, enough to avoid a black gap.
    * Except for the vertical paper-colored divider line, all the cells have zero-width strokes.
    * I created the table in a text frame, that I fitted down to the size of the table, but I can't remember if I selected, copied, and pasted the text frame into the text flow to anchor it, or if I selected the table in the text frame, copied and pasted it into the flow. Not sure if it makes a difference in work efficiency.
    If I understand your general needs, this should work well for changing text and adjusting the column widths.
    HTH
    Regards,
    Peter Gold
    KnowHow ProServices

  • Passing objects into a class constructor

    I've created a frame for inputting a patient's medical results. There are lots of buttons for temperature, blood pressure, etc. I've created a separate, inner class that creates a NumberPad frame to enter the patient's results. Press the "Temp." button, for example, a NumberPad appears, you enter the temperature, press the "Enter" button on the NumberPad and the number appears in a Label next to the "Temp." button.
    The problem is that I've written the constructor of the NumberPad with a label parameter so that the right label gets updated. e.g.
    class NumberPad extends Frame implements ActionListener
    public NumberPad(Label aLabel)
    However, because I've implemented ActionListener, I've had to override the actionPerformed method and I don't know how to pass the aLabel object that I give to the NumberPad constructor into the actionPerformed method so that, for example
    public actionPerformed(ActionEvent evt)
    String s = evt.getActionCommand
    if (s.equals("ENTER")
    aLabel.setText(display of numberpad);
    How do I transfer the aLabel object into the actionPerformed method so that the correct label gets updated?

    Store the label as an instance variable.
    class NumberPad extends Frame implements ActionListener {
        private Label aLabel;
        public NumberPad(Label aLabel) {
            this.aLabel = aLabel;
        public actionPerformed(ActionEvent evt) {
            String s = evt.getActionCommand;
            if (s.equals("ENTER") {
                aLabel.setText(display of numberpad);
    }[/code}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

  • Can you embed object into text in Indesign?

    I am about to do a large project in Indesign that will involve a lot of charts and text. I used to work in Quark and I could embed objects into the text so that the object and text flowed together (instead of the object sitting on top of the text). Can anyone tell me if this is possible in Indesign?

    Cathy,
    You're using the word "embed" in a way that is confusing.
    An embedded object in InDesign is one where the link to the outside photoshop or other graphic file format has been cut and the entire file information is contained within the InDesign file.
    This has nothing to do with having images move along with the text.
    InDesign has two types of images that move along with the text.
    b Anchored objects
    are the easier ones to create but they have less controls.
    b Inline graphics
    are the more complex.
    Look these up in the Help files.
    Or get yourself a good book on InDesign.

  • Loading a large number of strings into memory quickly

    Hello,
    I'm working on an iPhone application where I need to load a large number of strings into memory. Currently I'm simply reading from a file where each string is stored in plain text on a single line. I read the file contents into a string using stringWithContentsOfFile and then I create an NSSet object using NSSet setWithArray:[string componentsSeparatedByString:@"\n"]];
    This works like a charm but takes around 8 seconds to load on the iPhone. I'm looking for ways to speed this up. I've already tried a few things which weren't any faster:
    1) I used [NSKeyedArchiver archiveRootObject:myList toFile:appFile]; to store the NSSet data structure. Then instead of reading the text file storage. I read this file using [NSKeyedUnarchiver unarchiveObjectWithFile:appFile]; This was actually very slow and created a strings file that was about 2x the size of the original plain text.
    2) Instead of using an NSSet, I used and NSDictionary and used writeToFile and dictionaryWithContentsOfFile. This was also no faster.
    3) Finally I tried using the NSDictionary to write to a binary file format using NSPropertyListSerialization. This was also not any faster.
    I've been thinking about using SQLite instead of the flat file read, but I haven't had a chance to prototype that out to see if it would be faster. It's important that I can do fast searches for specific strings, which is why I originally used a set.
    Does any one else have any ideas how to load this into memory faster? If all else fails, I'm simply going to load the strings into memory using a separate thread on application launch so I don't prevent the user from getting to the main menu for 8 seconds.
    Thanks,
    -Keith

    I'd need to know more about what you're doing, but from what you're describing I think you should try to change your algorithm.
    For example: Instead of distributing one flat file, split your list of strings into 256 files, based on the first two hex digits of their MD5 hashes*. (Two digits might not be enough--you might need three or four. You may also want to use folders, especially if you need more than two digits.) When testing if a string exists, first calculate its MD5 hash and extract the necessary number of digits, then load that file into memory and scan its list. (You can cache these lists in memory so that you only have to load each file once--just make sure that a didReceiveMemoryWarning message will empty those caches.)
    Properly configured, SQLite may be faster than the eight second load time you talk about, especially if you ensure it indexes the column you store the strings in. But it's probably overkill for this application.
    \* A hash is a numeric code calculated from a string; on average, changing a single bit anywhere in the string should change half the bits in the hash, so even very similar strings should generate very different hashes. I suggest using MD5 instead of -\[NSString hash\] because the hash method is not guaranteed to return the same results on Mac OS and iPhone OS, or on different releases of either OS. You could also use a different algorithm, like a CRC; these are faster but I'm not as familiar with them. This thread discusses calculating MD5 hashes on iPhone OS: http://discussions.apple.com/thread.jspa?messageID=7362074
    Message was edited by: Brent Royal-Gordon

  • Convert an Object into a String

    Good day to everyone. I am trying to do a little coding in which a user will input his desired username. I want to check the database if the desired username of the current user is existing, so there'll be no duplication. I'm using JPA for the model and JSF for the controller and view.
    Here's my Model.
    @Entity
    public class WebUser implements Serializable {
        private static final long serialVersionUID = 1L;
        @Id
        @GeneratedValue(strategy = GenerationType.AUTO)
        private Long id;
        private String lastName;
        private String firstName;
        private String middleName;
        private String username;
        private String password;
        public Long getId() {
            return id;
        public void setId(Long id) {
            this.id = id;
        @Override
        public int hashCode() {
            int hash = 0;
            hash += (id != null ? id.hashCode() : 0);
            return hash;
        @Override
        public boolean equals(Object object) {
            // TODO: Warning - this method won't work in the case the id fields are not set
            if (!(object instanceof WebUser)) {
                return false;
            WebUser other = (WebUser) object;
            if ((this.id == null && other.id != null) || (this.id != null && !this.id.equals(other.id))) {
                return false;
            return true;
        @Override
        public String toString() {
            return "web.model.WebUser[id=" + id + "]";
        public String getLastName() {
            return lastName;
        public void setLastName(String lastName) {
            this.lastName = lastName;
        public String getFirstName() {
            return firstName;
        public void setFirstName(String firstName) {
            this.firstName = firstName;
        public String getMiddleName() {
            return middleName;
        public void setMiddleName(String middleName) {
            this.middleName = middleName;
        public String getUsername() {
            return username;
        public void setUsername(String username) {
            this.username = username;
        public String getPassword() {
            return password;
        public void setPassword(String password) {
            this.password = password;
    }The controller and jsp are auto-generated from Netbeans. My question is, how will I convert the 'WebUser' object into a String so I can compare the value from input text to the value in the database?
    Here is my validation method inside WebUserController.java
    public void validateUsername(FacesContext facesContext, UIComponent component, Object value) throws ValidatorException {
            String newUsername = (String) value;
            System.out.println("new user name: " +newUsername);
            System.out.println("component: " +component);
            int webUserSize;
            StringBuffer sb = new StringBuffer();
            //webUsers.toString();
            if ((webUsers == null) || (webUsers != null)) {
                System.out.println("web users null");
                List<WebUser> allWebUsers = getWebUsers();
                webUserSize = allWebUsers.size();
                System.out.println("all web users: " +allWebUsers);
                System.out.println("size " +webUserSize);
                for(int i=1; i<=webUserSize; ++i) {
                    sb.append(allWebUsers);
                    System.out.println("username: " +sb.append(allWebUsers));
            }Honestly I'm new to java and having a hard time with this one. Hope someone can help me. Thanks a lot.
    I know my code is a mess. :(

    Thanks for the help guys! I used the "unique constraint" annotation of JPA and my problem is solved.
    Here's the code:
    @Entity
    @Table(
        name="WebUser",
        uniqueConstraints={@UniqueConstraint(columnNames={"username"})}
    public class WebUser implements Serializable {
        private static final long serialVersionUID = 1L;
        @Id
        @GeneratedValue(strategy = GenerationType.AUTO)
        private Long id;
        private String lastName;
        private String firstName;
        private String middleName;
        private String username;
        private String password;
        public Long getId() {
            return id;
        public void setId(Long id) {
            this.id = id;
        @Override
        public int hashCode() {
            int hash = 0;
            hash += (id != null ? id.hashCode() : 0);
            return hash;
        @Override
        public boolean equals(Object object) {
            // TODO: Warning - this method won't work in the case the id fields are not set
            if (!(object instanceof WebUser)) {
                return false;
            WebUser other = (WebUser) object;
            if ((this.id == null && other.id != null) || (this.id != null && !this.id.equals(other.id))) {
                return false;
            return true;
        @Override
        public String toString() {
            //return "web.model.WebUser[id=" + id + "]";
            return "web.model.User[username=" + username +"]";
        public String getLastName() {
            return lastName;
        public void setLastName(String lastName) {
            this.lastName = lastName;
        public String getFirstName() {
            return firstName;
        public void setFirstName(String firstName) {
            this.firstName = firstName;
        public String getMiddleName() {
            return middleName;
        public void setMiddleName(String middleName) {
            this.middleName = middleName;
        public String getUsername() {
            return username;
        public void setUsername(String username) {
            this.username = username;
        public String getPassword() {
            return password;
        public void setPassword(String password) {
            this.password = password;
    }Thanks again guys!

  • Loading images into memory

    I have an applet that I need to load images into memory. For instance, lets say I am in the first section of the applet. While that section is being showed, I want to load the background image of the second section into memory.
    To load the initial image into momory, I am just using a Mediatracker. No problem there. But I don't want to load all 20 backgrounds into memory at the same time as they take a lot of time. My understanding is that if I create a new MediaTracker while the first chapter is running, it will potentially cause some chaos, as that will stop my thread from running while I have an image loading.
    Somebody told me perhaps I could create a new thread and have that thread load the backgroudn into momory? Perhaps something like this?
    public class TestClass extends JApplet {
         private TestClass thisClass;
         public void init() {
              thisClass = this;
              Runnable r = new Runnable() {
                   public void run() {
                        MediaTracker tracker = new MediaTracker(thisClass);
                        Image nextImage = getImage( getDocumentBase(), getImagePath() +"img1.jpg");
                        tracker.addImage(nextImage,0);
                        try {
                             tracker.waitForID(0);
                        } catch (InterruptedException ie) {
                             System.out.println(ie.toString());
              Thread t = new Thread(r);
              t.setDaemon(false);
              t.start();
              while(t.isAlive()) {
                   int i = 1;     
              t.stop();
              t.destroy();
    }No idea if I am on the right track or not? Another friend told me something about swing helpers but couldn't tell me much more?
    Thanks in advance!

    I use media tracker when I need information about how percent the image is loaded. you can use JLabel to load the images since it has own image observer in it.
    hope you just want to deal with it. easiest way I can offer :)

Maybe you are looking for