The Best Way to Check a Database.

Hello All,
I'm working on an app. where I'll retrieve upwards of 1 million items.
For each retrieval, I have to make sure that the item was not previously retrieved and stored. That means on the 1millionth item, I have to test the previous 999,999 items . If a match is found, I would ignore it. Other wise I would add it to the store.
I thought about storing all items in a simple hashtable. But 1 million items in a hashtable probably wouldn't be supported on a standard 128MB pc.
So my only other option seems to be database.
But the time required to retrieve and compare to the database seems like it would take forever.
My best guess would be to retrieve say 3000 items then do all my tests at that time and store. Clear the memory and repeat. Is that the strategy you guys would try ?
Does that seem doable -- i'll be using a standard jdbc-odbc bridge by the way
stev
Then I'm going to dump it into a database (MSAccess 2000) using the batch method -if the addBatch method is supported by

If you create your database table to have an index which requires unique values, then an insert of a duplicate will throw an exception. I don't know whether addBatch will be practical here -- it might throw the exception and not add the rest of the batch, but then again it might not. Of course, adding 1 million items is going to take some time, but the other advantage of the database is that the data is permanently stored in a form that's easy to access in a variety of ways.

Similar Messages

  • What is the best way to create a database schema from XML

    What is the best way to create a database schema from XML?
    i have  a complex XML file that I want to create a database from and consistently import new XML files of the same schema type. Currently I have started off by mapping the XSD into Excel and using Mysql for Excel to push into MySQL.
    There must be a more .net microsoft solution for this but I cannot locate the topic and tools by searching. What are the best tools and way to manage this?
    Taking my C# further

    Hi Saythj,
    When mentioning "a database schema from XML", do you mean the
    XML Schema Collections? If that is what you mean, when trying to import XML files of the same schema type, you may take the below approach.
    Create an XML Schema Collection basing on your complex XML, you can find
    many generating tools online to do that.
    Create a Table with the above created schema typed XML column as below.
    CREATE TABLE youTable( Col1 int, Col2 xml (yourXMLSchemaCollection))
    Load your XML files and try to insert the xml content into the table above from C# or some other approaches. The XMLs that can't pass the validation fail inserting into that table.
    If you have any question, feel free to let me know.
    Eric Zhang
    TechNet Community Support

  • I am getting pop ups on safari and firefox lately and am worried i may have malware or something now on my computer doing this. What is the best way to check this out for sure and remove it?

    I am getting pop ups on safari and firefox lately and am worried I may have malware or something now on my computer doing this. What is the best way to check this out for sure and remove it?

    Please review the options below to determine which method is best to remove the Adware installed on your computer.
    The Easy, safe, effective method:
    http://www.adwaremedic.com/index.php
    If you are comfortable doing manual file removals use the somewhat more difficult method:
    http://support.apple.com/en-us/HT203987
    Also read the articles below to be more prepared for the next time there is an issue on your computer.
    https://discussions.apple.com/docs/DOC-7471
    https://discussions.apple.com/docs/DOC-8071
    http://www.thesafemac.com/tech-support-scam-pop-ups/

  • Whats the best way to check table before form deletion or update ?

    Hi, I have a func_staff table made up of a composite key - func_no and staff_id. I have a form where the datablock references this table.
    I need to be able to check that before a func_no and staff_id combination on the form are written to the database that that combination does not already exist in the table.
    What would be the code for that ? I've tried stuff like IF EXIST statements and even a select statement but I totally worded it it wrongly.
    To DELETE an entry in that same table, would the best way to be create an LOV for both func_no and staff_id, the problem with that would be if a user selected a combination that does not exist ? Could I create a combined LOV that would populate each field ?? (this way I could be 100% sure that the selected combination is valid)
    Thanks Peeps.

    hi,
    you can try and use the count(*), before deleting and updating
    lets say
    select count(*)
    into l_count
    from table_name
    where column1 = :block.column1
    and column2 = :block.column2;
    if l_count <> 1
    then
    insert into..........
    else
    show error message;
    raise form trigger failure;
    end if;
    similarly u can use l_count before deleting.
    regards,
    Mayank

  • What's the best way to check whether a user is logged in or not?

    I have a question about basic session handling.
    I'm running Tomcat 5.0.30 and have a web application where users can register with a username and password, and then log into a "member site".
    What is the best way of making sure that a user actually has logged in or not?
    What I've done in previous applications I've made is that I've just put a simple variable into the users session after he has successfully entered his password (i.e. Boolean loggedIn=true). Then I just test if this flag is true to grant him access to the member site. Is that a smart way of doing it?
    Are there any libraries I can use which handles sessions for me in a secure way?
    All comments and suggestions are appreciated!

    Maintain a flag using session attributes it to one when the user is logs in set it to value.... and change set it zero or invalidate that session... when the user is logged out...
    use something like this while user had logged in
    session.setAttribute("flag",<unique_number>);
    for checking whether the user had logged in or not...
    if(Integer.parseInt(session.getAttribute("flag").toString())=!<unique_number>){
    out.println("The session had expired");
    out.close();
    }

  • Mac Pro won't start up.  What's the best way to check the power supply?

    Hello. I'm having a severe problem with my 2007 Mac Pro. The symptoms are very easy to describe. The computer apparently shut itself off overnight, and when I press the power button to turn it on, absolutely nothing happens. No lights, no chime, no signs of any power whatsoever in the computer. I have never had major problems with the computer before. (Although I had recently noticed some minor skipping problems with streaming videos and Firefox had started hanging occasionally.)
    So far, I have checked the power cable and the power outlet. I have also reset the SMC (holding in the Power button for 5 seconds with the unit unplugged). I am unable to see any light in the diagnostic LEDs (including LED 1) when I press the DIAG_LED button on the logic board.
    While I welcome other suggestions, all I can figure is that this is either a power supply problem or a major logic board failure, but I don't know which. I have a multimeter and would like to test very directly if the power supply is the problem, because they are very expensive and obviously I do not want to order a replacement supply if it would not fix the computer. Does anyone know what the best way to test the supply directly is? What pins on what cable should I test, and what voltage should I look for?
    Also: No the computer is not under any warranty. If at all possible, I would like to fix this myself. Thanks for any help you can give.

    Well, I may have found the answer to my question as to why replacing the fuse dows not seem to be encouraged. I cut out the failed fuse (fast fuse, ceramic, 16 Amp 250V), and replaced it with another that was as close to it in spec as I could find (15 A instead of 16). Then I put the supply back in the case.
    When I tried to plug it in, I got an immediate spark, followed by an acrid smell from the power supply. Later inspection revealed that the new fuse blew, as did another component nearby (not sure what it was) where I could see blackening and the smell was concentrated. At this point, I would consider myself lucky if the power supply didn't take anything else on the computer out with it.
    Lesson learned; will try a new power supply and go from there.

  • Whats the best way to check the speed of my macbook Pro

    Hi people.
    I just wanted to get some advice on how my Mac is running, as i think it has become slower since i bought it and wondered if it is anything i have done or should be doing
    First of all, i have a MBP bought mid 2011, has thunderbolt port, its 2.3Ghz Intel i5, 4GB 1333Mhz DDR3, 320Gb HD and now running Mountain Lion.
    When i first got it i thought it was quicker than anything i have used previously, an example would be when i click to open an apple program e.g. iphoto, the icon jumped up and then seconds later it was open. Now any program i try to open, it seems to take much longer, it bounces for a while and then finally opens.
    I have not got any clutter on the laptop, i keep all my files on a network hard drive. The only thing i have is 80Gb of photos stored under iphoto and i have Windows XP running on a 40Gb partition using Bootcamp.
    I am thinking about getting rid of the XP on my mac to see if it helps, but thought i would ask here first. I need XP as it allows me to run programs which dont support Apple OS X.
    I was also wanting to know the best way to store photos, would i be able to store the photos on my network drive and have iphoto work exactly the same..?
    Help much appreciated.
    Thank you.

    Hi people.
    I just wanted to get some advice on how my Mac is running, as i think it has become slower since i bought it and wondered if it is anything i have done or should be doing
    First of all, i have a MBP bought mid 2011, has thunderbolt port, its 2.3Ghz Intel i5, 4GB 1333Mhz DDR3, 320Gb HD and now running Mountain Lion.
    When i first got it i thought it was quicker than anything i have used previously, an example would be when i click to open an apple program e.g. iphoto, the icon jumped up and then seconds later it was open. Now any program i try to open, it seems to take much longer, it bounces for a while and then finally opens.
    I have not got any clutter on the laptop, i keep all my files on a network hard drive. The only thing i have is 80Gb of photos stored under iphoto and i have Windows XP running on a 40Gb partition using Bootcamp.
    I am thinking about getting rid of the XP on my mac to see if it helps, but thought i would ask here first. I need XP as it allows me to run programs which dont support Apple OS X.
    I was also wanting to know the best way to store photos, would i be able to store the photos on my network drive and have iphoto work exactly the same..?
    Help much appreciated.
    Thank you.

  • The best way to check the differences between two lists? Or substracting

    Hi,
    I am Googling on this and find many different answers and I am not sure what the best approach is. I have two lists:
    List A: with 10000 entries (String with length 6 all of them)
    List B: with 10002 entries (String with length 6 all of them)
    List A and B are almost identical but with 2 differences. I want to find these two differences or in other words generating a third list, List C with these 2 differences.
    What is the best way to approach? To us Array, ArrayList, Vectors, Hashmap, Hashtable...
    Thank you for any hunches

    Maria1990 wrote:
    I think I found the easiest approach...
    listA = [a, b, c];
    listB = [a, b];
    listC = listA.removeAll(listB);
    ...but if it is the best approach with performance in mind.. I have no idea...It really depends on what sort of comparison you want. Lists allow duplicate elements, Sets do not; so a comparison of Sets based on the contents of your Lists would only return distinct element differences.
    My suggestion: sort both lists and do a "staircase" comparison. That is (assuming they're both in ascending order):
    1. Read through both until you find a mismatch.
    2. Read through the one containing the "lesser" element, until you find a match, or until it's exhausted.
    3. If you find a match, go back to 1.
    4. If you exhaust one list, then any remaining elements in the other are also "mismatches".
    HIH
    Winston
    PS: With this approach, you can also flag your mismatches as "A not in B" or "B not in A".
    Edited by: YoungWinston on Jul 27, 2010 4:24 AM

  • What's the best way to check my macbook pro for malware?

    I've noticed of late a couple of random popups and suspicious-looking installers on my macbook pro. How can I check for malware? And if my lap top's infected, how can I disinfect it?

    There is no need to download anything to solve this problem. Your desciption isn't very specific, but I'll venture a guess.
    You may have installed the "Genieo" or "InstallMac" ad-injection malware. Follow the instructions on this Apple Support page to remove it.
    Back up all data before making any changes.
    Besides the files listed in the linked support article, you may also need to remove this file in the same way:
    ~/Library/LaunchAgents/com.genieo.completer.ltvbit.plist
    If there are other items with a name that includes "Genieo" or "genieo" alongside any of those you find, remove them as well.
    One of the steps in the article is to remove malicious Safari extensions. Do the equivalent in the Chrome and Firefox browsers, if you use either of those.
    After removing the malware, remember to reset your home page in all the web browsers affected, if it was changed.
    If you don't find any of the files or extensions listed, or if removing them doesn't stop the ad injection, then you may have one of the other kinds of adware covered by the support article. Follow the rest of the instructions in the article.
    If you find Apple's instructions too hard to follow, ask for an alternative that doesn't require you to trust a black-box application without knowing what it does.
    Make sure you don't repeat the mistake that led you to install the malware. Chances are you got it from an Internet cesspit such as "Softonic" or "CNET Download." Never visit either of those sites again. You might also have downloaded it from an ad in a page on some other site. The ad would probably have included a large green button labeled "Download" or "Download Now" in white letters. The button is designed to confuse people who intend to download something else on the same page. If you ever download a file that isn't obviously what you expected, delete it immediately.
    In the Security & Privacy pane of System Preferences, select the General tab. The radio button marked Anywhere  should not be selected. If it is, click the lock icon to unlock the settings, then select one of the other buttons. After that, don't ignore a warning that you are about to run or install an application from an unknown developer.
    Still in System Preferences, open the App Store or Software Update pane and check the box marked
              Install system data files and security updates (OS X 10.10 or later)
    or
              Download updates automatically (OS X 10.9 or earlier)
    if it's not already checked.

  • What is the best way to install standby database?

    Hello.
    I'm trying to develop a good scenario for implementing standby database. I've already installed and configured standby. It's working.
    Is there any best practice for installing database instance for standby (i.e. using SWPM)?

    Hi
    If you are talking about an oracle standby database, logic assumption as you are posting in the Oracle space, SWPM won't help you, it is meant for installing SAP instance.
    For an Oracle standby DB you need to install an Oracle instance and restore the master DB on it.
    Regards
    http://docs.oracle.com/cd/E11882_01/server.112/e17022/rcmbackp.htm
    http://www.pythian.com/blog/creating-a-physical-standby/

  • Whats the best way to check weather a file exists online, or not?

    Hi!
    I need to show an Image File in my app which may have different sizes. _1.jpg, _2.jpg and so on.. If, for example, _2.jpg does not exist, I have to take _1.jpg.
    So my question is: is there a better way instead of making/allocating a pic and asking pic != nil ?
    thx!
    alex

    Hi Alex -
    The subject line says "file exists online, or not?". Are you trying to check existence at a network site, or the local file system? I'll cover the local fs first, since that's the easiest. If you need to search a remote site, I think we'll need some more details.
    Assuming an iPhone project (not specified, but your usual topic), NSFileManager will probably be all you need unless the files are localized and/or in the application bundle, in which case you'll want NSBundle. For example, say you're storing all the jpg's in the user's cache directory:
    #define MAXFN 5
    // return nil or the path to the file with the highest numeric suffix
    - (NSString)getPathForBestFile:(NSString)baseName extension:(NSString*)ext {
    NSString *bestFilePath = nil;
    NSFileManager *fileManager = [NSFileManager defaultManager];
    // locate the user's cache directory
    NSArray *paths = NSSearchPathForDirectoriesInDomains(
    NSCachesDirectory, NSUserDomainMask, YES);
    NSString *cacheDirectory = [paths objectAtIndex:0];
    NSLog(@"cacheDirectory=%@", cacheDirectory);
    // loop through file names with numeric suffixes
    for (int i = 1; i <= MAXFN; i++) { />/
    NSString *fileName =
    [NSString stringWithFormat:@"%@_%d.%@", baseName, i, ext];
    NSString *filePath =
    [cacheDirectory stringByAppendingPathComponent: fileName];
    NSLog(@" filePath=%@", filePath);
    if ([fileManager fileExistsAtPath:filePath])
    bestFilePath = filePath;
    return bestFilePath;
    If you need to use NSBundle, see pathForResource:ofType:. If you need to determine the existence of a file at a network site, please give us some details about the kind of server and how much you'll know about the possible file locations.
    Hope that helps!
    - Ray

  • What's the best way to check what's on my hard drive?

    Hello Everyone...
    I'm fairly new to the mac experience so I apologize in advance if this information is readily available elsewhere...
    I'm trying to find out what is taking up the most space on my hard drive with hopes to move or delete files... I did several searches here on the forums and well as googled and came up with some freeware programs like Disk Inventory X, What Size?, and Omni... Any opinions on what works best? or any other ways I can find these files? Thanks for the help...

    The following articles have some good tips:
    Freeing space on your Mac OS X startup disk
    Freeing Up Hard Disk Space
    Running maintenance routines is critical to keeping the drive clear of unneccesary files:
    http://www.macworld.com/2005/01/features/preventmacdisasters/index.php
    http://www.thexlab.com/faqs/maintscripts.html
    Since you're a newcomer to the Mac, see these:
    Switching from Windows to Mac OS X,
    Basic Tutorials on using a Mac,
    MacFixIt Tutorials,
    MacTips, and
    Switching to the Mac: The Missing Manual, Leopard Edition.
    Additionally, *Texas Mac Man* recommends:
    Quick Assist.
    Welcome to the Switch To A Mac Guides, and
    A guide for switching to a Mac.

  • Is it pattern matching the best way to check presence and orientatio​n of components on a pcb board?

    Hi, like i ask on the title i need some hint about this. I need to program a simple AOI machine that must inspect a pcb board and find if all the components are correctly placed and oriented. I though to use the pattern matching with a good template of the object to inspect. Can it work?
    All kinds of tips that may come to your mind please tell me, because this is the first time i try a task like this. I already have some experience in image manipulation but not in AOI machines.
    Francesco.

    Hi Francesco,
    you can find some useful information here, here and here.
    I hope this will help!
    Bye,
    Licia

  • Best way to check whether the database is demo or sys?

    Hi Gurus,
    Whats the best way to check whether the installed peoplesoft database is a demo or a sys?
    Thanks for the help?
    Regards,
    Anoop

    There is nothing set by default.
    However, if it has been configured properly by the administrator after db creation, through the menu Peopletools>Utilities>Administration>Peopletools Options, the following query should return the type of the database :
    select systemtype from psoptions;Otherwise the following could help to understand what database you are on.
    From HRMS9.1 DMO database :
    SQL> select count(*) from ps_employees;
          2792From HRMS9.1 SYS database :
    SQL> select count(*) from ps_employees;
      COUNT(*)
             0Nicolas.

  • Best Way to Check for same Word in string?

    If I have an array of words, would the best way to check for the same word be to use 2 for loops?

    Huh?
    Sounds like homework...
    What is a word? Presumably a String.
    For equality of Strings you use...
        String s1 = ....
        String s2 =....
        if (s1 == s2) { equal depending on null case
        else if ((s1 != null) && (s1.equals(s2))) { equal }
    For the case of checking one array to another....
          while items in array1
                 get itema from array1
                      while items in array2
                           get itemb array2
                                  is itemb equal (see above) to itema
                                         yes - then do something

Maybe you are looking for

  • Ver.9.5 The volume for file name cannot be found

    HI, My work Mac Book pro was recently upgraded to Mavericks from 10.6.8, with iPhoto 11 (ver. 9.5) also an included upgrade. One album of photos is no longer accesible - clicking on each thumbnail provokes the following window "The volume for <file n

  • Setting an external monitor as primary on a MacBook in XP (w/ Bootcamp)?

    Hi, I'm trying to use a 17" LCD with my macbook to take the advantage of "Extended Desktop". I know it's very easy in OSX where you just need to drag the menubar from one screen to the other in the display setting window. I was wondering if there is

  • How to avoid po release strategy replicating in RFQ

    Dear Sapiens, I am aware  as we are using same class for PO RFQ and Contract the release strategy will repliacte in rest, but I dont want PO release strategy to get repliacted in RFQ,because I dont have realease for RFQ how to avoid po release strate

  • Custom components and the navigator object

    Hi all, I'm starting on a mobile project which is a ViewNavigator based application. I've created a custom component which contains a button. I'd like the button to be able to use the pushView method however as it's a custom component, it doesn't app

  • IPhoto Upgrade Failure

    Hello, I recently upgraded my Mid 2009 Intel iMac from Mavericks to Yosemite, and since then iPhoto has failed to work. Every time I click onto the dock, it tells me that my version isn't compatible with Yosemite (obviously as it was updated and inst