What data storage method should I use?

I am currently writing a program and among other things need to store data.
The program listens to two computers talking to each other in a known protocol (the program know what protocol it is,the user give the program the protocol structure ), if the packet that arrived to the program has unseen value in one or more of the fields
it add it to the database.
I am not sure what data storage method(such as sql server,xml etc.) to choose since I don't know whether the data stored in the database would be large or not, it depends on the computer's conversation
Also I will need to be able to get all the data from the database with no trouble and might need to extract some data and not the other (for example all the packets with the value 24 in field x)
By protocol I can't just give you one because the user of the program defines which protocol the conversation is in.
By conversation I mean that two computers talk to each other it doesn't matter if a user operate the computers or not.
The maximum number of entries in the database would be the number of variations that the packet in the certain protocol can be. I mean if the protocol is 8 bytes long the maximum number of entries would be 2^64 but if the header is longer the number goes up,
but the number of entries could be less depending on the conversation and it is out of my control.
The program will only store unique headers, I mean that the same combinations of values in the header won't be in the DB twice, the program store the header if it is unique and it isn't stored in the DB
I thought of using a sql server such as mysql but I am not sure what data storage method would suit this situation the most?

Since access to the data storage will not be concurrent and given only the "monitoring" program will write to it (according to your answers to some comments), my recommendation is:
The safe way to go now seems to be a database, but you shouldn't be concerned right now with the administration of one
Use SQLite locally
in the machine the monitor runs in, and steer your effort towards improving and/or completing the protocol part
If SQLlite database grows to large, rename it, archive it and create it again empty
Later you can move to another database engine with no change of code ( if you do things the standard way ).

Similar Messages

  • What RAID storage system should I use?

    To set the stage here. I'm somewhat of a newbie to the video industry. I've worked as a videographer for a non-profit for 5 years. At that job, we just skimped by on what we could afford, which wasn't much. I just started a new "professional level" job for a school district and have been given the keys to a fairly substantial budget to get whatever I need to do the job.
    I want to do right by them and not waste money, so I want my purchase decisions to be educated. I'm an intermediate computer user, but have never used raid configurations before, so please be kind. Also, we can really only purchase through a few vendors. B&H is where I'm getting all my other video equipment, so I'm only looking at options available there for my storage needs as well.
    Right now we record using Canon XA10 and XA20 model cameras. I'm hoping to upgrade to XF300's with this new budget, but still we're only talking MXF files, 1920x1080 at 50Mbps 4:2:2. So I'm not dealing with huge uncompressed footage.
    Still I record a fair amount of footage. In the first 2 months on the job I've accumulated about 460GB of raw video, and I don't expect demand to go down in the future.
    Right now, my idea is to purchase two Western Digital 12TB Raid Arrays in Raid 0.
    http://www.bhphotovideo.com/c/product/1053138-REG/wd_wdblwe0120jch_nesn_12tb_my_book_duo.h tml
    (I should note, I'm using a Windows 7 PC, so I only have access to USB 3.0, not Thunderbolt)
    The first raid array would be my scratch disk, the second would be used for manual backup at the end of every day. (Using a utility like SyncBack)
    Once my projects are complete, and I'm sure I won't need to access them, I'd like to move them off to a 3rd RAID array like this set up in Raid 5 for redundancy.
    http://www.bhphotovideo.com/c/product/1018063-REG/owc_other_world_computing_mercury_qx2_4_ by_hw.html#specOWCM3QX2K0GB
    This array would serve primarily as an archival unit, with only occasional transfers to it and use in only rare circumstances where I need access to several month old footage.
    Do you have any suggestions of a better system or workflow?
    Thanks so much!

    Would be good to raise this over on the:
    Hardware Forum ...
    https://forums.adobe.com/community/premiere/hardware_forum
    Neil

  • What data storage method to choose?

    I need to write a relativley simple application for "itinerary orders" (legislative term from Croatia translated? by me). I would like to use the pover of SQL but still avoid distributing SQL server along with my application. Is there a way to have the pover of SQL server within my own java app?

    You might want to have a look at Java embedded databases such as HSQL (http://hsqldb.sourceforge.net/).
    HSQL can be bundled and run in the same VM as your application.

  • What method should be used for resizing the particular JTable Column width

    I have a four table. Only one table which are on top have a table header. I want that when user resize the topmost table with a mouse other table colume also be resized automatically. So I want to know that what method should be used for resizing the particular JTable Column width.
    Thanks
    Lalit

    maybe you can implement a interface ComponentListener as well as ComponentAdapter with your topmost Table.
    for example:
    toptable.addComponentListener(
    new ComponentAdapter(){
    public void componentResized(ComponentEvent e){
    table1.setSize(....);
    table2.setSize(....);
    /*Optionally, you must also call function revalidate
    anywhere;*/
    );

  • What data sturcture I should use?

    I want to implement a table like this:
    dest     Cost      Next hop
    B     4     B
    C     6     B
    D     5     D
    E     3     D
    Opertaions on the table would be:
    1.inset a row
    2.update a row
    3.sort based on dest and Next hop
    Could someone give me an idea that what data sturcture i should use and why?

    I want to implement a table like this:
    dest     Cost      Next hop
    B     4     B
    C     6     B
    D     5     D
    E     3     D
    Opertaions on the table would be:
    1.inset a row
    2.update a row
    3.sort based on dest and Next hop
    Could someone give me an idea that what data
    sturcture i should use and why?JTable.
    It is designed to store tabular data, and supports insertion, deletion, and is not difficult to modify for sorting.
    See the Java tutorial at http://java.sun.com/docs/books/tutorial/uiswing/components/table.html
    � {�                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

  • What type of restore should I use?

    My computer is properly backed up using Time Machine. I created a partition using Bootcamp so I could install Windows (I haven't actually installed Windows yet). However, after testing Parallels, which is an amazing program!, I want to get rid of the partition and use Parallels instead.
    My understanding is that I have to wipe my hard drive to erase the partition. Assuming this is true, what version of restore should I use? I want the computer to be exactly as it is in the backup, except for the partition.

    Ian Chamandy1 wrote:
    When I go into the Partition tab, the diagram and text shows my HD as having 148.73 gigs of space (it appears in blue on top in the diagram) and "available space" being 49.5 gigs (it appears in white on the bottom).
    Once you deleted the Boot Camp partition, you now have only a single partition. The light-blue shading represents the data in the partition; the white are represents the rest ("available space").
    Is this how it should be? I clicked on both the blue area and the white area but that doesn't select them. There is no line between them for me to drag. Also, the "-" and "Options" buttons are grayed out.
    If there's only one partition on the disk, the whole diagram, containing both the blue shading and white area, covering the entire disk, should be outlined in medium blue. The "-" sign is grayed-out because there's only the one partition. If there were 2 or more, you could use it to delete one.
    If you wanted to make a second partition, you would click the plus sign. DU would then change the diagram, splitting the existing partition. Then you'd see a line between the two, which you could drag up or down, etc. The Options button is used when setting-up a new or erased drive "from scratch," to specify the *Partition Map Scheme* for the entire drive.
    Browse Disk Utility's help for a general idea of what it can do. (It's one of the few apps that has fairly comprehensive explanations and instructions.)
    I should mention that I installed Parallels so I could run Windows and Mac OS simultaneously (it is quite amazing!). Could it be that Parallels creates an unlabeled partition?
    I don't do Windoze, but it's my understanding that Parallels makes one huge OSX file that contains the entire Windows system, rather than the way Boot Camp uses a separate partition. I also understand that this huge file is automatically excluded from Time Machine, so it doesn't back the whole thing up every time anything is changed. You can check this via TM Preferences > Options.

  • Prefered data storage method for AIR apps?

    So I'm starting on my AIR application journey, in the end I would like to develop applications for Droid.  I'm used to developing for the web and mostly use MySQL, I've been looking at SQLite and XML as data repositories.  I feel that for most of my applications at this point XML files would most likely do the job, but is there a prefered data storage method for AIR applications?
    EDIT:  I've been looking around for a primer on Flash & SQLite interaction if someone could point me in the right direction it would be very thankful.
    Thank you much!

    For document management, I believe simple file/Directory would be sufficient. have files under a file server, with your app do the proper indexing, concurrency etc from that file server folder, which shall properly stores and index the documents.

  • What Disk Allocation Units should I use, RAID0 or JBOD

    Hi, what Disk Allocation ubits should I use for drives.  The default in NTFS is 4k, is that OK or should I make them bigger?  If I use different drives to store different things should the different drives have different Allocation Units.
    I have a USB3 enclosure with 2 TB drives in it.  Would it be better to use JBOD (i.e. pass throught) so I effectivly have 2 seperate drives I can put different stuff on or RAID0 whitch gives better performance in a single volume?

    Thanks, was not sure if it should be in HW forum as it was conserning configuration of storage not chosing hardware.  Sorry about that.

  • What external Hard drive should I use for Final Cut Pro?

    Hi:
    I have a Mac Pro and I'm planning to work with HD. What external hard drive should I use?
    PD: I have a NVIDIA on a PCI Express graphic slot.
    Thanks,
    Cabezon

    You'll need to be much more specific; there are 20 or 60 different kinds of high definition. Some require nothing, some require superfast RAID, some require specialized hardware support.
    bogiesan

  • What XA JDBC driver should I use with Oracle 8i 8.1.6

    I need to use an XA JDBC driver for Oracle 8i 8.1.6. I am using the driver in
    a ConnectionPool in a TxDataSource for use by some entity beans using EJB 2.0.
    I'm running WebLogic Server 6.0 sp2, EJB 2.0, Java 2 SDK 1.3.1 Server VM, Solaris
    2.7, and Oracle 8i 8.1.6.
    I know of three different Oracle XA drivers: WebLogic jDriver for Oracle XA, Oracle
    OCI type 2 driver XA, and Oracle Thin tyoe 2 XA. Are there any other drivers
    that I can use for Oracle 8.1.6? Which driver and which version should I use?
    For example, can I use the Oracle 8.1.7 JDBC drivers with the Oracle 8.1.6 server,
    or would the Oracle 9i drivers work with the Oracle 8i server? Which drivers
    are faster, and which support more features? I assume that I should not use the
    type 4 driver since I am not using an applet, but I don't know if there are bugs
    in the OCI C library for Solaris or in the JDBC drivers that use it that prevent
    XA from woking. Which version of the Oracle C OCI library should I use if I use
    a type 2 driver?
    Thanks,
    Ross Goldberg

    Hi,
    I think you can't use the 8.1.6 thin driver for XA, you would have to
    use the OCI driver or 2PC-enable your datasource. AFAIK this has been
    changed for 8.1.7
    Check http://e-docs.bea.com/wls/docs61/////jta/thirdpartytx.html.
    Daniel
    -----Original Message-----
    From: Gene Chuang [mailto:[email protected]]
    Posted At: Thursday, August 30, 2001 2:04 AM
    Posted To: jdbc
    Conversation: What XA JDBC driver should I use with Oracle 8i 8.1.6
    Subject: Re: What XA JDBC driver should I use with Oracle 8i 8.1.6
    I've asked this question before. This is the reply from Joe
    Weinstein:
    For reliability and JDBC 2.0 compliance, I recommend the
    Oracle thin driver.
    If not that driver, you should verify that a type-2 driver
    actually does
    deliver better performance, but then I would not have a
    strong preference
    between their type-2 and ours. Theirs may be faster, and JDBC
    2.0 compliant.
    Both are succeptible to OCI bugs, some of which only they can
    fix, but we
    listen.
    Gene
    "Ross Goldberg" <[email protected]> wrote in message
    news:[email protected]...
    >
    I need to use an XA JDBC driver for Oracle 8i 8.1.6. I am using thedriver in
    a ConnectionPool in a TxDataSource for use by some entity beans usingEJB 2.0.
    I'm running WebLogic Server 6.0 sp2, EJB 2.0, Java 2 SDK 1.3.1 ServerVM, Solaris
    2.7, and Oracle 8i 8.1.6.
    I know of three different Oracle XA drivers: WebLogic jDriver forOracle XA, Oracle
    OCI type 2 driver XA, and Oracle Thin tyoe 2 XA. Are there any otherdrivers
    that I can use for Oracle 8.1.6? Which driver and which versionshould I use?
    For example, can I use the Oracle 8.1.7 JDBC drivers with the Oracle8.1.6 server,
    or would the Oracle 9i drivers work with the Oracle 8i server? Whichdrivers
    are faster, and which support more features? I assume that I shouldnot use the
    type 4 driver since I am not using an applet, but I don't know ifthere are bugs
    in the OCI C library for Solaris or in the JDBC drivers that use itthat prevent
    XA from woking. Which version of the Oracle C OCI library should Iuse if I use
    a type 2 driver?
    Thanks,
    Ross Goldberg

  • Hi, i have a A1186 Mac Pro 2008 Tower. Im running windows 7 64Bit on it. Im not sure about what version of bootcamp should i use to get the drivers. Someone please help me.

    Hi, i have a A1186 Mac Pro 2008 Tower. Im running windows 7 64Bit on it. Im not sure about what version of bootcamp should i use to get the drivers. Someone please help me.

    Boot Camp support:
    http://www.apple.com/support/bootcamp/
    Boot Camp FAQ:
    http://support.apple.com/kb/HT5639?viewlocale=en_US

  • What word processing program should I use with my Mac, what word processing program should I use with my Mac, what word processing program should I use with my Mac

    what word processing program should I use with my Mac?

    steve359 wrote:
    For free you can download NeoOffice
    Not quite. Downloading the current version of NeoOffice is definitely not free (downloading the previous version is). The developers of NeoOffice (the most Mac-like of the StarOffice descendants) make a distinction between the application itself (which is free) and downloading it, for which they require what they are pleased to call a "donation" (since it's mandatory, I would call it a fee).
    shldr2thewheel wrote:
    I totally forget about TextEdit all the time..
    Indeed. For basic WP tasks, TextEdit can be quite satisfactory.
    embauerxz
    I only need it for reports
    Would that include the report on the state of the US economy -- you know, the one President Obama asked you to deliver to the White House no later than 7AM on January 4? The one in two columns, with numbered paragraphs, table of contents, footnotes, mathematical formulae, 40 figures, 60 charts, quotations in Arabic, Japanese, and classical Greek, and 15 pages of references in APA style?
    Grant Bennet-Alder is right -- "the real answer depends on what work you expect to do with it, and with whom you expect to do this work". There are dozens of word processors for Mac, from the venerable Word to the no less venerable Tex-Edit+. It would pointless to start listing them all, when many of them might be either too little or too much for your needs.

  • What frame rate timeline should I use for 1080p 60fps premiere pro

    When using 1080p 60fps clips what sequence should I use when creating a timeline that would work the best for slow mo application?
    I read that you should use a 24 fps timeline but i'm not sure?
    Also I have a few 720 @ 120 fps, should I create a separate sequence for that and what kind of timeline should I use since cs5 only has 1080 @ 60fps option?

    24 is a good frame rate for most animations but some people like 30 and for the max stress on user systems and animation smoothness, use 60.

  • What you tube app should I use on my ipad2?

    What you tube app should I use on my ipad2?

    Google's official Youtube App is the recommended one.
    https://itunes.apple.com/en/app/youtube/id544007664?mt=8
    But there are many othe rApps yu can use, it depends on what functionality you want.
    Just search the store for the many alternatives.

  • What malware scanning software should I use

    What malware scanning software should I use?  Any suggestions for good free software?  Also is there a good free antivirus software too?
    I use malwarebytes & FEP for Windows 7 pc but need to know what to install of this mac.
    Thank you

    None is really needed at present.
    Helpful Links Regarding Malware Protection
    An excellent link to read is Tom Reed's Mac Malware Guide.
    Also, visit The XLab FAQs and read Detecting and avoiding malware and spyware.
    See these Apple articles:
              Mac OS X Snow Leopard and malware detection
              OS X Lion- Protect your Mac from malware
              OS X Mountain Lion- Protect your Mac from malware
              About file quarantine in OS X
    If you require anti-virus protection I recommend using ClamXav.

Maybe you are looking for