A robust data structure for a histogram of events over time.

Hi all,
I've been thinking for days, I am still lost on how to solve the following:
Let's say that we have a list of events that occur over a certain time interval. Each event has a start timestamp and a finish timestamp. The events occur in no particular order and expand over various length of periods.
With respect to the histogram, we mark the time period of an event occuring with a frequency of one. When events overlap over a time period, we mark that time period with a frequency of number of overlapping events.
Given this raw data, we want to examine the data's frequency under various granularity such as second, minutes, hours, days, weeks, month and years for 60sec, 60min, 24 hrs, 7 days, 4 wks ranges respectively.
So, what is a good intermediate (general) data structure that can store all the various events such that they can be easily transformed to a specific granularity. My goal here is to read the event list once, and histogram of events data with respect to various timeframe can be calculated.
Does a timeline like data strucuture that stores aggregated frequencies solve my problem? With a fine granuarlity say seconds and a long time range, the data structure will bound to overflow.
Any help is greatly appreciated!
Brian

You data structure would be in terms of Classes. You need a class Anevent{
String eventName;
Date startTime;
Date endTime;
In your main program have a Vector object which can store unllimited objects and which would not overflow. Store your event objects as they happen in this Vector. After your time interval of events is completed and you have the final Vector, you can pass the vector object to various methods in your main program to be simple or pass it to specailized graph drawing classes you would have to write to draw various Histograms.

Similar Messages

  • Data structure for web-based discussion forum

    Hello all. I have begun work on a discussion board forum but have not yet decided upon a data structure for the hierarchy of messages to be left on a board. Any one have any thoughts or experience in this matter? My first thought was to implement a basic linked list in which every reply to a message would be a new child of a node. This would require I use recursion in some areas for counting and finding messages but I was also thinking that this way I can store each thread object in a simple fashion using serialization.
    SJ

    THE STANDARS DIRECTORY STRUCTURE IS :-
    PROJECT FOLDER
    |
    |---------->WEB-INF
    |------> web.xml file
    |-------> classes (folder to keep java files)
    |-------> lib (folder to keep jar files)

  • Looking for suitable and robust data structure?

    I have an application where i have to give user an option to correlate the
    response returned by a method call with other methods arguments.
    I have provided a jtree struture where user can see methods and their
    responses and can correlate them by selecting appropriate arguments
    where these values could be used.
    I used a data structure to store such relations like
    A hashmap where every key is a unique number representing method number
    and value ia a matrix.
    In each matrix, the first element of every row would be the Response object and
    all remaining elments in the row would be Argument objects. :)
    Now maybe someone would ask, for every method there is always only one return type
    then why such a matrix with Reponse object as first element of each row.
    The reason is that i gave user an option to not only relate response as a whole Object to
    different arguments of different methods but user can select a field of this response object if this
    is a complex type or user can select any element if the response is a Collection object to relate
    with arguments.
    I am not sure whether this structure is more appropriate or i can use more robust
    and memory efficient structure.
    Please give me your useful suggestions.

    J2EER wrote:
    I have an application where i have to give user an option to correlate the
    response returned by a method call with other methods arguments.I really don't understand you.
    I have provided a jtree struture where user can see methods and their
    responses and can correlate them by selecting appropriate arguments
    where these values could be used.Huh?
    I used a data structure to store such relations likeErr, what happened to the rest of your sentence? Or is the next part the rest of it?
    A hashmap where every key is a unique number representing method number
    and value ia a matrix.Where does this matrix come from? In my book, a matrix is a 2 dimensional structure. What do you mean by it?
    In each matrix, the first element of every row would be the Response object and
    all remaining elments in the row would be Argument objects. :)
    Now maybe someone would ask, for every method there is always only one return type
    then why such a matrix with Reponse object as first element of each row.You're still talking about methods? If so, don't all methods have just one return type?
    The reason is that i gave user an option to not only relate response as a whole Object to
    different arguments of different methods but user can select a field of this response object if this
    is a complex type or user can select any element if the response is a Collection object to relate
    with arguments.
    I am not sure whether this structure is more appropriate or i can use more robust
    and memory efficient structure.
    Please give me your useful suggestions.Perhaps you could give a more detailed explanation and more importantly, provide some examples of what you mean.

  • What is the best data structure for loading an enterprise Power BI site?

    Hi folks, I'd sure appreciate some help here!
    I'm a kinda old-fashioned gal and a bit of a traditionalist, building enterprise data warehouses out of Analysis Service hypercubes with a whole raft of MDX for analytics.  Those puppies would sit up and beg when you asked them to deliver up goodies
    to SSRS or PowerView.
    But Power BI is a whole new game for me.  
    Should I be exposing each dimension and fact table in the relational data warehouse as a single Odata feed?  
    Should I be running Data Management Gateway and exposing each table in my RDW individually?
    Should I be flattening my stars and snowflakes and creating a very wide First Normal Form dataset with everything relating to each fact? 
    I guess my real question, folks, is what's the optimum way of exposing data to the Power BI cloud?  
    And my subsidiary question is this:  am I right in saying that all the data management, validation, cleansing, and regular ETTL processes are still required
    before the data is suitable to expose to Power BI?  
    Or, to put it another way, is it not the case that you need to have a clean and properly structured data warehouse
    before the data is ready to be massaged and presented by Power BI? 
    I'd sure value your thoughts and opinions,
    Cheers, Donna
    Donna Kelly

    Dear All,
    My original question was: 
    what's the optimum way of exposing data to the Power BI cloud?
    Having spent the last month faffing about with Power BI – and reading about many people’s experiences using it – I think I can offer a few preliminary conclusions.
    Before I do that, though, let me summarise a few points:
    Melissa said “My initial thoughts:  I would expose each dim & fact as a separate OData feed” and went on to say “one of the hardest things . . . is
    the data modeling piece . . . I think we should try to expose the data in a way that'll help usability . . . which wouldn't be a wide, flat table ”.
    Greg said “data modeling is not a good thing to expose end users to . . . we've had better luck with is building out the data model, and teaching the users
    how to combine pre-built elements”
    I had commented “. . . end users and data modelling don't mix . . . self-service so
    far has been mostly a bust”.
    Here at Redwing, we give out a short White Paper on Business Intelligence Reporting.  It goes to clients and anyone else who wants one.  The heart
    of the Paper is the Reporting Pyramid, which states:  Business intelligence is all about the creation and delivery of actionable intelligence to the right audience at the right time
    For most of the audience, that means Corporate BI: pre-built reports delivered on a schedule.
    For most of the remaining audience, that means parameterised, drillable, and sliceable reporting available via the web, running the gamut from the dashboard to the details, available on
    demand.
    For the relatively few business analysts, that means the ability for business users to create their own semi-customised visual reports when required, to serve
    their audiences.
    For the very few high-power users, that means the ability to interrogate the data warehouse directly, extract the required data, and construct data mining models, spreadsheets and other
    intricate analyses as needed.
    On the subject of self-service, the Redwing view says:  Although many vendors want tot sell self-service reporting tools to the enterprise, the facts of the matter are these:
    v
    80%+ of all enterprise reporting requirement is satisfied by corporate BI . . . if it’s done right.
    v Very few staff members have the time, skills, or inclination to learn and employ self-service business intelligence in the course of their activities.
    I cannot just expose raw data and tell everyone to get on with it.  That way lies madness!
    I think that clean and well-structured data is a prerequisite for delivering business intelligence. 
    Assuming that data is properly integrated, historically accurate and non-volatile as well, then I've just described
    a data warehouse, which is the physical expression of the dimensional model.
    Therefore, exposing the presentation layer of the data warehouse is – in my opinion – the appropriate interface for self-service business intelligence.
    Of course, we can choose to expose perspectives as well, which is functionally identical to building and exposing subject data marts.
    That way, all calculations, KPIs, definitions, and even field names, and all consistent because they all come from the single source of the truth, and not from spreadmart hell.
    So my conclusion is that exposing the presentation layer of the properly modelled data warehouse is – in general - the way to expose data for self-service.
    That’s fine for the general case, but what about Power BI?  Well, it’s important to distinguish between new capabilities in Excel, and the ones in Office 365.
    I think that to all intents and purposes, we’re talking about exposing data through the Data Management Gateway and reading it via Power Query.
    The question boils down to what data structures should go down that pipe. 
    According to
    Create a Data Source and Enable OData Feed in Power BI Admin Center, the possibilities are tables and views.  I guess I could have repeating data in there, so it could be a flattened structure of the kind Melissa doesn’t like (and neither do I). 
    I could expose all the dims and all the facts . . . but that would mean essentially re-building the DW in the PowerPivot DM, and that would be just plain stoopid.  I mean, not a toy system, but a real one with scores of facts and maybe hundreds of dimensions?
    Fact is, I cannot for the life of me see what advantages DMG/PQ
    has over just telling corporate users to go directly to the Cube Perspective they want, that has already all the right calcs, KPIs, security, analytics, field names . . . and most importantly, is already modelled correctly!
    If I’m a real Power User, then I can use PQ on my desktop to pull mashup data from the world, along with all my on-prem data through my exposed Cube presentation layer, and PowerPivot the
    heck out of that to produce all the reporting I’d ever want.  It'd be a zillion times faster reading the data directly from the Cube instead of via the DMG, as well (I think Power BI performance sucks, actually).
    Of course, your enterprise might not
    have a DW, just a heterogeneous mass of dirty unstructured data.  If that’s the case,
    choosing Power BI data structures is the least of your problems!  :-)
    Cheers, Donna
    Donna Kelly

  • Data structure for simulation of message queue

    Hello,
    I have undertaken a project of simulating the point to point and publish/subscribe protocols of message queueing. This whole project would be done just in Java. There won't be any system level programming. Which data structure in Java would be the most efficient one for the message queue?

    Hello,
    I have undertaken a project of simulating the point to point and publish/subscribe protocols of message queueing. This whole project would be done just in Java. There won't be any system level programming. Which data structure in Java would be the most efficient one for the message queue?

  • Data Structure for New General Ledger for mySAP 2004

    Guys,
    We are preparing to upgrade to mySAP 2004.
    mySAP 2004 doesn't have data migration tool for upgrade "classic" ledger to new ledger with advanced functionality. To utilize all new functionality, data should be migrated to the new ledger. Data migration tool in this case should be developed (ABAP). What is the difference in data structure of new ledger?
    if anybody who already went thru upgrade and could share some examples of programs for mySAP 2004 upgrade, my e-mail : [email protected]
    Thanks in advance,
    Mike

    Now I have the same problem. Anyone has a solution for this matter?

  • Open ended data structure for retrieving SQL data

    I have a flex project that reads data from a database and
    sends it to my flex app to display in a chart. I need to do this in
    Actionscript since I have to dynamically create different charts
    based on the database definitions. Up til now, I have inserted the
    data for each series into a separate
    ArrayList(java)/ArrayCollection(flex) in this way:
    BindingUtils.bindProperty(lineSeries, "dataProvider", series,
    "pointList");
    lineSeries.xField="point1";
    lineSeries.yField="point2";
    The series variable is a custom Actionscript/Java object
    which contains an ArrayCollection called pointList. And the array
    is made up of custom DataObjects with two fields, point1 &
    point2.
    But in the new paradigm, I'd basically like to create a big
    array of arrays similar to a database table and then just point the
    xField to one column, and the yField to another column. Anyway, it
    seems like the problem is that my point1 and point2 in the current
    implementation are variable names, but with this new dynamic
    structure, I won't be able to create variables on the fly. So how
    do I let flex know to look at column 3, for example for the yField?
    The attached code is from the Flex documentation and is an example
    of what I'd like to emulate (except I need to do it in
    actionscript). I'd like to be able to select "month" or "amount",
    etc. But coming from the Java side, there is no way for me to name
    the array contained in each column.

    I guess I wasn't clear enough. I am actually serializing a
    java object and mapping it to an actionscript object via blazeDS. I
    know about how the objects translate, for example for
    ArrayCollections in flex, I use ArrayLists in java. The problem is
    that I'm not sure how to get the associative aspects of the arrays
    in flex into my java data structures in a way that will translate
    to the way flex can define arrays, such as Month:"January".
    It seems that in flex, you can create an Array and assign it
    an element like {month:"January", amount:"450"} and it will create
    an Object with variables for month and amount. But I don't know how
    to do this on the fly in Java (or in Actionscript for that matter)
    when I have no way of knowing beforehand how many variables my
    object will need. Each series will need an x and y, though usually
    the x will be dates which are the same for each series. So if I had
    10 series sharing an axis, I'd need 11 variables in my object-
    date, series1, series 2...series 10.

  • What is the new data structure for .RTM files in LV8?

    We have software that translates our LV
    program to another language.  Part of that software package reads RTM
    files, decodes them, replaces text, and resaves them.  That code is
    broken for menus that are saved in LV8+, however.  What is the new data
    structure?
    The simple attached zip file has vi's which
    are not ours but serve to illustrate the point.  If it's run on an
    older RTM file, it correctly outputs and XML version of it, on newer
    RTM files it just throws an error.
    -- This is the second time I posted this.  I was curious what the Solution? icon did on somebody elses post and I apparently marked the whole thread as solved (there should really be an undo on that feature btw --
    Attachments:
    RunTime_Menu_to_XML.zip ‏33 KB

    Hi Thomas,
     What error are you getting when running the program?  I was actually able to run it without errors in LV 8.6.  
    There are ways to programmatically read and customize RTM files in LabVIEW.  I hope that the following links will help:
    http://zone.ni.com/reference/en-XX/help/371361B-01/lvhowto/customizing_shortcut_menus_programm/
    http://digital.ni.com/public.nsf/allkb/17803AA31C8C07C986256CFD0080D609?OpenDocument
    Cheers, 
    Marti C
    Applications Engineer
    National Instruments
    NI Medical

  • Storing a lot of data in an indexed data structure for quick access.

    I'm designing an app. which will need to store a large amount of data in memory. Records will be flowing into the app. via a socket. The app will receive about 30 records/second which is about 108,000 records/hour and about 600,000 records/day. I need to store the records in an indexed data structure so that I can access them quickly. For example, at 9:00am I will need to access records received at 8:30am, 8:35am, 8:40am, etc. This program will be multithreaded and as I understand Vector is the only data structure that is thread safe. Is Vector my only choice? How do I access objects in a Vector using an index? Is there something better that I can use?

    Is Vector my only choice?If you want to access the objects by key then you should use something like a HashMap. But if you want to access them by an array index then an ArrayList would be more appropriate.
    as I understand Vector is the only data structure that is thread safeYou can get a thread-safe version of any Collection object by using the Collection.synchronizedCollection method.
    How do I access objects in a Vector using an index? I'd suggest you read the API documentation. And probably the Sun tutorial on Collections at http://java.sun.com/docs/books/tutorial/collections/index.html
    600,000 records/day. Unless you plan to dump old data after a short period of time, you may want to consider using a database to avoid running out of memory.

  • Best data structure for tablemodel

    Hi everybody
    I'm building an application that has a JTable which gets its data from a resultSet. I'd like to know which data structure is the best for this situation. I've seen some examples that passes the data from the resultset to a Vector or an ArrayList, but if you have a scroollable resultset, why will I instantiate another data structure?
    Thanks very much
    Mauricio

    Hi everybody
    I'm building an application that has a JTable which gets its data
    from a resultSet. I'd like to know which data structure is the best
    for this situation. I've seen some examples that passes the data from
    the resultset to a Vector or an ArrayList, but if you have a
    scroollable resultset, why will I instantiate another data structure?
    Thanks very much
    Mauricio I've used TableModels that get their data directly form the ResultSet and had some success. But you are at the mercy of the implementation of the ResultSet. If it is slow your table model will be slow too. I would suggest that you use an array of Object arrays, i.e. Object[][], as your data stucture and copy your result set into it.

  • What is the best type of data structure for this type of data?

    Animal --> (List of types like Mammal, Reptile, etc)
    Mammal --> (List of families)
    Etc.
    So basically there is a key then a list of other keys with more items.
    At first I thought a Map<String, Map<String, Map<String,...etc but the number is unknown so I'm not sure how to structure it.

    I'd just use a flat Map<String,String> and stick all categories in that

  • Soap2JDBC : Problem while designing data structure for Query

    Hi ALL,
    I am doing webservices(SOAP) to JDBC(syn) scenario without using BPM.We are sending the request from portal and getting the response from database.
    I have lot of Queries.I am going to design based on the database Query.Please find the following Query.
    SELECT ALL DBTEST.REQMASTBL.RQTR,
    DBTEST.REQMASTBL.RQN_NO,
    DBTEST.REQMASTBL.RQN_DATE,
    DBTEST.REQMASTBL.APPROVE_BY,
    TO_NUMBER(DBTEST.REQMASTBL.RQN_AMT), DBTEST.REQMASTBL.RQN_TYPE,
    DBTEST.REQMASTBL.ROS_DATE,
    DBTEST.REQMASTBL.RQN_STATUS
    FROM DBTEST.REQMASTBL
    WHERE DBTEST.REQMASTBL.RQN_STATUS='AP'  AND
    DBTEST.REQMASTBL.APPROVE_BY LIKE UPPER('GM%') AND DBTEST.REQMASTBL.RQTR='GM211' AND DBTEST.REQMASTBL.RQN_DATE BETWEEN (:frmDate) AND (:toDate) ORDER BY  DBTEST.REQMASTBL.RQN_NO
    All most all the queries having the same functionality.Could anyone please guide me how to design and how to map the source and the target fields using the Boolean(and,or.like,between) functions.
    Now the problem is How to design the data stucture in Integration Repository using this query and how to extract these(and,or.like,between) functions.
    urgent response is highly appreciated
    rgds,
    Veena

    Hi Gurus
    Iam getting the error like this
    HTTP error:could not post file
    '/XISOAPAdapter/MessageServlet?channel=:BS_SS:soap_communicationchannel&amp;version=3.0&amp;Sender.service=BS_SS&amp;Interface=http%3A%2F%2Fsoap2db1%5Eportalsoap_outbound_messinterface' on server
    this is the message iam getting from Portal side.Here iam testing using altova xml spy tool.
    in this context 
    BS_SS      is my sender business system
    soap_communicationchannel  i   s my sender communication channel
    Sender.service=BS_SS      (again my sender business system)
    portalsoap_outbound_messinterface       is my message interface
    I badly required the help from experts.
    Regards
    Veena

  • How To... Change the Data Type for a SELECT-OPTIONS at run time.

    Hello,
    I am trying to restrict the values available for entry into a SELECT-OPTIONS at run time depending on user input.
    The logic is as follows. The user has two input fields. A PARAMETER field which has the type RSDIOBJNM and allows them to choose an InfoObject. And the user has a SELECT-OPTIONS field to allow them to select the Characteristic values for that InfoObject.
    I would like the following example to be possible:
    The user enters 0MATERIAL into the PARAMETER. When the user clicks on the SELECT-OPTIONS control code will derive a list of possible options the user can enter in the SELECT-OPTIONS. In this case only values found in the master data or at least no values greater than 18 characters.
    I have looked at the following function module SELECT_OPTIONS_RESTRICT and this do not appear to be helpful as they only restrict on the signs allowed for the values (unless I misunderstand, it is a complex function module!).
    The code I have so far is (thus the user enters a InfoObject into p_char1 and the select options so_char1 should only accept active values of that InfoObject):
    declaration of variables for user interface
      DATA c_char(32) TYPE c.
    declaration of count variable
      DATA i_count TYPE i.
    declaration of user interface
      SELECTION-SCREEN BEGIN OF BLOCK a1 WITH FRAME TITLE text-001.
        PARAMETERS: p_ipack TYPE RSLOGDPID.
      SELECTION-SCREEN END OF BLOCK a1.
      SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-002.
        PARAMETERS: p_char1 TYPE RSDIOBJNM.
        SELECT-OPTIONS: so_char1  for c_char NO INTERVALS.
        PARAMETERS: p_char2 TYPE RSDIOBJNM.
        SELECT-OPTIONS: so_char2  for c_char NO INTERVALS.
        PARAMETERS: p_char3 TYPE RSDIOBJNM.
        SELECT-OPTIONS: so_char3  for c_char NO INTERVALS.
      SELECTION-SCREEN END OF BLOCK b1.
    Is what I am trying to do possible???
    Thanks for any help. Ross.

    You really want to restrict possible values of a select-option based on another field, not change the length of type of the select-option field, right?
    Here is what you do:  Code a custom F4 value help for the select-option at event AT SELECTION-SCREEN ON VALUE REQUEST FOR..  The first thing you do here is read the value of the parameter field (p_ipack in your example).  You can use function module DYNP_VALUES_READ.  Based on this value, you can propose values for the select-option fields.  Note that the use can still enter whatever s(he) wishes in to the select-option field without pressing F4. In this case, you will have to code some input validations taking into account the value in the p_ipack field.

  • Firefox for Android becomes painfully slow over time. Clearing data fixes, but erases bookmarks. Where is the "Reset" function ??

    After a fresh install, Firefox loads websites very fast with almost no lag. After a few weeks, it becomes slower and slower. Getting to the point where it wil take aboit 20 seconds to display the Top Sites/Bookmarks window. Then sometimes opening a page, the loading circle stops moving and it freezes the phone. After about 15 seconds everything works again. The only way I found to fix this horrendous problem is to go into control panel for Firefox and Clear Data. I dont like doing that because I lose all of my bookmarks. It seems the Cache always shows "0 bytes" so there never is nothing to clear out. I would like to know where this "Reset" function is for Android. I see they have it for Windows versions..but what about Android?
    My phone is an HTC Rezound running stock Android 4.0.3
    Thanks in advance.

    Bruce,<br/>whilst waiting for Roland to get back you may wish to try this
    Enter into the address bar ''about:memory'' you get interactive dynamic information about memory usage, and maybe also ''about:compartments'' .
    That should at least help you see where the memory is going to.

  • Hints how to get a report for "number of active threads over time", e.g.?

    Hi there,
    We drive a number of WLS setups monitored using Grid Control 11g, and due to the usual economic aspects up to four WLS domains on a single system. To track the development of the load for each single domain we would like to have a report for the active threads, CPU e.g.. In the list of the pre-defined reports we can't find anything matching, so any hints for us based on someone's experience?
    TIA
    Oliver

    Hi Oliver,
    I would suggest you to have a look at the below WLST script which would give you the required report of the active threads and it would be send an email too.
    Topic: Sending Email Alert for Threads Pool Health Using WLST
    http://middlewaremagic.com/weblogic/?p=5433
    Topic: Sending Email Alert for Hogger Threads Count Using WLST
    http://middlewaremagic.com/weblogic/?p=5423
    Also you can use the below script in case of the stuck threads, this script would send you an email with the thread dumps during the issue occurred.
    Topic: Sending Email Alert For Stuck Threads With Thread Dumps
    http://middlewaremagic.com/weblogic/?p=5582
    Regards,
    Ravish Mody

Maybe you are looking for