SCHEMA: Flat vs. hierarchical

Question

All,
I'd like your feedback on another topic relating to the schema, this
one at a higher level. Basically, the question is whether you think we
should adopt a flat schema, or a hierarchical schema. To explain:
- With a flat schema, we assign some number of "buckets" to hold event
data, give a name to each bucket, and then for any event that comes in
we extract the relevant data and drop it into that bucket
- With a hierarchical schema, we would have a set of "objects" with
attributes that we could attach to events, and then for any given event
that we receive we extract the relevant data and set the various
attributes of the various objects.
You may have already noted that to some extent we already have a
hierarchical schema, in that we define four top-level containers - the
Initiator, Action, Target, and Observer - and then we have some
pseudo-objects under those like User, Host, etc. Right now we sort of
"fake" an object-based schema by using CamelCase, e.g. InitUserName,
InitUserDomain, etc. With a true hierarchical schema, we'd actually have
an object calling Initiator, which might have a child object called
User, which might have attributes Name, Domain, ID.
Please again ignore details about how this would actually be
implemented; you could always convert from one to the other for internal
storage if need be. Instead focus on whether it would be easier to
access the data you want to see by using a hierarchical model or a flat
model.
These are the pros and cons as I see it:
- The flat schema is a little easier to display in a table and easier
to read in a single-line format, but on the other hand the object schema
yields much more interesting, interactive displays (the SLM event
display, again, sort of "fakes" an object schema but putting the Init*
fields at left and the Target* fields at right.
- If we go to an object schema, we could actually reference almost any
type of object we wanted by re-using something like DMTF's Common
Information Model, which describes virtually any manageable IT resource.
Right now if we want to include, say, a MAC address and we hadn't
thought about that before, we have to completely revise our flat schema
and define a new field. The potential downside is that not every event
would then have even a standard set of fields if the values for those
fields were null.
- The other downside of course is how we migrate from one schema to
another if we fundamentally change how this works. I think this is
do-able, however, if we come up with some migration plans and perhaps
support both models in some way for a while (the flat schema, for
example, is just a representation of some subset of object attributes).
So let's give some examples. Let's say that we have a user opening a
file - simple enough. In a flat schema, this might look like:
{ "InitUserName": "user2", "InitUserID": "104", "InitHostName": "dc01",
"TargetDataName": "syslog-ng.conf", "TargetDataContainer":
"/etc/syslog", "TargetHostName": "dc01" }
Note that it's a little ambiguous that this particular username
represents an account on host 'dc01'
But in an object schema, this might look like:
{ Initiator: { Account: { Name: user2, UserID: 104, Host:
dc01}},
"Target": { "File": { "Name": "syslog-ng.conf", "Container":
"/etc/syslog", "Host": "dc01"}}}
OK, so what do you all think?
DCorlette
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=419793

Answer 1

Hi Rokie,
  What's the difference between Flat & Qualified Flat table?    
    Genarally to avoid the redundancy of data, both  Lookup[Flat] and Qualified Flat will be used.
    But  What's the difference between Flat & Qualified Flat table?
    Lookup[Flat] contains some set of legal values which can be shared by main table .But in some cases
    we may have the data in some fields changes frequently based on other fields in that table.
    Suppose Price of a particular product changes based on region.Here what we need to understand is
    there are multiple values for each main table product.Obvisouly it increases redundancy of data.
    Qualifer table contains qualifier and non-qualifier fields.
    Data which is frequently changes based on other fields considered as Qualifier fields.
    Non-Qualifier fields contains the data which is responsible of changes in qualifier fields.
     I am giving differences between Lookup[Flat] and Qualified lookup.
     1) Lookup[Flat] contains predefined set of records (Ex: List of manufactures for particular product)
        whereas Qualified lookup contains conditional set of records (Ex: Pricing based on manufaturer and
         region)
   2) Lookup within lookup is possible whereas qualified lookup is possible only on main table
   3) Lookup table record need to be maintained  before main table record  whereas qualified record is
      maintained  after main table gets created.
  4) Lookup[Flat] contains relatively small no of records when compared to main table whereas qualified
     lookup contains large no of records when compared to main table.
    For better understanding, go through the above blogs given by others.
   Hope it helps,
   Reward points,if found useful.
   Thanks,
   Narendra.M