Find columns which are null

Hello all, I am currently developing a data warehouse that holds time-series values. In the 5-min interval fact table, there is a column for every 5-minute segment of a 12 hour period. So I have a few foreign keys which make up a composite primary key in the fact table, followed by generally-named columns m0,m5,m10...m1145,m1150,m1155 (m0 represents 00:00 [12:00] and m1155 represents 11:55). There is a flag in the row to denote AM or PM. This database collects various metrics from a Coherence environment for a live monitoring solution that also has a Historian to track trends, get a graphical historical overlay against the current values coming into the live environment, etc.
So I have this table with aggregation at a 5 minute level and will need to roll this up to hourly and daily aggregation levels during an ETL process. I'd like to be able to pinpoint which data, if any, has holes in it IE the collectors hiccupped somewhere for whatever reason and did not report a 5 minute segment to me.
Is there anyway to find rows which have null values and to report back which column names contain null values? A use case would be that I have a row in the 5 min fact table that contains a null value for column m455, which means that there is a gap in my time series at the 4:55 hour for that object. During ETL roll ups, I'd like to be able to record which time slot has "holes", or null values in them so that we can pinpoint why they occur to prevent them from occurring again during the development process of this solution.
Thanks for anyone's help!!
Regards,
TimS

TimS wrote:
create table f_metric_5_min (
     metric_key number(10) not null,
     object_key number(19) not null,
     date_key number(5) not null,
     segment_id number(1) not null,
     m0 number(19,4),
     m5 number(19,4),
     m10 number(19,4),
     m15 number(19,4),
     m20 number(19,4),
     m25 number(19,4),
     m30 number(19,4),
     m35 number(19,4),
     m40 number(19,4),
     m45 number(19,4),
     m50 number(19,4),
     m55 number(19,4),
     m100 number(19,4),
     m105 number(19,4),
     m110 number(19,4),
     m115 number(19,4),
     m120 number(19,4),
     m125 number(19,4),
     m130 number(19,4),
     m135 number(19,4),
     m140 number(19,4),
     m145 number(19,4),
     m150 number(19,4),
     m155 number(19,4),
     m200 number(19,4),
     m205 number(19,4),
     m210 number(19,4),
     m215 number(19,4),
     m220 number(19,4),
     m225 number(19,4),
     m230 number(19,4),
     m235 number(19,4),
     m240 number(19,4),
     m245 number(19,4),
     m250 number(19,4),
     m255 number(19,4),
     m300 number(19,4),
     m305 number(19,4),
     m310 number(19,4),
     m315 number(19,4),
     m320 number(19,4),
     m325 number(19,4),
     m330 number(19,4),
     m335 number(19,4),
     m340 number(19,4),
     m345 number(19,4),
     m350 number(19,4),
     m355 number(19,4),
     m400 number(19,4),
     m405 number(19,4),
     m410 number(19,4),
     m415 number(19,4),
     m420 number(19,4),
     m425 number(19,4),
     m430 number(19,4),
     m435 number(19,4),
     m440 number(19,4),
     m445 number(19,4),
     m450 number(19,4),
     m455 number(19,4),
     m500 number(19,4),
     m505 number(19,4),
     m510 number(19,4),
     m515 number(19,4),
     m520 number(19,4),
     m525 number(19,4),
     m530 number(19,4),
     m535 number(19,4),
     m540 number(19,4),
     m545 number(19,4),
     m550 number(19,4),
     m555 number(19,4),
     m600 number(19,4),
     m605 number(19,4),
     m610 number(19,4),
     m615 number(19,4),
     m620 number(19,4),
     m625 number(19,4),
     m630 number(19,4),
     m635 number(19,4),
     m640 number(19,4),
     m645 number(19,4),
     m650 number(19,4),
     m655 number(19,4),
     m700 number(19,4),
     m705 number(19,4),
     m710 number(19,4),
     m715 number(19,4),
     m720 number(19,4),
     m725 number(19,4),
     m730 number(19,4),
     m735 number(19,4),
     m740 number(19,4),
     m745 number(19,4),
     m750 number(19,4),
     m755 number(19,4),
     m800 number(19,4),
     m805 number(19,4),
     m810 number(19,4),
     m815 number(19,4),
     m820 number(19,4),
     m825 number(19,4),
     m830 number(19,4),
     m835 number(19,4),
     m840 number(19,4),
     m845 number(19,4),
     m850 number(19,4),
     m855 number(19,4),
     m900 number(19,4),
     m905 number(19,4),
     m910 number(19,4),
     m915 number(19,4),
     m920 number(19,4),
     m925 number(19,4),
     m930 number(19,4),
     m935 number(19,4),
     m940 number(19,4),
     m945 number(19,4),
     m950 number(19,4),
     m955 number(19,4),
     m1000 number(19,4),
     m1005 number(19,4),
     m1010 number(19,4),
     m1015 number(19,4),
     m1020 number(19,4),
     m1025 number(19,4),
     m1030 number(19,4),
     m1035 number(19,4),
     m1040 number(19,4),
     m1045 number(19,4),
     m1050 number(19,4),
     m1055 number(19,4),
     m1100 number(19,4),
     m1105 number(19,4),
     m1110 number(19,4),
     m1115 number(19,4),
     m1120 number(19,4),
     m1125 number(19,4),
     m1130 number(19,4),
     m1135 number(19,4),
     m1140 number(19,4),
     m1145 number(19,4),
     m1150 number(19,4),
     m1155 number(19,4),
     constraint f_metric_5_min_pk primary key (metric_key, object_key, date_key, segment_id),
     constraint f_metric_5min_fk_metric_key foreign key (metric_key) references d_metric (metric_key),
     constraint f_metric_5min_fk_object_key foreign key (object_key) references d_object (object_key),
     constraint f_metric_5min_fk_date_key foreign key (date_key) references d_date (date_key)
) partition by range(date_key) (
     PARTITION def values less than (2558)
) tablespace FACT;The reason to define separate columns for time intervals is because need to see data coming from Coherence enviroment in 5 minute intervals, creating a row for each 5 min interval for each object (thousands) is too time-consuming. Using MERGE is much faster to update columns in the same row. And secondly, because my job tells me to go with this logical model :).
I can understand the "job-tells-me" reason but not sure am convinced with the "time-consuming" issue,
especially as it is a datawarehouse. But I assume you must have verified the same.:)
The NULL-finding exercise will be done after the data is loaded into the 5 minute metric table and before the hourly roll-up ETL.
Not sure what you mean in what format, I plan to build a list and place them into a form of auditing table within the database that will be queried in the future to determine where the gaps are and for what objects so that we can trace them back to a certain JVM and coherence collector and fix the issue from occuring again; hence the point of a monitoring tool.
Storing time in a table can take up a lot of space and is harder to compute procedurally. In this model the database does not care about the time, only the data; the date and time is not a focal point for this database, the database is abstract, with the Java behind it controlling the purpose of its data. The reporting engine after-the-fact cares about dates and such which can be easily determined by referencing to dimension tables to determine which column represents which time in the 12 hour sequence.
Going back to the "null-finder" do you think there is any way to do this? Thanks for your help!
Regards,
TimSDo you mean that you want to get only those columns where there is a NULL value, for a record-at-a-time ?
In that case, the only option that I can think of is building a Dynamic SQL, based on checking whether each column has a NULL value or not. Although, I hope somebody can come up with a better way to achieve the same.
(Else, your not considering "time-consuming" approach above becomes not much of advantage, isn't it ?) :)
(Of course, only if "null-finding" is a MUST exercise...)

Similar Messages

APEX don't allow modify rows in columns which are Primary Key ?

I have tables:
http://img508.imageshack.us/my.php?image=21269582oe8.jpg
book(id_book - "Primary Key", title, year); book_author(id_book - "Primary Key", id_author - "Primary Key"); author(id_author - "Primary Key", name)
I have created a new page -> Form -> Tabular Form for table 'author' because I would like to add new authors, modify and delete. During creating this page I have choosen column 'id_author' as 'primary key column 1' and everything is OK (I can't modify column 'id_author' - this column is autoincrement and I can modify column 'name').
BUT I have also created a new page -> Form -> Tabular for table 'book_author' because I would like to write numbers as id_book and id_author, modify and delete them (so add relations between tables: book, book_author and author). During creating this page I have choosen column 'id_book' as 'primary key column 1' and 'id_author' as 'primary key column 2'. And on website I can't modify (edit) these fields. And I can't also add new row because I see in each column in new row: (null).
http://img444.imageshack.us/my.php?image=11324615yk9.jpg
APEX don't allow modify rows in columns which are Primary Key ? This is stupid....... What can I do ?
Edited by: user10731158 on 2008-12-20 11:40

Single-column and non-meaningful so you never want to update them. In the case of your example, you would need to add an ID column to the book_author intersection table. Honestly, I was so blown away (and pleasantly surprised) by the lack of rebuttal and the "thx" that I went ahead and put together an example of how I would define the book_author table:
create table book_author
   (id varchar2(32),
    book_id varchar2(32),
     author_id varchar2(32),
     modified_on     date,
     modified_by varchar2(255),
     constraint book_author_pk primary key (id),
     constraint book_auth_book_fk foreign key (book_id) references books(id),
     constraint book_auth_author_fk foreign key (author_id) references authors(id)
create unique index book_author_uq on book_author (book_id,author_id)
create or replace trigger biu_book_author before insert or update on book_author
for each row
begin
     if inserting then
          :new.id := sys_guid();
     end if;
     modified_on := sysdate;
     modified_by := nvl(v('APP_USER'),user);
end;
/Good luck,
Tyler

Trying to use 2 different Dimension tables and make a hierarchy on some columns which are split into these dimensions .. how do I do that

Trying to use 2 different Dimension tables and make a hierarchy on some columns which are split into these dimensions .. how do I do that

If you need to make a hierarchy in an Attribute view you need to have all relevant fields/columns in the same Dimension table..

Where can I find out which areas in London Actuall...

Where can I find out which areas in London Actually have got the Fibre Optics promised last year? I have tried with bt and on the net but cant actually find anything that would tell me where the elusive fibre optics are. I am being asked all the time for a postcode and if I knew that I would be living there.

The problem you're going to have, is not only finding where it's available but also if there are any connections still available in the local cabinet. I'm in London, in a slow-spot (500kpbs) for normal broadband, and when the engineer did my Infinity install last Friday he told me that I got one of the last connections available (they only put in a 100, based apparently on take-up in other areas, never mind that given the speed round here over the copper wire, a lot of people are going to want Infinity...)

How to get the data by excluding the columns which are having null values

I have a table with 10 columns , 3 columns are having all null values . I want to retrive the data by excluding these 3 columns. That means output table should contain only 7 columns .
Can u give a query on this?
Thanks,
Mahesh

select <list of column with not null values>
from table_nameThat is you need to specify the column name.

Finding Columns with only NULL as values

I have a list of tables around 30 in number. Among them i need to generate a report with Table Name and Column Names, for which all values are NULL.
In other words, i need to find all the columns for which all the values are NULL.
I thought of dealing it with cursors, and comparing the total count with null count in each column. But i need to loop "Number of columns" times. Is there any better way to do this?

I think your way is a good solution. It has the advantage that you can check several columns in one select.
select count(*), count(col1), count(col2) ... from mytabAnother approach could be to check statistical information about your table data.
However this will not give you always the correct data.
It depends on the time of the last statistic collection, the sample size and on the detail level.
Here is a sample statement to play with:
select owner, table_name, column_name, num_distinct, num_nulls, num_buckets, sample_size, low_value, high_value
from all_tab_columns
where owner not in ('SYS','SYSTEM')
and nullable = 'Y' -- null values allowed
and num_distinct = 0 -- only nulls found
--and density = 0
;Note that this select would not return tables for which no statistics had been calculated. Num_distinct is NULL in such a case.
Edited by: Sven W. on Feb 11, 2010 2:50 PM

How to find aggregates which are not used.

Hi ,
There are lot of aggregates in my system , so i should deactivate the aggregates which are not used from long back. so how to find out all those , can u plz guide me ,

You can also check the usage column of the aggregates. If usage value is high then the aggregate is used very frequently.
If there is no value at the last used then the aggreagete is not used at all.
Hope it helps
Regards
Sadeesh

How to find reports which are using sales tables

Hi Guys,
we are using OBIEE 10g version.Here i need to identify the the reports which are using sales tables.
Table names are given but how to find which report is using these tables.Is there any method to find
or we have to check all reports manually?
Could any one pls suggest me here!
Regards,
sk
Edited by: 912736 on Jun 8, 2012 1:24 PM

Hi SK,
You can run a report from catalog manager that willl give you all answers requests and the subject area columns in use, you can map these back to the sales tables either manually or by linking (vlookup) to an RPD report that you can run from the Admin tool.
The Usage Tracking method is pretty good but you will have to match up the reports using the Logical SQL.
I'd do both methods and cross ref your results to ensure nothing slips the net.
Regards
Alastair

How to find documents which are not linked to a project?

Hi all,
i'm just looking for a variant to search for documents which are not linked to a project within ta SOLAR_EVAL.
I need a way to report how much documents are not linked to a project; just stored in KW.
Can anyone give a hint?
Thanks a lot!
Jan

Hi Jan,
This report SOLMAN_UNUSED_DOCUMENTS will help you identify the documents which are not linked to the project.
(OR) Use Tcode: SI80 to find any document in SAP Solution Manager KW.
Regards,
Sanjai

Find activitys which are related to an opportuntiy

Hello Experts
I can related an acctivity with a Opportunity.
I want to select this information from the sysmtem. In which tabels i find the information?
How can i finde the activitys which are related to an opportuntiy?
Regards,
Sven

Hello Vellory,
You can query the table CRMD_LINK for retrieving the follow up document. But, here you will run into a little bit confusion as to select which one.
So, a much better and cleaner approach would be to use the Function Module - CRM_ORDER_READ. Using this FM you can very easily retrieve all the following and preceding documents in the Export Tables Parameter ET_DOC_FLOW. From here you can again call the CRM_ORDER_READ to get the additional details for each one order document.
You can even execute the Report CRM_ORDER_READ in SE38 Tcode to have a feel of the actual FM.
Hope this will help.
Thanks,
Samantak.

Find System which are automatically updating by internet

Hi team,
can any one help me on this , In my environment I have around 1000 servers , we are using wuss for windows update
but some server are missing they are updating by internet means automatic update.
can I find the server name which are taking update from internet ,I don't want check manually all the servers.
Regards, Triyambak

Hi Triyambak,
I’m writing to just check in to see if the suggestions were helpful. If
you need further help, please feel free to reply this post directly so we will be notified to follow it up.
If you have any feedback on our support, please click here.
Best Regards,
Anna
TechNet Community Support

How To know Which Columns are not null and Which are null

Hi Freinds,
I want to Know ,How by Wrting a Query we can get the Names of The Columns from the Table which is set to not-null or null,
Thanks
Shoaib

SELECT * FROM user_tab_columns
WHERE table_name='Your_Table_Name'Check NULLABLE column.

How to create table with dynamic amount of columns which are nested columns

M trying to fetch data and show in a javaFX table.
My table displays the details of different items from ItemVO , where :
public class ItemVO()
String itemName;
List<ItemTypeVO> type;
and My ItemTypeVO has the following attributes :
public class ItemTypeVO()
String typeName;
Integer quantity;
Integer price;
Now, i want to display the details of an item in a table in which the itemname and quantity will be displayed, the quantity column will have nested columns showing different types(typeName) as found from List<ItemTypeVO> inside ItemVO.
This question is similar to this link but my not able to find how to link a list with itemVO for nested columns. :(
Please help !!
M still unable to find a solution..
Edited by: abhinay_a on Jan 14, 2013 10:50 AM

Hi Abhilash,
Thanks for the quick reply.
Actually the problem is with the image, as I am not able to rotate 270 degree. Crystal report cannot support the rotation of image.
i have another problem, I have to create a report in which
Lables are fixed on the left side of report and 3 columns per portrait page. Those columns are
dynamically created and shown in the report.
The format is like the above. Can you please help me in doing this report, as I tried it doing
with CrossTab. I am really stuck to this report.

Create a view, some of the columns default are NULL

Windows XP/10g 10.2.0.1.0
I have a table tab1(test1, test2, test3)
And, I need to create a views as:
create or replace VW1(test1, test2, viewtest1, viewtest2) as (select test1, test2, viewtest1, viewtest2 from tab1, dual);
And need the column value viewtest1 and viewtest2 to be default as null.
Thanks.

Are you looking for something like this ?
create or replace VIEW VW1(test1, test2, viewtest1, viewtest2) as
(select test1, test2, NULL, NULL from tab1);

Finder columns width are very wide within an app

I have searched for an answer but can't find it. Since using Mavericks when I open a finder window within an app to open a document, the finder window is way too large. The columns are very wide and fall on to my second display. If I adjust this (and I have tried with the ALT button as well) the next time the problem keeps coming back.
Any solutions?
Thanks

I have this issue too - though I've been able to work around it a little by entering the file name in the search box.

Find columns which are null

Similar Messages

Maybe you are looking for