Designing metadataLogging table for data warehouse

Hi:
I am in the process of creating metadataLogging tables for dw.
I have thought 5 tables to track errors and check etl execution.
Master table for all major jobs. Enviournment - oracle 9i. Dw size not more than 30 mb. to start with.
CREATE TABLE ADMIN_JOB_MASTER_TBL
JOBNUMBER NUMBER,
JOBNAME VARCHAR2(768 BYTE),
MINTHRESHRECORDS NUMBER,
MAXTHRESHRECORDS NUMBER,
MINTHRESHTIME VARCHAR2(64 BYTE),
MAXTHRESHTIME VARCHAR2(64 BYTE),
CREATEDATE DATE
Audit Table for auditing jobs.
CREATE TABLE ADMIN_AUDIT_JOBS_TBL
JOBNUMBER NUMBER,
JOBNAME VARCHAR2(768 BYTE),
STARTDATE DATE,
ENDDATE DATE,
NUMBERRECORDS NUMBER,
FAIL_OR_SUCCESS VARCHAR2(64 BYTE)
Step Master: for jobs involving various steps, logic such as branching, loop, flow etc. breaking the job_master to a lower level for a more insight.
audit_step
CREATE TABLE ADMIN_AUDIT_STEP_TBL
RECORDID VARCHAR2(256 BYTE),
JOBAUDITID VARCHAR2(256 BYTE),
STEPNUMBER NUMBER,
PARAMETERS VARCHAR2(512 BYTE),
NUMBERRECORDS NUMBER,
STARTDATE DATE,
ENDDATE DATE,
USERNAME VARCHAR2(256 BYTE)
Error_table: to track error during execution.
CREATE TABLE ERROR_TBL
ERROR_NO NUMBER,
TABLE_NAME VARCHAR2(124 BYTE),
ERR_NUM NUMBER,
ERR_MSG VARCHAR2(124 BYTE),
ERROR_TIME DATE
I am thinking to load the master tables manually with expected values.
And during each run, loading the auditing tables.
Error table would ofcourse be loaded during the run.
So everytime a threshold has changed, I would have to update master table manually. I dont mind doing that initially.
Would the following tables and the stated appraoch be good for etl.
Please guys, let me know your thoughts. Suggest changes or tables or approach, if you feel, that its wrong.
Thanks in advance. All inputs are highly appreciated !!!
Regards
Som

Hi,
What better than Oracle suggests...is there ????
Have you read Oracle doc titled...
"Data Warehousing Guide" and the "Handling Data Errors with an Error Logging Table " section for example...
Regards,
Simon

Similar Messages

Foreign keys in SCD2 dimensions and fact tables in data warehouse

Hello.
I have datawarehouse in snowflake schema. All dimensions are SCD2, the columns are like that:
ID (PK) SID NAME ... START_DATE END_DATE IS_ACTUAL
1 1 XXX 01.01.2000 01.01.2002 0
2 1 YYX 02.01.2002 01.01.2004 1
3 2 SYX 02.01.2002 1
4 3 AYX 02.01.2002 01.01.2004 0
5 3 YYZ 02.01.2004 1
On this table there are relations from other dimension and fact table.
Need I create foreign keys for relation?
And if I do, on what columns? SID (serial ID) is not unique. If I create on ID, I have to get SID and actual row in any query.

>
I have datawarehouse in snowflake schema. All dimensions are SCD2, the columns are like that:
ID (PK) SID NAME ... START_DATE END_DATE IS_ACTUAL
1 1 XXX 01.01.2000 01.01.2002 0
2 1 YYX 02.01.2002 01.01.2004 1
3 2 SYX 02.01.2002 1
4 3 AYX 02.01.2002 01.01.2004 0
5 3 YYZ 02.01.2004 1
On this table there are relations from other dimension and fact table.
Need I create foreign keys for relation?
>
Are you still designing your system? Why did you choose NOT to use a Star schema? Star schema's are simpler and have some performance benefits over snowflakes. Although there may be some data redundancy that is usually not an issue for data warehouse systems since any DML is usually well-managed and normalization is often sacrificed for better performance.
Only YOU can determine what foreign keys you need. Generally you will create foreign keys between any child table and its parent table and those need to be created on a primary key or unique key value.
>
And if I do, on what columns? SID (serial ID) is not unique. If I create on ID, I have to get SID and actual row in any query.
>
I have no idea what that means. There isn't any way to tell from just the DDL for one dimension table that you provided.
It is not clear if you are saying that your fact table will have a direct relationship to the star-flake dimension tables or only link to them through the top-level dimensions.
Some types of snowflakes do nothing more than normalize a dimension table to eliminate redundancy. For those types the dimension table is, in a sense, a 'mini' fact table and the other normalized tables become its children. The fact table only has a relation to the main dimension table; any data needed from the dimensions 'child' tables is obtained by joining them to their 'parent'.
Other snowflake types have the main fact table having relations to one or more of the dimensions 'child' tables. That complicates the maintenance of the fact table since any change to the dimension 'child' table impacts the fact table also. It is not recommended to use that type of snowflake.
See the 'Snowflake Schemas' section of the Data Warehousing Guide
http://docs.oracle.com/cd/B28359_01/server.111/b28313/schemas.htm
>
Snowflake Schemas
The snowflake schema is a more complex data warehouse model than a star schema, and is a type of star schema. It is called a snowflake schema because the diagram of the schema resembles a snowflake.
Snowflake schemas normalize dimensions to eliminate redundancy. That is, the dimension data has been grouped into multiple tables instead of one large table. For example, a product dimension table in a star schema might be normalized into a products table, a product_category table, and a product_manufacturer table in a snowflake schema. While this saves space, it increases the number of dimension tables and requires more foreign key joins. The result is more complex queries and reduced query performance. Figure 19-3 presents a graphical representation of a snowflake schema.

Data Access Object for Data Warehouse?

Hi,
Does anyone know how the DAO pattern looks like when it is used for a data warehouse rather than a normal transactional database?
Normally we have something like CustomerDAO or ProductDAO in the DAO pattern, but for data warehouse applications, JOINs are used and multiple tables are queried, for example, a query may contains data from the Customer, Product and Time table, what should the DAO class be named? CustomerProductTimeDAO?? Any difference in other parts of the pattern?
Thanks in advance.
SK

In my opinion, there are no differences in the Data Access Object design pattern which have any thing to do with any characteristic of its implementation or the storage format of the data the pattern is designed to function with.
The core pupose of the DAO design pattern is to encapsulate data access code and separate it from the business logic code of the application. A DAO implementation might vary from application to application. The design pattern does not specify any implementation details. A DAO implementation can be applied to group of XML data files, an Excel-based CSV file, a relational database, or an OS file system. The design is the same for all these, it is the implementation that varies.
The core difference between an operational database and a strategic data warehouse is the purpose of why and how the data is used. It is not so much a technical difference. The relational design may vary however, there may be more tables amd ternary relationships in a data warehouse to support more fine-tuned queries; there may be less tables in a operational database to support insert/add efficiencies.
The DAO implementation for a data warehouse would be based on the model of the databases. However the tables are set up, that is how the DAO is coded.

Are there any timesten installation for data warehouse environment?

Hi,
I wonder if there is a way to install timesten as an in memory database for data warehouse environment?
The DW today consist of a large Oralcle database and I wonder if and how a timesten implementation can be done.
what kind of application changes involve with such an implementation and so on?
I know that the answer is probably complex but if anyone knows about such an implementation and some information about it , it would be great to learn from that experience.
Thanks,
Adi

Adi,
It depends on what you want to do with the data in the TimesTen database. If you know the "hot" dataset that you want to cache in TimesTen, you can use Cache Connect to Oracle to cache a subset of your Oracle tables into TimesTen. The key is to figure out what queries you want to run and see if the queries are supported in TimesTen.
Assuming you know the dataset you need to cache and you have control of your application code to change the connection to TimesTen (using ODBC or JDBC), you can give it a try. If you are using a third party tool, you need to see if the tool supports JDBC or ODBC access to the database and change the tool to point to your TimesTen database instead of the Oracle database.
If you are using the TimesTen Cache Connect to Oracle product option, data synchronization between Oracle and TimesTen is handled automatically by the product.
Without further details of what you'd like to do, it's difficult to provide more detailed recommendation.
-scheung

Example database for data warehouse

Hi!
Does anybody know where can be found exaple database for data warehouse?
- schema is good
- schema with data is better
Best regards

slkLinuxUser wrote:
Hi!
Does anybody know where can be found exaple database for data warehouse?
- schema is good
- schema with data is better
Best regardsJust like an OLTP database, the schema design and its data is 100% dependant on the business needs and (if done properly) the result of a thorough data analysis. Any kind of a pre-designed sample would be near worthless to any actual application.

Is RAC suited for Data Warehouse

Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?

VahidS wrote:
Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?
I am not sure that which documents you have read based on which you have got the impression that RAC is for DWH. RAC is a part of the Oracle's High Availability solution architecture. It's not designed to cater any specific kind of workload but to provide HA in the case of a node or instance crash. And also, RAC itself doesn't give any performance benefit. If you have a database which is poorly performing, it would remain the same or may be , will become even worse after implementing RAC due to too many things being involved in the architecture now e.g storage, network etc. So if you do want to implement RAC, give it a careful thought, do some testing and then only project it. If you are determined to go for RAC and you are a pure DWH shop, I would suggest to go fro Exadata running RAC . Smart Scan, Compression are just few of those benefits which would come with Exadata along with the RAC's HA solution that would be serving better for your requirements.
HTH
Aman....

RMAN crosscheck and expire guidelines for data warehouse environment

O/S: Windows Server 2008
DB: Oracle 11gR2
Are there any guidelines to how often one should do a RMAN crosscheck and set an expiration on archivelogs for data warehouse environments?
It would seem once a day would be enough for the crosscheck in a data warehouse environment that gets refreshed nightly. Expiration I would expect no less than 1 week.
Cheers!

I agree with damorgan
refer the below links for best practices.
http://www.oracle.com/technetwork/database/features/availability/311394-132335.pdf
https://blogs.oracle.com/datawarehousing/entry/data_warehouse_in_archivelog_m
Hope this helps,
Regards
http://www.oracleracepxert.com
Understand the Power of Oracle RMAN
http://www.oracleracexpert.com/2011/10/understand-power-of-oracle-rman.html
Duplicating RAC database using RMAN
http://www.oracleracexpert.com/2009/12/duplicate-rac-database-using-rman.html

RAC for Data Warehouse

Hello,
We have a research project for restructructuring our data warehouse system.
I would like to get some opinions about whether RAC architecture can be
a good solution for Data Warehouse application.
We have using parallel queries massively. Does running these kind of queries
on different servers on RAC with multiple server result in performance degradation rather than
running on single monolithic server with multiple CPUs
I will appreciate any comments using RAC architecture for Data Warehouse
systems?
Regards,

Maurice Muller wrote:
Just keep in mind that during the last 4 years (I guess your current system is about 4 years old) the CPUs became much faster.
A cpu can't work without data which means that the I throughput has to be fast enough to feed all your cores with data.
The main bottleneck of all DWHs I have seen during the last 8 years was allways the IO never the CPUs. And not just data warehousing Maurice, but a basic principle for any data processing platform - the slowest layer is always the I/O layer.. and can be the most expensive one to solve too.
Which is why newer technology like Infiniband is exciting as it can also serve as the I/O layer. Instead of using the traditional HBA which is typically configured with 2Gb fibre channels to the storage layer, using HCA cards you can wire this directly into an Infiniband storage array... and this can run at up to speeds of 40Gb. Dual connections means a total theoretical pipe size of 80Gb. I do not know of any other standard technology (like GigE) that can provide any similar bandwidth speed.
Back to RAC though - with RAC, when you add a new server that comes with a new set of I/O pipes.. plus of course more RAM and more CPU cores. SMP server architecture does not scale like this at all. You only have x number of slots for PCI cards, CPUs and RAM. A very specific ceiling that cannot be moved. With MPP this ceiling is a a lot higher and more flexible.
You can also replace a dual core dual CPU nodes with a 6 core AMD Istanbul CPUs next year.. and possibly 12 core CPUs year after that. So even a smallish 4 node cluster with 16 cores in total can be grown significantly and remain a 4 node cluster. Together with advances in HPC (High Performance Computing) like Infiniband.
I'm not seeing much use of non-RAC RDBMS architecture in the future. Databases are getting ever bigger because we have the technology to crunch more data, and crunch it a lot more intelligently than ever before. My first production database was 4MB in size, and ran on a Novell File Server with two 20MB disks. I'm currently testing a 24TB array for use for a single database.
Technology is inevitable, as is the growth in data volumes. And I cannot see a non-RAC architecture rising to that challenge. Especially not in something like data warehousing.

Are analytic functions usefull only for data warehouses?

Hi,
I deal with reporting queries on Oracle databases but I don't work on Data Warehouses, thus I'd like to know if learning to use analytic functions (sql for anaylis such as rollup, cube, grouping, ...) might be usefull in helping me to develop better reports or if analytic functions are usually usefull only for data warehouses queries. I mean are rollup, cube, grouping, ... usefull also on operational database or do they make sense only on DWH?
Thanks!

Mark1970 wrote:
thus does it worth learning them for improving report queries also not on DHW but on common operational databases?Why pigeonhole report queries as "+operational+" or "+data warehouse+"?
Do you tell a user/manager that "<i>No, this report cannot be done as it looks like a data warehouse report and we have an operational database!</i>"?
Data processing and data reporting requirements not not care what label you assign to your database.
Simple real world example of using analytical queries on a non warehouse. We supply data to an external system via XML. They require that we limit the number of parent entities per XML file we supply. E.g. 100 customer elements (together with all their child elements) per file. Analytical SQL enables this to be done by creating "buckets" that can only contain 100 parent elements at a time. Complete process is SQL driven - no slow-by-slow row by row processing in PL/SQL using nested cursor loops and silly approaches like that.
Analytical SQL is a tool in the developer toolbox. It would be unwise to remove it from the toolbox, thinking that it is not applicable and won't be needed for the work that's to be done.

OBE for Data Warehouse 9.2

Dear All
Can anyone tell me the link of Oracle By Example: Hands on Tutorials for Data Warehouse 9.2.
I have found Oracle By Example: Hands On Tutorials Data Warehouse OWB 10g release 1, 10g release 2 and 11g at the following link
http://www.oracle.com/technology/obe/admin/owb_main.html
Thanks in Advance.
Best Regards
Thunder2777

Dear Dallan
I was looking for the Oracle By Example sort of tutorials for OWB 9.2.
Actually I want to learn OWB form 9.2 to 10.2. I have OWB 10.1 and OWB 10.2,
what missing was OWB 9.2.
That's why I have put it on forum so that anyone who knows where they can guide me.
But your comment that there is a little difference in OWB 9.2 and OWB 10.1, if I won't get
OWB 9.2 then it's better to start with OWB 10.1.
Ayway Thanks.
Thunder2777

Best practice of metadata table in data warehouse environment ?

Hi guru's,
In datawarehouse, we have 1. Stage schema 2. DWH(Data warehouse reporting schema). In stageing we have about 300 source tables. In DWH schema, we are creating the tables which are only required from reporting prespective . some of the tables in stageing schema, have been created in DWH schema as well with different table name and column names. The naming convention for these same tables and columns in DWH schema is more based on business names.
In order to keep track of these tables we are creating metadata table in DWH schema say for example
Stage                DWH_schema
Table_1             Table_A
Table_2             Table_b
Table_3             Table_c
Table_4              Table_DMy question is how do we handle the column names in each of these tables. The stage_1, stage_2 and stage_3 column names have been renamed in DWH_schema which are part of Table_A, Table_B, Table_c.
As said earlier, we have about 300 tables in stage and may be around 200 tables in DWH schema. Lot of the column names have been renamed in DWH schema from stage tables. In some of the tables we have 200 column's
so my concern is how do we handle the column names in metadata table ? Do we need to keep only table names in metadata table not column names ?
Any idea will be greatly appriciated.
Thanks!

hi
seems quite a buzzing question.
In our project we designed a hub and spoke like architecture.
Thus we have 3 layer, L0 is the one closest to the source and L0 table's name are linked to the corresponding sources names by mean of naming standard (like tabA EXT_tabA tabA_OK1 so on based on implementation of load procedures).
At L1 we have the ODS , normalized model , we use business names for table there and standard names for temporary structures and artifacts
Both L0 an L1 keep source's column names as general rule, new columns like calculated one are business driven and metadata are standard driven.
Datamodeler fits perfect for modelling L1 purpose.
L2 is the dimensional schema business names take place for tables and columns eventually rewritten at presentation layer ( front end tool )
hope this helps D.

Good pratice for data warehouse

Hi all,
I am looking for good pratices on designing and maintaining a healthy data warehouse. Any tips, appreicated it if you can give me some ideas as well as url with related topics. Besides, someone told me that enabling the Oracle parameter "star_transformation_enabled" can help gaining performance. Is that ture? How??
Many Thanks,
Chris,

I have the similar question. I have Oracle 8i batabase in which a table whose records which is increasing at a rate of 4000 every day, and the very recent records are the only one which is required to accesss ery often. Is there any method available, so that I can use a materialised view for the latest records, say for recent 7 days old records would be accessed only in a query which is having several joins, unions and sums.
The record is having structure like
Date
Time
A Counter to join other table
time_Division
Equimpent Code
Equipment Status
Duration
etc....
Any suggestion would be welcome...

What are the best solutions for data warehouse configuration in 10gR2

I need help on solutions to be provided to my Client for upgrading the data warehouse.
Current Configuration: Oracle database 9.2.0.8. This database contains the data warehouse and one more data mart on the same host.Sizes are respectively 6 Terabyte(retention policy of 3 years+current year) and 1 Terabyte. The ETL tool and BO Reporting tools are also hosted on the same host. This current configuration is really performing poor.
Client cannot go for a major architectural or configuration changes to its existing environment now due to some constraints.
However, they have agreed to separate out the databases on separate hosts from the ETL tools and BO objects. Also we are planning to upgrade the database to 10gR2 to attain stability, better performance and overcome current headaches.
We cannot upgrade the database to 11g as the BO is at a version 6.5 which isn't compatible with Oracle 11g. And Client cannot afford to upgrade anything else other than the database.
So, my role is very vital in providing a perfect solution towards better performance and take a successful migration of Oracle Database from one host to another (similar platform and OS) in addition to upgrade.
I have till now thought of the following:
Move the Oracle database and data mart to separate host.
The host will be the same platform, that is, HP Superdome with HP-UX 32-bit OS (we cannot change to 64-bit as ETL tool doesn't support)
Install new Oracle database 10g on the new host and move the data to it.
Exploring all new features of 10gR2 to help data warehouse, that is, SQL MODEL Clause introduction, Parallel processing, Partitioning, Data Pump, SPA to study pre and post migrations.
Also thinking of RAC to provide more better solution as our main motive is to show a tremendous performance enhancement.
I need all your help to prepare a good road map for my assignment. Please suggest.
Thanks,
Tapan

SGA=27.5 GB and PGA=50 MB
Also I am pasting part of STATSPACK Report, eliminating the snaps of DB bounce. Please suggest the scope of improvement in this case.
STATSPACK report for
Snap Id Snap Time Sessions Curs/Sess Comment
Begin Snap: 582946 11-Mar-13 20:02:16 46 12.8
End Snap: 583036 12-Mar-13 18:24:24 60 118.9
Elapsed: 1,342.13 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
Buffer Cache: 21,296M Std Block Size: 16K
Shared Pool Size: 6,144M Log Buffer: 16,384K
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
Redo size: 1,343,739.01 139,883.39
Logical reads: 100,102.54 10,420.69
Block changes: 3,757.42 391.15
Physical reads: 6,670.84 694.44
Physical writes: 874.34 91.02
User calls: 1,986.04 206.75
Parses: 247.87 25.80
Hard parses: 5.82 0.61
Sorts: 1,566.76 163.10
Logons: 10.99 1.14
Executes: 1,309.79 136.35
Transactions: 9.61
% Blocks changed per Read: 3.75 Recursive Call %: 43.34
Rollback per transaction %: 3.49 Rows per Sort: 190.61
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.90 Redo NoWait %: 100.00
Buffer Hit %: 96.97 In-memory Sort %: 100.00
Library Hit %: 99.27 Soft Parse %: 97.65
Execute to Parse %: 81.08 Latch Hit %: 99.58
Parse CPU to Parse Elapsd %: 3.85 % Non-Parse CPU: 99.34
Shared Pool Statistics Begin End
Memory Usage %: 7.11 50.37
% SQL with executions>1: 62.31 46.46
% Memory for SQL w/exec>1: 26.75 13.47
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
CPU time 492,062 43.66
db file sequential read 157,418,414 343,549 30.49
library cache pin 92,339 66,759 5.92
PX qref latch 63,635 43,845 3.89
db file scattered read 2,506,806 41,677 3.70
Background Wait Events for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
-> ordered by wait time desc, waits desc (idle events last)
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
log file sequential read 176,386 0 3,793 22 0.2
log file parallel write 2,685,833 0 1,813 1 3.5
db file parallel write 239,166 0 1,350 6 0.3
control file parallel write 33,432 0 79 2 0.0
LGWR wait for redo copy 478,120 536 75 0 0.6
rdbms ipc reply 10,027 0 47 5 0.0
control file sequential read 32,414 0 40 1 0.0
db file scattered read 4,101 0 30 7 0.0
db file sequential read 13,946 0 29 2 0.0
direct path read 203,694 0 14 0 0.3
log buffer space 363 0 13 37 0.0
latch free 3,766 0 9 2 0.0
direct path write 80,491 0 6 0 0.1
async disk IO 351,955 0 4 0 0.5
enqueue 28 0 1 21 0.0
buffer busy waits 1,281 0 1 0 0.0
log file single write 172 0 0 1 0.0
rdbms ipc message 10,563,204 251,286 992,837 94 13.7
pmon timer 34,751 34,736 78,600 2262 0.0
smon timer 7,462 113 76,463 10247 0.0
Instance Activity Stats for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
Statistic Total per Second per Trans
CPU used by this session 49,206,154 611.0 63.6
CPU used when call started 49,435,735 613.9 63.9
CR blocks created 6,740,777 83.7 8.7
Cached Commit SCN referenced 423,253,503 5,256.0 547.2
Commit SCN cached 19,165 0.2 0.0
DBWR buffers scanned 48,276,489 599.5 62.4
DBWR checkpoint buffers written 6,959,752 86.4 9.0
DBWR checkpoints 454 0.0 0.0
DBWR free buffers found 44,817,183 556.5 57.9
DBWR lru scans 137,149 1.7 0.2
DBWR make free requests 162,528 2.0 0.2
DBWR revisited being-written buff 4,220 0.1 0.0
DBWR summed scan depth 48,276,489 599.5 62.4
DBWR transaction table writes 5,036 0.1 0.0
DBWR undo block writes 2,989,436 37.1 3.9
DDL statements parallelized 3,723 0.1 0.0
DFO trees parallelized 4,157 0.1 0.0
DML statements parallelized 3 0.0 0.0
OS Block input operations 29,850 0.4 0.0
OS Block output operations 1,591 0.0 0.0
OS Characters read/written 182,109,814,791 2,261,447.1 235,416.9
OS Integral unshared data size ################## 242,463,432.4 ############
OS Involuntary context switches 188,257,786 2,337.8 243.4
OS Maximum resident set size 43,518,730,619 540,417.4 56,257.5
OS Page reclaims 159,430,953 1,979.8 206.1
OS Signals received 5,260,938 65.3 6.8
OS Socket messages received 79,438,383 986.5 102.7
OS Socket messages sent 93,064,176 1,155.7 120.3
OS System time used 10,936,430 135.8 14.1
OS User time used 132,043,884 1,639.7 170.7
OS Voluntary context switches 746,207,739 9,266.4 964.6
PX local messages recv'd 55,120,663 684.5 71.3
PX local messages sent 55,120,817 684.5 71.3
Parallel operations downgraded 1 3 0.0 0.0
Parallel operations not downgrade 4,154 0.1 0.0
SQL*Net roundtrips to/from client 155,422,335 1,930.0 200.9
SQL*Net roundtrips to/from dblink 18 0.0 0.0
active txn count during cleanout 16,529,551 205.3 21.4
background checkpoints completed 43 0.0 0.0
background checkpoints started 43 0.0 0.0
background timeouts 280,202 3.5 0.4
branch node splits 4,428 0.1 0.0
buffer is not pinned count 6,382,440,322 79,257.4 8,250.7
buffer is pinned count 9,675,661,370 120,152.8 12,507.9
bytes received via SQL*Net from c 67,384,496,376 836,783.4 87,109.3
bytes received via SQL*Net from d 6,142 0.1 0.0
bytes sent via SQL*Net to client 50,240,643,657 623,890.4 64,947.1
bytes sent via SQL*Net to dblink 3,701 0.1 0.0
calls to get snapshot scn: kcmgss 145,385,064 1,805.4 187.9
calls to kcmgas 36,816,132 457.2 47.6
calls to kcmgcs 3,514,770 43.7 4.5
change write time 369,373 4.6 0.5
cleanout - number of ktugct calls 20,954,488 260.2 27.1
cleanouts and rollbacks - consist 6,357,174 78.9 8.2
cleanouts only - consistent read 10,078,802 125.2 13.0
cluster key scan block gets 69,403,565 861.9 89.7
Instance Activity Stats for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
Statistic Total per Second per Trans
cluster key scans 41,311,211 513.0 53.4
commit cleanout failures: block l 413,776 5.1 0.5
commit cleanout failures: buffer 414 0.0 0.0
commit cleanout failures: callbac 41,194 0.5 0.1
commit cleanout failures: cannot 174,382 2.2 0.2
commit cleanouts 11,469,056 142.4 14.8
commit cleanouts successfully com 10,839,290 134.6 14.0
commit txn count during cleanout 17,155,424 213.0 22.2
consistent changes 145,418,277 1,805.8 188.0
consistent gets 8,043,252,188 99,881.4 10,397.7
consistent gets - examination 3,180,028,047 39,489.7 4,110.9
current blocks converted for CR 9 0.0 0.0
cursor authentications 14,926 0.2 0.0
data blocks consistent reads - un 143,706,500 1,784.6 185.8
db block changes 302,577,666 3,757.4 391.2
db block gets 336,562,217 4,179.4 435.1
deferred (CURRENT) block cleanout 2,912,793 36.2 3.8
dirty buffers inspected 627,174 7.8 0.8
enqueue conversions 1,296,337 16.1 1.7
enqueue releases 13,053,200 162.1 16.9
enqueue requests 13,239,092 164.4 17.1
enqueue timeouts 185,878 2.3 0.2
enqueue waits 114,120 1.4 0.2
exchange deadlocks 7,390 0.1 0.0
execute count 105,475,101 1,309.8 136.4
free buffer inspected 1,604,407 19.9 2.1
free buffer requested 258,126,047 3,205.4 333.7
hot buffers moved to head of LRU 22,793,576 283.1 29.5
immediate (CR) block cleanout app 16,436,010 204.1 21.3
immediate (CURRENT) block cleanou 2,860,013 35.5 3.7
index fast full scans (direct rea 12,375 0.2 0.0
index fast full scans (full) 3,733 0.1 0.0
index fast full scans (rowid rang 192,148 2.4 0.3
index fetch by key 1,321,024,486 16,404.5 1,707.7
index scans kdiixs1 406,165,684 5,043.8 525.1
leaf node 90-10 splits 50,373 0.6 0.1
leaf node splits 697,235 8.7 0.9
logons cumulative 884,756 11.0 1.1
messages received 3,276,719 40.7 4.2
messages sent 3,257,171 40.5 4.2
no buffer to keep pinned count 569 0.0 0.0
no work - consistent read gets 4,406,092,172 54,715.0 5,695.8
opened cursors cumulative 20,527,704 254.9 26.5
parse count (failures) 267,088 3.3 0.4
parse count (hard) 468,996 5.8 0.6
parse count (total) 19,960,548 247.9 25.8
parse time cpu 323,024 4.0 0.4
parse time elapsed 8,393,422 104.2 10.9
physical reads 537,189,332 6,670.8 694.4
physical reads direct 292,545,140 3,632.8 378.2
physical writes 70,409,002 874.3 91.0
physical writes direct 59,248,394 735.8 76.6
physical writes non checkpoint 69,103,391 858.1 89.3
pinned buffers inspected 11,893 0.2 0.0
prefetched blocks 95,892,161 1,190.8 124.0
prefetched blocks aged out before 1,495,883 18.6 1.9
Instance Activity Stats for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
Statistic Total per Second per Trans
process last non-idle time ################## ############## ############
queries parallelized 417 0.0 0.0
recursive calls 122,323,299 1,519.0 158.1
recursive cpu usage 3,144,533 39.1 4.1
redo blocks written 180,881,558 2,246.2 233.8
redo buffer allocation retries 5,400 0.1 0.0
redo entries 164,728,513 2,045.6 213.0
redo log space requests 1,006 0.0 0.0
redo log space wait time 2,230 0.0 0.0
redo ordering marks 2,563 0.0 0.0
redo size 108,208,614,904 1,343,739.0 139,883.4
redo synch time 558,520 6.9 0.7
redo synch writes 2,343,824 29.1 3.0
redo wastage 1,126,585,600 13,990.0 1,456.4
redo write time 718,655 8.9 0.9
redo writer latching time 7,763 0.1 0.0
redo writes 2,685,833 33.4 3.5
rollback changes - undo records a 522,742 6.5 0.7
rollbacks only - consistent read 335,177 4.2 0.4
rows fetched via callback 1,100,990,382 13,672.1 1,423.3
session connect time ################## ############## ############
session cursor cache count 1,061 0.0 0.0
session cursor cache hits 1,687,796 21.0 2.2
session logical reads 8,061,057,193 100,102.5 10,420.7
session pga memory 1,573,228,913,832 19,536,421.0 2,033,743.8
session pga memory max 1,841,357,626,496 22,866,054.4 2,380,359.0
session uga memory 1,074,114,630,336 13,338,399.4 1,388,529.0
session uga memory max 386,645,043,296 4,801,374.0 499,823.6
shared hash latch upgrades - no w 410,360,146 5,095.9 530.5
sorts (disk) 2,657 0.0 0.0
sorts (memory) 126,165,625 1,566.7 163.1
sorts (rows) 24,048,783,304 298,638.8 31,088.3
summed dirty queue length 5,438,201 67.5 7.0
switch current to new buffer 1,302,798 16.2 1.7
table fetch by rowid 6,201,503,534 77,010.5 8,016.8
table fetch continued row 26,649,697 330.9 34.5
table scan blocks gotten 1,864,435,032 23,152.6 2,410.2
table scan rows gotten 43,639,997,280 541,923.3 56,414.3
table scans (cache partitions) 26,112 0.3 0.0
table scans (direct read) 246,243 3.1 0.3
table scans (long tables) 340,200 4.2 0.4
table scans (rowid ranges) 359,617 4.5 0.5
table scans (short tables) 9,111,559 113.2 11.8
transaction rollbacks 4,819 0.1 0.0
transaction tables consistent rea 824 0.0 0.0
transaction tables consistent rea 1,386,848 17.2 1.8
user calls 159,931,913 1,986.0 206.8
user commits 746,543 9.3 1.0
user rollbacks 27,020 0.3 0.0
write clones created in backgroun 7 0.0 0.0
write clones created in foregroun 4,350 0.1 0.0
Buffer Pool Statistics for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
-> Standard block size Pools D: default, K: keep, R: recycle
-> Default Pools for other block sizes: 2k, 4k, 8k, 16k, 32k
Free Write Buffer
Number of Cache Buffer Physical Physical Buffer Complete Busy
P Buffers Hit % Gets Reads Writes Waits Waits Waits
D 774,144 95.6############ 233,869,082 10,089,734 0 0########
K 504,000 99.9############ 3,260,227 1,070,338 0 0 65,898
R 63,504 96.2 196,079,539 7,511,863 535 0 0 0
Buffer wait Statistics for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
-> ordered by wait time desc, waits desc
Tot Wait Avg
Class Waits Time (s) Time (ms)
data block 7,791,121 14,676 2
file header block 587 101 172
undo header 151,617 71 0
segment header 299,312 58 0
1st level bmb 45,235 7 0
bitmap index block 392 1 3
undo block 4,250 1 0
2nd level bmb 14 0 0
system undo header 2 0 0
3rd level bmb 1 0 0
Latch Activity for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
->"Get Requests", "Pct Get Miss" and "Avg Slps/Miss" are statistics for
willing-to-wait latch get requests
->"NoWait Requests", "Pct NoWait Miss" are for no-wait latch get requests
->"Pct Misses" for both should be very close to 0.0
Pct Avg Wait Pct
Get Get Slps Time NoWait NoWait
Latch Requests Miss /Miss (s) Requests Miss
Consistent RBA 2,686,230 0.0 0.2 0 0
FAL request queue 86 0.0 0 0
FAL subheap alocation 0 0 2 0.0
FIB s.o chain latch 1,089 0.0 0 0
FOB s.o list latch 4,589,986 0.5 0.0 2 0
NLS data objects 1 0.0 0 0
SQL memory manager worka 5,963 0.0 0 0
Token Manager 0 0 2 0.0
active checkpoint queue 719,439 0.3 0.1 0 1 0.0
alert log latch 184 0.0 0 2 0.0
archive control 4,365 0.0 0 0
archive process latch 1,808 0.6 0.6 0 0
begin backup scn array 3,387,572 0.0 0.0 0 0
cache buffer handles 1,577,222 0.2 0.0 0 0
cache buffers chains ############## 0.5 0.0 430 354,357,972 0.3
cache buffers lru chain 17,153,023 0.1 0.0 1 385,505,654 0.5
cas latch 538,804,153 0.3 0.0 7 0
channel handle pool latc 1,776,950 0.5 0.0 0 0
channel operations paren 2,901,371 0.3 0.0 0 0
checkpoint queue latch 99,329,722 0.0 0.0 0 11,153,369 0.1
child cursor hash table 3,927,427 0.0 0.0 0 0
commit callback allocati 8,739 0.0 0 0
dictionary lookup 7,980 0.0 0 0
dml lock allocation 6,767,990 0.1 0.0 0 0
dummy allocation 1,898,183 0.2 0.1 0 0
enqueue hash chains 27,741,348 0.1 0.1 4 0
enqueues 17,450,161 0.3 0.1 6 0
error message lists 132,828 2.6 0.2 1 0
event group latch 884,066 0.0 0.7 0 0
event range base latch 1 0.0 0 0
file number translation 34 38.2 0.9 0 0
global tx hash mapping 577,859 0.0 0 0
hash table column usage 4,062 0.0 0 8,757,234 0.0
hash table modification 16 0.0 0 2 0.0
i/o slave adaptor 0 0 2 0.0
job workq parent latch 4 100.0 0.3 0 494 8.7
job_queue_processes para 1,950 0.0 0 2 0.0
ksfv messages 0 0 4 0.0
ktm global data 8,219 0.0 0 0
lgwr LWN SCN 2,687,862 0.0 0.0 0 0
library cache 310,882,781 0.9 0.0 34 104,759 4.0
library cache load lock 30,369 0.0 0.3 0 0
library cache pin 153,821,358 0.1 0.0 2 0
library cache pin alloca 126,316,296 0.1 0.0 4 0
list of block allocation 2,730,808 0.3 0.0 0 0
loader state object free 566,036 0.1 0.0 0 0
longop free list parent 197,368 0.0 0 8,390 0.0
message pool operations 14,424 0.0 0.0 0 0
messages 25,931,764 0.1 0.0 1 0
mostly latch-free SCN 40,124,948 0.3 0.0 5 0
Latch Sleep breakdown for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
-> ordered by misses desc
Get Spin &
Latch Name Requests Misses Sleeps Sleeps 1->4
cache buffers chains ############## 74,770,083 1,062,119 73803903/884
159/71439/10
582/0
redo allocation 170,107,983 3,441,055 149,631 3292872/1467
48/1426/9/0
library cache 310,882,781 2,831,747 89,240 2754499/6780
6/7405/2037/
0
shared pool 158,471,190 1,755,922 55,268 1704342/4836
9/2826/385/0
cas latch 538,804,153 1,553,992 6,927 1547125/6808
/58/1/0
row cache objects 161,142,207 1,176,998 27,658 1154070/1952
0/2560/848/0
process queue reference 1,893,917,184 1,119,215 106,454 78758/4351/1
36/0/0
Library Cache Activity for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
->"Pct Misses" should be very low
Get Pct Pin Pct Invali-
Namespace Requests Miss Requests Miss Reloads dations
BODY 3,137,721 0.0 3,137,722 0.0 0 0
CLUSTER 6,741 0.1 4,420 0.2 0 0
INDEX 353,708 0.8 361,065 1.2 0 0
SQL AREA 17,052,073 0.3 54,615,678 0.9 410,682 19,628
TABLE/PROCEDURE 3,521,884 0.2 12,922,737 0.1 619 0
TRIGGER 1,975,977 0.0 1,975,977 0.0 1 0
SGA Memory Summary for DB: P7IN1 Instance: P7IN1 Snaps: 582946 -583036
SGA regions Size in Bytes
Database Buffers 22,330,474,496
Fixed Size 779,288
Redo Buffers 17,051,648
Variable Size 7,180,648,448
sum 29,528,953,880

Spatial tables in Data warehouse database or separate?

Hi all,
I have a question from a customer who's asking what the recommendation is of storage of the spatial tables and the warehouse tables.
Should they reside on the same database for best performance or doesn't it matter where the spatial tables are located.
Is there any official recommednation from Oracle on this?
Best regards,
Hakan

Hi Hakan,
it's not easy to give a quick answer here.
a) you want to compare warehouse tables with spatial tables
To answer this very well you should define what is a warehouse table and what is a spatial table. In this forum you will get for spatial tables the definition "A spatial table is a table with one or more columns of type SDO_GEOMETRY". How do you define a spatial table and how do you define a warehouse table?
b) you ask for performance
Performance is not depending only on table type. Performance is depending on a lot of steps. For example on the work which has to be done to answer the question, can data read from buffer cache or must they be read from physical devices, execution plan .....
I don't know if there is a official statement from Oracle which is usefull to answer your query.
Can you ask a little bit particular please?
In my experience there it is not adviseable to store tables in different databases (instances) when I have to query it together. If you split the tables in different instances, the execution has much more overhead because you have to talk to 2 instances and it will very hard for the optimizer to find the best execution plan.
Regards
U. Martin

Update data automatically in fact table in Data Warehouse

Hi,
I'm working on the creation of a data warehouse that include different data source like SQL Server performance (more than one), Active Directory users, Server performance (more than one), Exchange server mailboxes. The problem is that performance data change
frequently (like CPU and Memory), so my question is how to update data in fact table every 5 seconds automatically with SSIS.
Thank you for any advice

I'm assuming you have already figured out how to capture the data e.g. Powershell, extended events, MDW etc. and just need to know what dimensions or fact tables do you need.
You need to decide how often you are going to capture this data and based on that you will have dimensions with appropriate grain. Don't try to cram everything in the same fact table if it not of the same granularity. Also, separate process usually
have separate fact tables.
In addition to the Date dimension, you will need a Time dimension with a grain of 1 second (or maybe 5 seconds if that is when you get your data) then run the SSIS every 5 seconds to capture and append that data in the fact table.
- Aalamjeet Rangi | (Blog)

Designing metadataLogging table for data warehouse

Similar Messages

Maybe you are looking for