Recommended throughput for Oracle data warehouse

Hi, I know up front this is going to be a vague question...but I'm trying to determine approximate I/O bandwidth for a data mart server. Right now we're hosting 3 or 4 different marts on it, but that number is going to increase.
Oracle's DW "2 day" class recommends starting with either maximum throughput from user queries, or basing it off of batch windows. Right now the server is barely used for end user queries, as we haven't yet implemented a BI tool to allow users easy access (that's underway right now). So I find it hard to base any info on that. However, it's on the way, and I'm in charge of the BI took (OBIEE). I'm having nightmares that we get OBIEE deployed, and our queries end up taking 5 minutes each to get answers... Right now, on the system basically by myself, if I do a simple "select sum(amount) from fact_ledger", where fact_ledger is a 1 Gb table (with 40 million rows), it takes almost a full minute to run. It feels like I could add this up by hand and get an answer faster...and this certainly doesn't compare with other Oracle marts / DWs I've worked on in the past.
From a batch window standpoint, all I can say is that it feels really, REALLY too slow to me. Right now, some jobs that start with a 40 million row table and join it to 6 or 7 other small tables (looking up surrogate keys) and writing to a non-logged, non-indexed output table takes over 2 1/2 hours to complete. To me this should be a 15 minute job.
We've asked IT to do a "root cause analysis" of why performance is so bad - but as part of that, the architecture group wants something more concrete than "it just feels way too slow". So does anyone have some general guidelines they can provide? I guess our detailed info would be:
- three marts, each of which has a fact table around the 30 - 60 million row level
- simple "join 30 million row staging table to look up surrogate keys" and writing results is taking 2.5+ hours
- we expect at some point to have mabe 50 - 100 users running data concurrently (spread across the marts)
- users will be performance both canned and ad-hoc analysis against it...and they are high level business users, aren't going to be happy with waiting 2 minutes for a simple answer
My start was to swag this as requiring 6 CPUs or so, which would indicate (according to Oracle's best practice docs) of needing somewhere betweeen 1.2 GB/s to 2.4 GB/s throughput. I'm assuming if it takes almost a full minute to read a 1 GB table, that our IO is currently 60 to 120 times too slow. Does that make sense?
Thanks and sorry for the lack of details...we just don't know yet.
Thx,
Scott

Why don't you start by taking an AWR report from those two hours so you can see what is the bottleneck for your system ?

Similar Messages

Tables between OBAW and Oracle Data Warehouse in OBIA rpd are different

Hi,
The tables in the Data Warehouse DB is different as compared to the tables in the physical layer of the Oracle data Warehouse in OracleBIAnalytiscApps.rpd.
When I click update Row Count, it gives me below error message:
There was an error while updating row count for "Oracle Data Warehouse"."Catalog"."DBO"."W_CTRY_REGN_D":
[nQSError :17001] Oracle Error Code: 942, message: ORA-00942:Table or view does not exist at OCICallSTmTEXECUTE
Whereas when I click on some other tables for Row count there is not error.
Also, When I am trying to look for some specific table which exist in Data Warehouse (that has the data), it does not exist in the rpd.
Kindly assit me.

hi,
Have a look
Not able toView Data in Answers
Let me know
thanks,
saichand.v

Designer Vs. Oracle Data warehouse builder

Dear all,
Currently I'm responsible of building a Data warehousing project using Oracle database. I'm trying to decide on a tool for modelling my datawarehouse. I have two options:
1) Designer: we have some experience with this tool and we are using it for our main OLTP application.
2) Oracle Data Warehouse builder: we are using this to design our ETL processes.
I want to get some advice on whether the OWB is capable of modelling my datawarehouse and of doing a retrofit action. also, I try to standardize on the tools that are using in the Data Warehouse department (currently we are using only OWB).
I will appreciate for any other advice to help in my selection process.
Best Regards,
Bilal

Hi,
In my experience this choice depends on the implementation of the datawarehouse. If you are building a "pure" Kimball style dimensional data warehouse you should be able to do this using OWB. I have architected such a DW in the past using only OWB, so I am speaking from experience.
If on the other hand you are planning to implement an Inmon style CIF, if your requirements includes an operational data store (ODS), or if you for any other reason anticipate that you are going to be doing a lot of ER modeling, then I would not recommend using the current release of OWB for modelling. (Note however that there are significant improvements to the modelling capabilities in the Paris release of OWB, so this may change in the future)
The advantage of improved maintainability when using a single tools needs to be weighted against the improved functionality if you choose a combination of the two. In the "two tool" scenario strict development and deployment routines need to be enforced to avoid that the model in Designer comes out of sync with the metadata in OWB. (Consider the effect of a developer making a change to a table definition in OWB and deploying it directly to the database without updating the model in Designer.)
Hope this helps.
Regards,
Roald

PAS - Connection to Oracle data warehouse

Hi,
We are looking to connect to an Oracle data warehouse and have installed an Oracle client on the server.
We created the link ID and are using the built in connection to Oracle OCI.DLL. However we get an error back saying that it could not load the OCI.DLL
Any ideas?
Colin Cooper

Hi Robert,
I'm confused Pedro. What is the NW Database Instance? Is this another installation of CE? Or is this the relational database running behind CE? More confusing is the fact that you say SQL Server is installed on the "Database Server". If SQL Server is on this machine, why is Oracle in the picture?
Because of the system landscape this customer have, we use a distributed system instalation of NetWeaver (NW), that is composed by:
- Central services instance (SCS)
- Database instance (DB)
- Primary application server instance
The database instance is the one installed in the Database Server.
Also MS SQL Server is installed in the same server, because it's used as sytem database for SSM (store all the metadata of SSM).
Finnaly, Oracle is the database used by the Datawarehouse, the place we should connect to get all the data to load into PAS database. This is a different server.
I understand that. My question is, does Oracle have a fix for this issue available? Have you checked metalink to see whether or not a recent Oracle patch set, or the most recent Oracle client, already has a fix for this issue?
I did not check that. I will check with the customer what is the version currently used and if exists any fix this issue.
I have a machine running Windows 2003 Server 64 with the 32bit Oracle client installed and have no issues with setting up an ODBC connection to Oracle. What I had to do though was ensured that I ran "C:\WINDOWS\SysWOW64\odbcad32.exe" and not the standard "%SystemRoot%\system32\odbcad32.exe" which is what the ODBC applet in control panel usually calls. Because you're running 32bit Oracle, you have to run the 32bit ODBC applet in order to successfully create the System DSN which can be called by PAS.
That's exactly what I need. As soon I try this I give you a feedback.
Thanks
BR
Pedro

Microsoft OLE DB Provider for Oracle: Data type is not supported.

I got the error:
Microsoft OLE DB Provider for Oracle: Data type is not supported.
Shortly after upgrading from Oracle 8 to Oracle 9. I was advised to download more up to date oracle drivers, but I was wondering if there was a way to tell what version of the 'OLE DB Provider for Oracle' is already at. Is there a command I can use via SQL Plus or something?

I have found Microsoft ODBC for Oracle to be more stable than the Microsoft OLEDB for Oracle driver. I have also found both Microsoft ODBC and OLEDB drivers to be more stable than the drivers from Oracle.
You could always get the latest MDAC (Microsoft Data Access Components) from Microsoft's MSDN Download site and then get the ODAC (Oracle Data Access Components) from Oracle's OTN Download site. ODAC requires MDAC. And ODAC has the latest drivers.
I suppose it would help to have the latest patches for your Oracle client software too. Maybe Oracle MetaLink would have these?
It may even help to have the latest service pack for Visual Studio 6 (Visual C++ 6 and Visual Basic 6) too.

Oracle Data Warehouse DB and OLAP_OBIEE DB [OBIEE11g]

Hi Experts,
I have a problem regarding joining the 2 DB in an Analysis report.
The scenario is this:
Oracle Data Warehouse is DB_1 and OLAP_OBIEE is DB_2
DB_1
-col1
-col2
DB_2
-col3
In Analysis report when they are queried or run separately, obviously there will be a report generated.
But when this 2 DB columns are joined together in one Analysis report, I always get 'No Results'.
Please help how do I fix this issue :(
Thanks,

Hi,
check the following links:
http://www.rittmanmead.com/2007/10/reporting-against-multiple-datasources-in-obiee/
http://108obiee.blogspot.com/2009/01/fragmentation-in-obiee.html
http://gerardnico.com/wiki/dat/obiee/multiple_subject_area

Where to find best practices for tuning data warehouse ETL queries?

Hi Everybody,
Where can I find some good educational material on tuning ETL procedures for a data warehouse environment? Everything I've found on the web regarding query tuning seems to be geared only toward OLTP systems. (For example, most of our ETL
queries don't use a WHERE statement, so the vast majority of searches are table scans and index scans, whereas most index tuning sites are striving for index seeks.)
I have read Microsoft's "Best Practices for Data Warehousing with SQL Server 2008R2," but I was only able to glean a few helpful hints that don't also apply to OLTP systems:
often better to recompile stored procedure query plans in order to eliminate variances introduced by parameter sniffing (i.e., better to use the right plan than to save a few seconds and use a cached plan SOMETIMES);
partition tables that are larger than 50 GB;
use minimal logging to load data precisely where you want it as fast as possible;
often better to disable non-clustered indexes before inserting a large number of rows and then rebuild them immdiately afterward (sometimes even for clustered indexes, but test first);
rebuild statistics after every load of a table.
But I still feel like I'm missing some very crucial concepts for performant ETL development.
BTW, our office uses SSIS, but only as a glorified stored procedure execution manager, so I'm not looking for SSIS ETL best practices. Except for a few packages that pull from source systems, the majority of our SSIS packages consist of numerous "Execute
SQL" tasks.
Thanks, and any best practices you could include here would be greatly appreciated.
-Eric

Online ETL Solutions are really one of the biggest challenging solutions and to do that efficiently , you can read my blogs for online DWH solutions to know at the end how you can configure online DWH Solution for ETL using Merge command of SQL Server
2008 and also to know some important concepts related to any DWH solutions such as indexing , de-normalization..etc
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927103-data-warehousing-workshop-2-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927173-data-warehousing-workshop-3-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
Kindly let me know if any further help is needed
Shehap (DB Consultant/DB Architect) Think More deeply of DB Stress Stabilities

Someone has nomenclature (naming conventions) for oracle data integrator?

Someone has nomenclature (naming conventions) for oracle data integrator?

You should really move this question over to the Oracle Data Integrator forum(Data Integrator for a more expedient response.

Tools required to install to work with Oracle Data warehouse

Could anyone please guide me...as I am quite new to Oracle...
till now I was working on Data warehousing using Microsoft tools i.e. (SSIS, SSRS, SSAS, SQL Server)
So Please tell me which are the tools required to install to work with Oracle Data warehouse
I came to know about OWB and OBIEE, but not sure what exactly these are ?

In order to create and develop a webcenter portal application, you only need to install JDeveloper and download the WeBCenter extensions. That's true.
But this install does not include the WebCenter services like dicussions, document services, analytics,... If you want to use these and integrate them in your portal, then you would need to install the suite as described on my blog.

2-3 things to watch out for when SAP is the source for a Data Warehouse

Hello
What are 2-3 things to watch out for when SAP is the source for a Data Warehouse (Informatica for ETL and Cognos for reporting)?
Thanks
G. Vijay

Going through some or all of this might help:
Empty Safari's cache (from the Safari menu), then close Safari.
Go to Home/Library/Safari and delete the following files:
form values
download.plist
Then go to Home/Library/Preferences and delete
com.apple.Safari.plist
Repair permissions (in Disk Utility).
Start up Safari again, and things should have improved.
If not, MacFixit have published a very detailed (very!) article on speeding up a slow Safari, here:
http://www.macfixit.com/article.php?story=20070416000657464
Many, including me, have also followed the advice given by others here to add DNS codes to their Network Settings, with good results in terms of speed-up:
Open System Preferences/Network. Double click on your connection type, or select it in the drop-down menu. Click on TCP/IP and in the box marked 'DNS Servers' enter the following two numbers:
208.67.222.220
208.67.220.222
Click on Apply Now and close the window.
Restart Safari, and repair permissions.

Oracle for a Data Warehouse & Data Mining Project

Hello,
I am working for a marketing company and we have a pretty big OLTP database and my supervisor wants to make use of these data for decision making. The plan is to create a
data mart(if not a warehouse) and use data mining tools on top of it.
He does not want to buy one of those expensive tools so I was wondering if we could handle such a project just by downloading OWB and Darwin from Oracle site? None of us are data warehouse specialists so it will be very though for us. But I would like to learn. Actually, I was looking for some example warehouse + mining environment implementations to get the main ideas. I will appreciate any suggestions and comments.
Thank you

Go to
http://www.oracle.com/ip/analyze/warehouse/datamining/
for white papers, demos, etc. as a beginning.
Also, Oracle University offers a course on Oracle data Mining.

Should we be using RAC for a data warehouse?

We have an Oracle 11.1 data warehouse system. We were having some performance issues with the system so we shutdown one of the RAC nodes, to see if that was causing the problem. The problem was slow updates on a table (all 30+ million rows on one table had to be fixed). One other perforamnce problem is queries of large partitoned tables (even if the partitioin key is used). Both bulk collect and bulk inserts are very fast.
Question: for a 11.1 data warehouse system should we use RAC? Why?
Thank you...

a school of thought that suggests RAC potentially decreases system availability, rather than increasing it.RAC also has the potential of increasing availability. The potential "cuts both ways", so to speak.
I've worked with non-RAC and RAC databases on a variety of platforms. My experience doesn't show evidence that RAC decreases availability. Given that most servers, even in non-HA clusters, are very reliable (generally), downtime is low in both non-RAC and RAC environments. However, RAC does provide an availability advantage -- protection against node outage. And there are environments which do require the avaialability of RAC. Not all applications require it. RAC is too oversold, not in terms of advantages but in terms of installations.
the increased complexity and the increased risk of both software and human related errors in a RAC environmentI would say that a similar argument arises in DASD v SAN. A SAN is more complex. Human error on a SAN causes a much higher cost. Human error does occur on a SAN. However, no one rejects a SAN on these grounds alone.
RAC is complex to implement. It requires more skills to adminster and diagnose. However, if it is setup well, it doesn't suffer outages. An outage from human error is the same as in a non-RAC environment.
The issue isn't RAC. The issue is that too many customers buy RAC without evaluating seriously whether
a. they need the additional minute increase in availability
b. whether there applications are "RAC-aware" {TAF is still misunderstood}
c. whether they have the skills
RAC provides scalability. It also provides HA. Let me say that again : It also provides HA.
I've seen a high end Failover Cluster environment where one of the "best" vendors in the world talked of a 10-30minute outage for the Failover.
Hemant K Chitale
http://hemantoracledba.blogspot.com
Edited by: Hemant K Chitale on May 31, 2009 11:41 PM

How create oracle data Warehouse

hi master
sir i am use oracle 9i how i creat data Warehouse
and how use oracle portal for net
please give me idea how i start and which woftware i use
thank's
aamir

Do you want to craete a Data Warehouse database or you want to learn on how to design a Data Warehouse ?
Creating a Data Warehouse database is no different than creating an OLTP database, the only thing in mind should be is that DW are mostly read-only, so you should set initialization parameters suiting DW .

Generic Java for Oracle Data Integrator 10g (10.1.3.5.0)

Hi,
How can I install the Oracle Data Integrator Client in my MacBook. So the generic version should be the right one.
But there are versions for windows, linux, solaris, HPUX and AIX.. but no generic java version.
The version 10.1.3.4.0 has an generic version.
How can I fix this issue, since I tried the linux version but doesn't install.
Thanks.

Hi,
Install the 10.1.3.4 and make an "manual install" after that. It's means copy the oracledi directory over the old oracledi directory.
There is detailed instruction on install manual...
Did you already tried this?
Cezar Santos
[www.odiexperts.com]

Can I change the SQL Datatype for Oracle Date?

xMII 11.5 maps Oracle's Date type to its SQL type code 91 (Date). When used in a display template, this xMII type displays only the date component (for example, "5/5/2007"), but in Oracle it holds the date and time. In Java, the java.sql.Date type also holds the time component. To get xMII to display a date/time (for example, "5/5/2007 11:23:59") you must have a SQL type code of 93 (Timestamp).
Is there any way to get my xMII widgets to show the date and time components of an Oracle Date? Is there a mapping of some kind that tells xMII which database SQL types map to which xMII SQL type codes?
-tim

In Aqua Data Studio:
SELECT CREATED_DATE_TIME, MESSAGE FROM COLLABORATION_LOG
where CREATED_DATE_TIME > to_date('01/01/07','mm/dd/yy') and site='DEM1'
order by CREATED_DATE_TIME DESC
GO
results:
CREATED_DATE_TIME   MESSAGE
5/3/2007 8:30:25 PM   (403)Access Forbidden
5/3/2007 8:30:25 PM   (403)Access Forbidden
5/3/2007 8:30:25 PM   (403)Access Forbidden
If I put the exact same SQL (without the "GO" at the end which is an Aqua Data Studio keyword) into a Fixed SQL Query in xMII and click "Test" and choose html format I get:
CREATED_DATE_TIME MESSAGE
05/03/2007 00:00:00 (403)Access Forbidden
05/03/2007 00:00:00 (403)Access Forbidden
05/03/2007 00:00:00 (403)Access Forbidden
The time component is reported as "00:00:00" in xMII which is not the desired result. I want xMII to see that column as a Timestamp which should read and sore both the date and time components in xMII.
-tim

Recommended throughput for Oracle data warehouse

Similar Messages

Maybe you are looking for