TUNING DATA WAREHOUSE DATABASE INSTANCE

Hi,
I have to tune one of the DATA WAREHOUSE DATABASE INSTANCE.
Any advice for tuning this instance.
How different is tuning the data warehouse instance than normal instance;
Regards

First of all, touch nothing until you understand what your users are doing with the data warehouse, when they are doing it and what their expectations are.
Secondly, remember that a data warehouse is, generally, much bigger than an OLTP database. This changes the laws of physics. Operations you might expect to take a few minutes might take days. This means you need to be completely certain about what you do in production before you do it.
Thirdly, bear in mind that a lot of data warehouse tuning techniques implement physical solutions objects - different types of indexes, partitioning - rather than query tweaking. These things are easier to get right at the start than to retrofit to large volumes of data.
Good luck, APC

Similar Messages

Only Alert Data is not being inserted in SCOM 2012 Data Warehouse database

Hi All,
Alert data is not getting inserted into SCOM Data Warehouse database since 10 days though I could see latest Performance data in DW DB. No changes were made as far I know on SCOM servers or DB's. I had this issue few months back
and got resolved by executing a qiery to create an entry for Data Warehouse Synchronisation server entry.
Now I have checked the discovered inventory and could see an entry present and it is healthy. Still, latest Alert data is not getting inserted into DW DB. Please help me out.
http://social.technet.microsoft.com/Forums/en-US/2dac4f45-4911-40dc-a220-702993188832/alert-data-is-not-present-in-scom-2012-data-warehouse-database-since-two-weeks?forum=operationsmanagergeneral
Regards, Suresh

Hi,
Generally, data warehouse store a long-term data, and by default, it would keep 400 days data, I suggest check your configuration:
How to Configure Grooming Settings for the Reporting Data Warehouse Database
http://technet.microsoft.com/en-us/library/hh212806.aspx
Alex Zhao
TechNet Community Support

Alert data is not present in SCOM 2012 Data Warehouse database since two weeks

Alert data is not present in SCOM 2012 Data Warehouse database since a week though I could see Performance data for the latest dates. Old Alert data is present but I think the latest Alert data is not being inserted to Data warehouse. No activity was done
on the day from where we are missing data.
I could see 31554 events on all my Management servers and this proves that Data Insertion is happening. I am not sure why only Alert data is missing (or not getting inserted) in DW database. I am trying to use SQL queries to fetch the data as I dont have reporting
currently. The same query is working for other dates, so there is no issue with this query.
I have noticed that I could see the Alert Data present in SCOM OperationsmNager Db but NOT present in OperationsManagerDW database.
In SCOM 2007, data will be inserted in both Ops DB and DW simultaneously. I believe the same methodology in 2012 too.
Please help me to fetch Alert data from DW. Any suggestion pls?
Regards, Suresh

Hi,
Generally, data warehouse store a long-term data, and by default, it would keep 400 days data, I suggest check your configuration:
How to Configure Grooming Settings for the Reporting Data Warehouse Database
http://technet.microsoft.com/en-us/library/hh212806.aspx
Alex Zhao
TechNet Community Support

Advice on implementing oracle streams on RAC 11.2 data warehouse database

Hi,
I would like to know high level overview on implementing one-way schema level replication within same database using oracle streams on RAC 11.2 data warehouse database.
Are there any points that should be kept in mind before drafting the implementation plan.
Please share your thougts and experiences.
Thanks in advance
srh

Hi,
I would like to know high level overview on implementing one-way schema level replication within same database using oracle streams on RAC 11.2 data warehouse database.
Are there any points that should be kept in mind before drafting the implementation plan.
Please share your thougts and experiences.
Thanks in advance
srh

Spatial tables in Data warehouse database or separate?

Hi all,
I have a question from a customer who's asking what the recommendation is of storage of the spatial tables and the warehouse tables.
Should they reside on the same database for best performance or doesn't it matter where the spatial tables are located.
Is there any official recommednation from Oracle on this?
Best regards,
Hakan

Hi Hakan,
it's not easy to give a quick answer here.
a) you want to compare warehouse tables with spatial tables
To answer this very well you should define what is a warehouse table and what is a spatial table. In this forum you will get for spatial tables the definition "A spatial table is a table with one or more columns of type SDO_GEOMETRY". How do you define a spatial table and how do you define a warehouse table?
b) you ask for performance
Performance is not depending only on table type. Performance is depending on a lot of steps. For example on the work which has to be done to answer the question, can data read from buffer cache or must they be read from physical devices, execution plan .....
I don't know if there is a official statement from Oracle which is usefull to answer your query.
Can you ask a little bit particular please?
In my experience there it is not adviseable to store tables in different databases (instances) when I have to query it together. If you split the tables in different instances, the execution has much more overhead because you have to talk to 2 instances and it will very hard for the optimizer to find the best execution plan.
Regards
U. Martin

ACI Setup - How to Configure Data Warehouse Database - Partitoning

After reading the ACI Install Guide & Data Warehouse documentation, I have some questions regarding how to setup the database:
- Should database partitioning be setup? If so, what tables should be partitioned and what should they be partitioned by?
- Are there any other best practices or tips for setting up & tuning the database?
We are trying to avoid the (painful) situation of having to add partitioning later on; it is much easier to add it up front (if done correctly up front).
Thanks in advance for any advice!

On the tables recommended for partitioning, the partition key is nullable. If ATG inserts a null value into the timestamp column of one of the partitioned tables, we'll receive an ORA-14300 or ORA-14440 error. Oracle isn't able to figure out what partition to map that record to.
Can the columns be changed to NOT NULL? Or, can the application guarantee a nullable value won't be inserted?
Here are some example columns:
ARF_SITE_VISIT.START_VISIT_TIMESTAMP --> TIMESTAMP(6) null
ARF_REGISTRATION.REGISTRATION_TIMESTAMP --> TIMESTAMP(6) null
ARF_LINE_ITEM.SUBMIT_TIMESTAMP --> TIMESTAMP(6) null
ARF_PROMOTION_USAGE.USAGE_TIMESTAMP --> TIMESTAMP(6) null
ARF_RETURN_ITEM.SUBMIT_TIMESTAMP --> TIMESTAMP(6) null
Thanks

Data warehouse database


Today I came across one very interesting question


"Data Warehouse can be only deployed in relation database."


Above statement is true or false.


If we see it simply without any complication or may be go back 7-8 years it's answer may be false as I find out after doing research on it


"A data warehouse can be normalized or denormalized. It can be a relational database, multidimensional database, flat file, hierarchical database, object database, etc. Data warehouse data often gets changed. And data warehouses often focus on a specific activity or entity." Larry Greenfield


The data warehouse is normally (but does not have to be) a relational database. It must be organized to hold information in a structure that best supports not only query and reporting, but also advanced analysis techniques, like data mining. Most data warehouses hold information for at least 1 year and sometimes can reach half century, depending on the business/operations data retention requirement. As a result these databases can become very large.en.wikipedia.org


But I think when we look into the complication of designing and functionality which we are expecting from a data warehouse today and plus the concept which use when designing the data warehouse structure like star scheme, snow flake etc., I think it cannot be done in any other type of database which don't follow the relation database concept. We may call it multidimensional database or ORDBMS and we may talk about cubes, measures, dimension and so on but from the base it has to follows the relation database.


Let me know if anybody has anything to say about it.

 
Regards,
 
Raj 
www.oraclebrains.com<a>
 POWERED by the people, to the people and for the people WHERE ORACLE IS PASSION.

Thanks Justin!
 
I agree with you the concept exist before the relational database was invented. Last time I know they call it EIS then they name it MIS and now data warehouse. But what I am taking about is modern data warehouse technique, If you really think that any other type of database can support it without following relational database concept. Let me know which one?
 
I am still searching for my answers and have discuss with lot of people, but when I asked them have they seen any implementation of data warehouse without using relational database, the answer that I get is always negative. 
Raj 
www.oraclebrains.com<a>
 POWERED by the people, to the people and for the people WHERE ORACLE IS PASSION.

Where to find best practices for tuning data warehouse ETL queries?

Hi Everybody,
Where can I find some good educational material on tuning ETL procedures for a data warehouse environment? Everything I've found on the web regarding query tuning seems to be geared only toward OLTP systems. (For example, most of our ETL
queries don't use a WHERE statement, so the vast majority of searches are table scans and index scans, whereas most index tuning sites are striving for index seeks.)
I have read Microsoft's "Best Practices for Data Warehousing with SQL Server 2008R2," but I was only able to glean a few helpful hints that don't also apply to OLTP systems:
often better to recompile stored procedure query plans in order to eliminate variances introduced by parameter sniffing (i.e., better to use the right plan than to save a few seconds and use a cached plan SOMETIMES);
partition tables that are larger than 50 GB;
use minimal logging to load data precisely where you want it as fast as possible;
often better to disable non-clustered indexes before inserting a large number of rows and then rebuild them immdiately afterward (sometimes even for clustered indexes, but test first);
rebuild statistics after every load of a table.
But I still feel like I'm missing some very crucial concepts for performant ETL development.
BTW, our office uses SSIS, but only as a glorified stored procedure execution manager, so I'm not looking for SSIS ETL best practices. Except for a few packages that pull from source systems, the majority of our SSIS packages consist of numerous "Execute
SQL" tasks.
Thanks, and any best practices you could include here would be greatly appreciated.
-Eric

Online ETL Solutions are really one of the biggest challenging solutions and to do that efficiently , you can read my blogs for online DWH solutions to know at the end how you can configure online DWH Solution for ETL using Merge command of SQL Server
2008 and also to know some important concepts related to any DWH solutions such as indexing , de-normalization..etc
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927103-data-warehousing-workshop-2-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927173-data-warehousing-workshop-3-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
Kindly let me know if any further help is needed
Shehap (DB Consultant/DB Architect) Think More deeply of DB Stress Stabilities

Access Custom Configuration Item Class property in Data Warehouse database

Is there a way to access properties on a custom 'Configuration Item Class' in the DW database?
When creating a new CI class eg. 'Monitors' and extending this with a property like 'Serial' is there a way to access this in the DW database?
I can find Displayname and AssetStatus in the ConfigItemDim table, but is there a way to get the other custom created properties?

Ok !
Are you read this post ? :
http://social.technet.microsoft.com/Forums/systemcenter/en-US/8434e129-b969-451e-875c-f120d7b31d36/dw-reporting-on-custom-class?forum=systemcenterservicemanager
Remy BOVI

Predict SGA and PG for data warehouse database

Hi
How can I predict the size of SGA, PGA and other database parameters for Datawarehouse system? What should I analyze
Thanks
John

user8779435 wrote:
It's not new datawarehose. It's has been using more than 4 years.
the reason is that some of reports runs very slowly. I want to verify and suggest something to change and solve the problemLooking for some sort of magic recommendation for your memory allocations is a classic case of Ready, Fire, Aim. (Or worse, Fire, Ready, Aim)
You need to find out what the reports are waiting on. At a minimum, TKPROF is your friend.

Database Mode for Data warehouse

I am running Oracle 10gR2 on solaris 5.10.
The recommendation that the data warehouse database should be run in dedicated mode is right?
regards

Fahd Mirza wrote:
I am running Oracle 10gR2 on solaris 5.10.
The recommendation that the data warehouse database should be run in dedicated mode is right?
regards"dedicated" is not a mode that the databse runs in, it is a type of connection the client makes to the database. There's no such thing as "run the database in dedicated mode" vs. "run the database in shared mode". The database will always accept a dedicated connection request. It may accept a shared connection request, if it has been configured to do so. It's up to the client to decide, not the database

Data Warehouse issues

Hi there.
I'll try and be to the point as I can. I undertook an upgrade from SCOM 2012 SP1 to R2, this was successful. However after a few days had past, I noticed that the log file for the DW DB was growing rapidly, to the point where we hit a real issue. The drive
was a comfortable 50GB drive, but in the end had to be increased to just short of 500GB !! Long story short, I wasn't allowed to get a consultant in to help diagnose the issue, therefore, I was consulting Google, Technet articles etc.
The end decision was to DELETE the Data Warehouse DB & log file and create them again from scratch. I've done this, and used the following instructions on how to do so (http://www.3packetsofcrisps.com/2014/04/reset-scom-2012-data-warehouse-database.html).
I have data going into the DW DB, or rather did up until midnight, so had around 3.5 hrs worth.
After checking the Operations Manager event log on one of the Management Servers, I can see numerous errors, these include:2115, 4506 and occasionally 8000.
The event 4506 states: "Data was dropped due to too much outstanding data in rule "Rule Name" running for instance "instance name" with id "ID name" in management group "xxx".
After some consulting Google, I read that this might be expected because a lot of data is trying to be written to the DW DB, however, that doesn't explain why it was writing and for some reason randomly stopped.
A quick summary of our environment:
2x Management Servers running Win 2012 Std x64
1x Operations Database server running Win 2012 Std x64 & SQL Server 2012 Std x64
1x Data Warehosue DB server running Win 2012 Std x64 & SQL Server 2012 Std x64
I've checked account permissions using http://technet.microsoft.com/en-gb/library/hh457003.aspx
If you need any more information, please say, I've tried to keep the post as small as possible.
Dave.

Hi
Thanks for responding. It's been over a day so far, however, I've only just flushed the health service state & cache for the servers I'm monitoring. Would you suggest I do this for the Management Servers as well?
In regards to alerts in the SCOM console, I am getting active alerts. I've just checked and some were 49 minutes ago.
I've also taken another look at the DW DB (via Windows Explorer), and I can see the last modified date has changed to 09/09/2014 20:48. So once again, something has happened but it appears to be random..
This might need to be a separate post, but I've also got issues with my Reporting. If I go to the 'Reporting Server URL', I get the following message: "The version of the report server database is either in a format that is not valid, or it cannot
be read. The found version is ''. The expected version is '162'. (rsInvalidReportServerDatabase)"
Any ideas?
Dave

Data warehouse monitor initial state data synchronization process failed to write state.

Data Warehouse monitor initial state data synchronization process failed to write state to the Data Warehouse database. Failed to store synchronization process state information in the Data Warehouse database. The operation will be retried.
Exception 'SqlException': Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
One or more workflows were affected by this.
Workflow name: Microsoft.SystemCenter.DataWarehouse.Synchronization.MonitorInitialState
Instance name: Data Warehouse Synchronization Service
Instance ID: {0FFB4A13-67B7-244A-4396-B1E6F3EB96E5}
Management group: SCOM2012R2BIZ
Could you please help me out of the issue?

Hi,
It seems like that you are encountering event 31552, you may check operation manager event logs for more information regarding to this issue.
There can be many causes of getting this 31552 event, such as:
A sudden flood (or excessive sustained amounts) of data to the warehouse that is causing aggregations to fail moving forward.
The Exchange 2010 MP is imported into an environment with lots of statechanges happening.
Excessively large ManagedEntityProperty tables causing maintenance to fail because it cannot be parsed quickly enough in the time allotted.
Too much data in the warehouse staging tables which was not processed due to an issue and is now too much to be processed at one time.
Please go through the links below to get more information about troubleshooting this issue:
The 31552 event, or “why is my data warehouse server consuming so much CPU?”
http://blogs.technet.com/b/kevinholman/archive/2010/08/30/the-31552-event-or-why-is-my-data-warehouse-server-consuming-so-much-cpu.aspx
FIX: Failed to store data in the Data Warehouse due to a Exception ‘SqlException': Timeout expired.
Regards,
Yan Li
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact [email protected]

How to create a new data warehouse?

Hi,
I have a set up where multiple SCOM management groups all report to a shared data warehouse database. For various reasons, we need to “unplug” the management groups from this shared resource and give them each their own data warehouse to use (locally, rather
than centrally).
Is there a way of creating a new clean data warehouse without running a full installation? I’m imagined that I could probably put a copy of the existing shared one into each local management group and then use this process (http://technet.microsoft.com/EN-US/library/hh268492.aspx)
to repoint each locally – but it would be nicer to start fresh for each one. Is that possible, is there an installation I can run for JUST the DW elements of the SCOM installation?
Thanks.

Breaking a shared data warehouse configuration into separate data warehouses is really not supported - there is no installation path for this and no guidance available. If you absolutely need to do this, then you are basically stuck with reinstalling the
data warehouse and reporting server for each management group. You will lose all historical reporting in this scenario. Sure, the data can be saved and used elsewhere, but it will not be available as you would expect to see it in the reporting space of the
Operations Console.
Moving the data warehouse as Mai suggested will not solve the problem, and may actually end up in in other problems with retention and reports being unavailable for some instances. I would not suggest moving/copying the existing data warehouse to each management
group.
Jonathan Almquist | SCOMskills, LLC (http://scomskills.com)

Segregating sections of the data warehouse solution

I'm helping to set up a data warehouse for my company and had a question about the best way to organize my databases.
I have a pretty beefy server with a lot of memory, a powerful processor and plenty of drive space. For that reason and others we plan on putting everything on a single server.
We have a few areas defined for our warehouse: staging, reporting / application data and analysis services data. Staging will be made up of several databases as will the reporting / application area. We have two options in front of us now for how we configure
our environment. Either we:
1. Put all databases on a single instance and prefix them each with, for example, Staging_ for each database (e.x. Staging_Payments, Staging_Commissions, Staging_Core). In this case databases for each area will all be together. In order to avoid confusion
about the purpose of a database (the area it belongs in) we would have to prefix the database name with the area name.
OR
2. Create a separate instance on the same server named "Staging" into which we put all of our staging databases and then we just name them after their source (e.x. Payments, Commissions, Core)
I'm partial to option 2 because it feels cleaner but I don't know if there are reasons not to do this. We will have applications and ETLs that have to query data across instances but since we're on the same server I imagine the overhead would be negligible.
I appreciate anyone's insight.

Hi Omatase,
I have set up a lot of SQL server data warehouses....we are even about to get into hosted/cloud data warehouses.
Unless you have some compelling reason your staging area should be just one database and all staging data should flow into your staging area. You can have your data warehouse on the same machine or replicate a distributed version to another machine. Its all
pretty simple now.
The data warehouse should be a separate database. You should push data from your staging database to your data warehouse database. If the machine can handle the load? Great. We are seeing that faster and faster machines are handling these loads more and more
easily.
There is no reason to have separate SQL Server instances on the one machine that I can think of. It will not give you much advantage for some pretty severe disadvantages.
These forums are not for people like me to sell our wares so if you want to talk to me in more detail about how we do these things please feel free to email me on [email protected] Ok? We have very complete ways in which we set up data warehouses and ETL
that are much faster and cheaper than anyone else.
Peter Nolan

TUNING DATA WAREHOUSE DATABASE INSTANCE

Similar Messages

Maybe you are looking for