Architectural Design - New Data Warehouse

Hello All,
This is my first post to the oracle discussion forums and I'm looking forward to the interactions with other ODWB users.
I am just begining to implement a design for a new data warehouse, our team has already defined user requirements for a subset of the business (Sales/Marketing) and have committted a logical model to paper. We have installed our dev environment and are now ready to begin the work of creating our prototype.
I've read all the Oracle doc I can get my hands on regarding implementing your DW objects and have been pondering the approach. ROLAP or MOLAP.....
it seems to make sense that we should deploy into a ROLAP environment bringing in all our data from our staging area to create a stable relational data store. Then select most used or queried dimensions and facts to deploy in a MOLAP environment... has anyone used this approach? any lessons learned? do you have to choose one method or the other? or can you take a blended approach ? would you deploy both in the same database instance or seperate the two?
thx

I'm somewhat new to OWB coming from an Informatica background but in our environment, we are doing the same thing. Our Enterprise Data Warehouse will be based on ROLAP and I intend to use MOLAP for subsets of the EDW.
Dimensions in Oracle are somewhat interesting in that they are "leveled" and you can tie cubes or "fact tables" to any level of the dimension. This is a bit un-Kimball-like and has taken some getting used to. I think it is a powerful feature but I will have to experiment some until I understand it better.
One critical bug with 10.2 I've run into is with dimension roles - The time dimension for instance. Typically this is one table that is aliased many many times. If you exceed roughly 5 roles for the time dimension, the generation of the object fails since OWB generates a single anonymous PL/SQL block that exceeds 64k. Its a documented bug in development with no workaround according to metalink.
Other gotchas are that table changes always try to generate "create table" scripts even if you only add an index or change parallelism. We have had to do table maintenance outside OWB and then keep the metadata in sync up until now.
I haven't done any of the MOLAP yet but from what I read there are some restrictions - such as you can't have roles on dimensions for MOLAP and I believe you can't have SCDs in MOLAP. I don't know how Time dimensions are handled in MOLAP without roles! Do people really generate tables for every single time dimension in OWB???
Hope you share your experiences here!
- Mike Taylor

Similar Messages

Design the data warehouse around the reporting system?

Hi All,
A Jr. data warehouse developer resisted my suggestion to flatten out activity tables of differing grains into a single fact table. (Think sales order header, sales order detail, and even a 3rd level of details to each sales order detail.) Although
he agreed that flattening out the fact tables into a single fact would be proper for a data warehouse, he's concerned that report developers will have an easier time querying the data warehouse with the 3 separate fact tables. I'm not sure if it's because
the report developers don't like learning new schemas or if their reporting tool is just severely limited, mainly because I've never used Cognos. I assured him that a properly-designed data warehouse will save on query execution time, but he's concerned
about the reporting tool and how it may not work so well with the data warehouse.
Did I give him the proper advice? It seems like a data warehouse should be built properly regardless of reporting tool shortcomings. Assuming this tool is lousy, maybe they need a new reporting system for their new data warehouse.
Thanks,
Eric

Hi Eric,
one of the hard and fast rules of building a data warehouse is that from a logical point of view the fact table presents data at a certain level of granularity and that you do not mix facts in fact tables. This is data warehousing 101.
From your comment you seem to be suggesting mixing data of different granularity in the one table.
Now, we have ways and means of co-habiting data that will appear as different fact tables in the one physical table. We control the physical placement of data in fact tables. But on SQL Server we would never mix facts at different granularities or representing
different data in the one fact table. SQL Server supports that quite poorly.
It is sad that in 2015 people are still messing up data warehouse project from pure ignorance of what is available. We have data warehouse data models that are extremely extensive but people just have to start from scratch and reinvent the wheel and fail over
and over again. Sad but true.
Best Regards
Peter Nolan

Service Manager 2012 R2 UR4 new Data Warehouse server

The instructions for SCSM 2012 R2 UR4 say that the Data Warehouse needs to sync with SCSM at least once prior to upgrading. How is this possible if I already have all management servers at UR4 and am now trying to add a new Data Warehouse server?

I recently did the same thing. Installed a Management Server to UR4, used it for a while and after i wanted to add a Data Warehouse-server. I just installed the DW-server added it to SCSM waited for all the syncs to be completed and then upgraded.
Do you have any errors in the operations manager log on the SCSM mangement server and on the DW management server?
Cheers,
Thomas Strömberg
System Center Specialist
Blog:
Twitter: LinkedIn:
Please remember to 'Propose as answer' if you find a reply helpful

How to create a new data warehouse?

Hi,
I have a set up where multiple SCOM management groups all report to a shared data warehouse database. For various reasons, we need to “unplug” the management groups from this shared resource and give them each their own data warehouse to use (locally, rather
than centrally).
Is there a way of creating a new clean data warehouse without running a full installation? I’m imagined that I could probably put a copy of the existing shared one into each local management group and then use this process (http://technet.microsoft.com/EN-US/library/hh268492.aspx)
to repoint each locally – but it would be nicer to start fresh for each one. Is that possible, is there an installation I can run for JUST the DW elements of the SCOM installation?
Thanks.

Breaking a shared data warehouse configuration into separate data warehouses is really not supported - there is no installation path for this and no guidance available. If you absolutely need to do this, then you are basically stuck with reinstalling the
data warehouse and reporting server for each management group. You will lose all historical reporting in this scenario. Sure, the data can be saved and used elsewhere, but it will not be available as you would expect to see it in the reporting space of the
Operations Console.
Moving the data warehouse as Mai suggested will not solve the problem, and may actually end up in in other problems with retention and reports being unavailable for some instances. I would not suggest moving/copying the existing data warehouse to each management
group.
Jonathan Almquist | SCOMskills, LLC (http://scomskills.com)

Is RAC suited for Data Warehouse

Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?

VahidS wrote:
Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?
I am not sure that which documents you have read based on which you have got the impression that RAC is for DWH. RAC is a part of the Oracle's High Availability solution architecture. It's not designed to cater any specific kind of workload but to provide HA in the case of a node or instance crash. And also, RAC itself doesn't give any performance benefit. If you have a database which is poorly performing, it would remain the same or may be , will become even worse after implementing RAC due to too many things being involved in the architecture now e.g storage, network etc. So if you do want to implement RAC, give it a careful thought, do some testing and then only project it. If you are determined to go for RAC and you are a pure DWH shop, I would suggest to go fro Exadata running RAC . Smart Scan, Compression are just few of those benefits which would come with Exadata along with the RAC's HA solution that would be serving better for your requirements.
HTH
Aman....

Unable to register my data warehouse in Service Manager

I have been trying to register my data warehouse but keep getting the same error message each time - "Invalid URI: the hostname could
not be parsed." I know the issue is on the ServiceManager database side of things, but there is not a lot of information related to this error message and the info I do find is unrelated to ServiceManager. I have gone as far as building a
brand new data warehouse server and reinstalling all of the data warehouse databases from scratch. It doesn't matter if I tried to register with the original data warehouse environment or the new environment, I get the same error message. I also
received a powershell command from Microsoft Support to help clean up any residual entries in the ServiceManager database that might be left over. At this point I'm at a loss on how to proceed. Anyone ever run into this issue?
Here's the event log message:
Unable to register Service Manager installation with Data Warehouse installation.
Data Warehouse Server: DW_SCSM
Service Manager Management Server: FlowSMRC
Exception: Microsoft.EnterpriseManagement.UI.Core.Shared.PowerShell.PSServiceException: Invalid URI: The hostname could not be parsed.
at Microsoft.EnterpriseManagement.UI.Core.Shared.PowerShell.PSHostService.executeHelper(String cmd, Object[] input)
at Microsoft.EnterpriseManagement.UI.Core.Shared.PowerShell.PSHostService.Invoke(String cmd, Object[] input)
at Microsoft.EnterpriseManagement.ServiceManager.UI.Administration.DWRegistration.Registration.DWRegistrationHelper.AcceptChanges(WizardMode wizardMode)
Also, I tried registering the data warehouse using powershell Add-SCDWMgmtGroup and get an error saying that Cannot associate Service Manager installation on scsm_server with Service Manager Data Warehouse installation on dw_server.

Just got additional information about this error. If you run the following query in SQL pointing to the ServiceManager database you should see the name of the Management Server. The query is:
select * from MT_Microsoft$SystemCenter$ResourceAccessLayer$SdkResourceStore
There is a column named Server_<some GUID> that should contain the Management Sever NetBIOS name. If it has some other name, run an update query to ensure the column has the NetBIOS name of the Management Server. Restart the SCSM Console
and retry the Database Registration again and it should succeed.
Microsoft has said that this is normally caused by either moving the database or database server to another server and the value is never updated.
Try registering the DW Server again and see if you have better luck.

Data warehouse database


Today I came across one very interesting question


"Data Warehouse can be only deployed in relation database."


Above statement is true or false.


If we see it simply without any complication or may be go back 7-8 years it's answer may be false as I find out after doing research on it


"A data warehouse can be normalized or denormalized. It can be a relational database, multidimensional database, flat file, hierarchical database, object database, etc. Data warehouse data often gets changed. And data warehouses often focus on a specific activity or entity." Larry Greenfield


The data warehouse is normally (but does not have to be) a relational database. It must be organized to hold information in a structure that best supports not only query and reporting, but also advanced analysis techniques, like data mining. Most data warehouses hold information for at least 1 year and sometimes can reach half century, depending on the business/operations data retention requirement. As a result these databases can become very large.en.wikipedia.org


But I think when we look into the complication of designing and functionality which we are expecting from a data warehouse today and plus the concept which use when designing the data warehouse structure like star scheme, snow flake etc., I think it cannot be done in any other type of database which don't follow the relation database concept. We may call it multidimensional database or ORDBMS and we may talk about cubes, measures, dimension and so on but from the base it has to follows the relation database.


Let me know if anybody has anything to say about it.

 
Regards,
 
Raj 
www.oraclebrains.com<a>
 POWERED by the people, to the people and for the people WHERE ORACLE IS PASSION.

Thanks Justin!
 
I agree with you the concept exist before the relational database was invented. Last time I know they call it EIS then they name it MIS and now data warehouse. But what I am taking about is modern data warehouse technique, If you really think that any other type of database can support it without following relational database concept. Let me know which one?
 
I am still searching for my answers and have discuss with lot of people, but when I asked them have they seen any implementation of data warehouse without using relational database, the answer that I get is always negative. 
Raj 
www.oraclebrains.com<a>
 POWERED by the people, to the people and for the people WHERE ORACLE IS PASSION.

SQL Server Data Warehouse Creation External or Build Your own

Hi,
In our firm we have a data warehouse which has been created by an external supplier.
The warehouse is being used for some reporting and dashboards but only for a single client, we have yet to fully utilize it.
My question is should we build a new data warehouse or should we push forward with the data warehouse created by the external supplier. I know it would take some time to build a data warehouse however it means we will have full control over it and could
overcome any issues we have with it in house.
It would really help if someone could offer some advice.
Thanks

Hi Nasa1999,
I am trying to involve someone more familiar with this topic for a further look at this issue. Sometime delay might be expected from the job transferring. Your patience is greatly appreciated.
Thank you for your understanding and support.
Regards,
Katherine Xiong
Katherine Xiong
TechNet Community Support

How create oracle data Warehouse

hi master
sir i am use oracle 9i how i creat data Warehouse
and how use oracle portal for net
please give me idea how i start and which woftware i use
thank's
aamir

Do you want to craete a Data Warehouse database or you want to learn on how to design a Data Warehouse ?
Creating a Data Warehouse database is no different than creating an OLTP database, the only thing in mind should be is that DW are mostly read-only, so you should set initialization parameters suiting DW .

SQL Server 2012's Parallel Data Warehouse (PDW) Optional Add-on?

Hi, I work for the State of Illinois and though we have progressive IT shop, we tend to be a bit behind the curve. We are still using SQL Server 2005 for all our data needs and have been planning to move to 2012 for the past two years (may actually
occur soon, even though I have been trying to influence our department to just go straight to the new SQL Server 2014). Because I was selected to work in our new Data Warehouse team, I have a keen interest in learning what I can about the PDW.
However, I suspect that just because we are moving up to 2012 does not mean that any PDW licensing will automatically be included, or does it? Can someone point me to an article on PDW licensing (I have seen something hinting at more than one
level of PDW licensing)?

Hi Ron. PDW is a scale-out SQL Server data warehouse workload on the Microsoft Analytics Platform System appliance. You can learn more about APS here (http://www.microsoft.com/aps). As for the licensing, PDW is licensed separately from SQL Server 2012/2014.

Could you please recommend a book or two about data warehouse designing?

Want to read some books about data warehouse and how to build or deal with the problems during the data warehousing process.
Anyone could recommend any book regard to this?
I want the book to mainly talk about the common case scenarios in data warehouse area and the general solutions to those scenarios.
I am quite new in this area, so any recommendation would be highly appreciated.
Thanks.

Perhaps also these resources, if you've not already seen them
DW Best Practices Whitepaper
http://www.oracle.com/technetwork/database/features/bi-datawarehousing/twp-dw-best-practies-11g11-2008-09-132076.pdf
Greg Rahn on the core performance fundamentals of Oracle data warehousing
http://structureddata.org/2009/12/14/the-core-performance-fundamentals-of-oracle-data-warehousing-introduction/

Migrating SQL Management Data Warehouse to a new server

Is there a defined path for migrating a Managment Data Warehouse to a instance?
Does Microsoft provide any scripts for this? Is it a case of backing up and restoring databases to the new instance? And what about all the data collection sets running on the monitored instances? Do we have to re-point these to the new instance hosting
the MDW individually by logging on teach monitored instance? Or can we do this globally somehow?
Thanks.

Hi NiborTrassam,
Sorry the delay.
To migrate the Management Data Warehouse, you can back up the MDW database, and restore it on the new instance. Then, on the new instance, you need to start the Configure Management Data Warehouse Wizard to create a Management Data Warehouse based on the
migrated MDW database, as well as upgrade the login mapping if necessary.
On each instance that is monitored, you need to start the Configure Management Data Warehouse Wizard to setup new data collection to point to the MDW data warehouse on the new instance.
Regards,
Mike Yin
If you have any feedback on our support, please click
here
Mike Yin
TechNet Community Support

How do I design this in data warehouse?

I am working on building a data warehouse for insurance quote data.
Each quote will have an applicant and can have an optional co-applicant. Each applicant and co-applicant will have prior auto insurance history, prior home insurance history, current auto insurance information and current home insurance information.
So do I create Applicant and Insurance dimensions here?

Hi Ashan,
Just so you know.
I completely reworked our methodology of building data warehouses back in 2012. The new way of building data warehouses is quite different to the old way. The way you listed.
The methodology presentation is on this link.
https://www.youtube.com/watch?v=Df4CgOtrFq8
Video channels are here. http://www.instantbi.com/videos/
Downloads are here: http://www.instantbi.com/company/downloads/
I have been doing BI since 91 and what we have done now is industry leading.
I am an MSDN so we do our development on MSFT first and then deploy where ever our clients want us to deploy.
Best Regards
Peter Nolan

Designer Vs. Oracle Data warehouse builder

Dear all,
Currently I'm responsible of building a Data warehousing project using Oracle database. I'm trying to decide on a tool for modelling my datawarehouse. I have two options:
1) Designer: we have some experience with this tool and we are using it for our main OLTP application.
2) Oracle Data Warehouse builder: we are using this to design our ETL processes.
I want to get some advice on whether the OWB is capable of modelling my datawarehouse and of doing a retrofit action. also, I try to standardize on the tools that are using in the Data Warehouse department (currently we are using only OWB).
I will appreciate for any other advice to help in my selection process.
Best Regards,
Bilal

Hi,
In my experience this choice depends on the implementation of the datawarehouse. If you are building a "pure" Kimball style dimensional data warehouse you should be able to do this using OWB. I have architected such a DW in the past using only OWB, so I am speaking from experience.
If on the other hand you are planning to implement an Inmon style CIF, if your requirements includes an operational data store (ODS), or if you for any other reason anticipate that you are going to be doing a lot of ER modeling, then I would not recommend using the current release of OWB for modelling. (Note however that there are significant improvements to the modelling capabilities in the Paris release of OWB, so this may change in the future)
The advantage of improved maintainability when using a single tools needs to be weighted against the improved functionality if you choose a combination of the two. In the "two tool" scenario strict development and deployment routines need to be enforced to avoid that the model in Designer comes out of sync with the metadata in OWB. (Consider the effect of a developer making a change to a table definition in OWB and deploying it directly to the database without updating the model in Designer.)
Hope this helps.
Regards,
Roald

OBIEE Data warehouse Architecture

Is it common to install the BI server and Database on different servers or both on the same.
I have installed OBIEE on OEL Version 5 update 2 64 bit and oralce's Retail Data Warehouse RDW on an other server OEL Version 5 update 2 64 bit.
There is a bug that requires a 32 bit client to be installed on the Database. The fix is to create a soft link between the OracleBi/server/Bin/xxxxx.so file on BI server to $ORACLE_HOME/client_32 lib32 libraries xxxxx10.1.so file. I don't see anywhere in the documentation of creating a softlink beween server so I am thinking maybe it's not possible to have the database and BI server on different servers.
Thanks.

The error I got was
nQSError: 46029 Failed to load the DLL /u01/app/oracle/OracleBI/server/Bin/libnqsdbgatewayoci10g.so. Check if 'Oracle OCI 10G' database client is installed.
Fix
The symlink needs to go from libclntsh.so (which is the generic library name OBIEE will try to load) to your libclntsh.so.xxx library (libclntsh.so.10.1 in your case).
This is from another post Re: HPUX 11.31 / OBIEE 10.1.3.4 / 11g client - Unsatisfied code symbol error
However this is referring to both database and server being on the one server.

Architectural Design - New Data Warehouse

Similar Messages

Maybe you are looking for