Sort_area_size in data warehouse applications

Hi everyone,
I am working on optmizing the data warehouse application that involves with inserts and updates on bitmap indexes. How to decide proper value for sort_area_size?. I think my application is doing lot of I/Os when inserts/updates are performed on the column that has bitmap index. I tried with various values for sort_area_size with no improvement in the performance. The default sort_area_size is 5M and I tried by incrementing the value until it reached 100 MB. I could not determine the proper value for sort_area_size to have good performance. Reference to any documentation is also appreciated.
Your help is very much appreciated.
Thanks
Suresh.

Suresh,
Please refer to the following documentation:
http://otn.oracle.com/docs/products/oracle9i/doc_library/release2/server.920/a96533/memory.htm#39086
Also, this seems to be a server technology question, please post in the server technology forum for more details.
Regards:
Igor
Hi everyone,
I am working on optmizing the data warehouse application that involves with inserts and updates on bitmap indexes. How to decide proper value for sort_area_size?. I think my application is doing lot of I/Os when inserts/updates are performed on the column that has bitmap index. I tried with various values for sort_area_size with no improvement in the performance. The default sort_area_size is 5M and I tried by incrementing the value until it reached 100 MB. I could not determine the proper value for sort_area_size to have good performance. Reference to any documentation is also appreciated.
Your help is very much appreciated.
Thanks
Suresh.

Similar Messages

Which Data Warehouse application

How can I know which Data Warehaouse application can I use: Express Server, Experss Analyzer or Express Objects? Thanks

What do you mean by this question? What do you want to do?
By the way, if you are thinking of using Express products, you should look into the olap-option instead since it will replace express products if you are using 9.2 or higher.

Data Access Object for Data Warehouse?

Hi,
Does anyone know how the DAO pattern looks like when it is used for a data warehouse rather than a normal transactional database?
Normally we have something like CustomerDAO or ProductDAO in the DAO pattern, but for data warehouse applications, JOINs are used and multiple tables are queried, for example, a query may contains data from the Customer, Product and Time table, what should the DAO class be named? CustomerProductTimeDAO?? Any difference in other parts of the pattern?
Thanks in advance.
SK

In my opinion, there are no differences in the Data Access Object design pattern which have any thing to do with any characteristic of its implementation or the storage format of the data the pattern is designed to function with.
The core pupose of the DAO design pattern is to encapsulate data access code and separate it from the business logic code of the application. A DAO implementation might vary from application to application. The design pattern does not specify any implementation details. A DAO implementation can be applied to group of XML data files, an Excel-based CSV file, a relational database, or an OS file system. The design is the same for all these, it is the implementation that varies.
The core difference between an operational database and a strategic data warehouse is the purpose of why and how the data is used. It is not so much a technical difference. The relational design may vary however, there may be more tables amd ternary relationships in a data warehouse to support more fine-tuned queries; there may be less tables in a operational database to support insert/add efficiencies.
The DAO implementation for a data warehouse would be based on the model of the databases. However the tables are set up, that is how the DAO is coded.

RAC for Data Warehouse

Hello,
We have a research project for restructructuring our data warehouse system.
I would like to get some opinions about whether RAC architecture can be
a good solution for Data Warehouse application.
We have using parallel queries massively. Does running these kind of queries
on different servers on RAC with multiple server result in performance degradation rather than
running on single monolithic server with multiple CPUs
I will appreciate any comments using RAC architecture for Data Warehouse
systems?
Regards,

Maurice Muller wrote:
Just keep in mind that during the last 4 years (I guess your current system is about 4 years old) the CPUs became much faster.
A cpu can't work without data which means that the I throughput has to be fast enough to feed all your cores with data.
The main bottleneck of all DWHs I have seen during the last 8 years was allways the IO never the CPUs. And not just data warehousing Maurice, but a basic principle for any data processing platform - the slowest layer is always the I/O layer.. and can be the most expensive one to solve too.
Which is why newer technology like Infiniband is exciting as it can also serve as the I/O layer. Instead of using the traditional HBA which is typically configured with 2Gb fibre channels to the storage layer, using HCA cards you can wire this directly into an Infiniband storage array... and this can run at up to speeds of 40Gb. Dual connections means a total theoretical pipe size of 80Gb. I do not know of any other standard technology (like GigE) that can provide any similar bandwidth speed.
Back to RAC though - with RAC, when you add a new server that comes with a new set of I/O pipes.. plus of course more RAM and more CPU cores. SMP server architecture does not scale like this at all. You only have x number of slots for PCI cards, CPUs and RAM. A very specific ceiling that cannot be moved. With MPP this ceiling is a a lot higher and more flexible.
You can also replace a dual core dual CPU nodes with a 6 core AMD Istanbul CPUs next year.. and possibly 12 core CPUs year after that. So even a smallish 4 node cluster with 16 cores in total can be grown significantly and remain a 4 node cluster. Together with advances in HPC (High Performance Computing) like Infiniband.
I'm not seeing much use of non-RAC RDBMS architecture in the future. Databases are getting ever bigger because we have the technology to crunch more data, and crunch it a lot more intelligently than ever before. My first production database was 4MB in size, and ran on a Novell File Server with two 20MB disks. I'm currently testing a 24TB array for use for a single database.
Technology is inevitable, as is the growth in data volumes. And I cannot see a non-RAC architecture rising to that challenge. Especially not in something like data warehousing.

Diff b/w Data warehouse and Business Warehouse

Hi all,
what is the Diff b/w Data warehouse and Business Warehouse?

hi..
The diferrence between Datawarehousing and Business Warehouse are as follows.
DataWarehousing is the concept and BIW is a tool that uses this concept in Business applicaitons.
DataWarehousing allows you to analyze tons of data (millions and millions of records of data) in a convinent and optimum way, it is called BIW when applied to Business applications like analyzing the sales of a company.
Advantages- Consedering the volume of business data, BIW allows you to make decisions faster, I mean you can analyze data faster. Support for multiple languges easy to use and so on.
Refer this
Re: WHAT IS THE DIFFERENCE BETWEEN BIW & DATAWAREHOUSING
hope it helps...

Error while creating data warehouse tables.

Hi,
I am getting an error while creating data warehouse tables.
I am using OBIA 7.9.5.
The contents of the generate_clt log are as below.
>>>>>>>>>>>>>>>>>>>>>>>>>>
Schema will be created from the following containers:
Oracle 11.5.10
Universal
Conflict(s) between containers:
Table Name : W_BOM_ITEM_FS
Column Name: INTEGRATION_ID.
The column properties that are different :[keyTypeCode]
Success!
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
There are two rows in the DAC repository schema for the column and the table.
The w_etl_table_col.KEY_TYPE_CD value for DW application is UNKNOWN and for the ORA_11i application it is NULL.
Could this be the cause of the issue? If yes, why could the values be different and how to resolve this?
If not, then what could be the problem?
Any responses will be appreciated.
Thanks and regards,
Manoj.

Strange. The OBIA 7.9.5 Installation and Configuration Guide says the following:
4.3.4.3 Create ODBC Database Connections
Note: You must use the Oracle Merant ODBC driver to create the ODBC connections. The Oracle Merant ODBC driver is installed by the Oracle Business Intelligence Applications installer. Therefore, you will need to create the ODBC connections after you have run the Oracle Business Intelligence Applications installer and have installed the DAC Client.
Several other users are getting the same message creating DW tables.

Connect to MS SQL Server 2000 data warehouse

Hi,
I use a MS SQL Server 2000 database for my web application where I use JSP. I suppose to create data warehouse using MS SQL Server's Data Transformation Service. But I don't know it's possible to connect to a MS SQL Server's data warehouse using JSP. So I want to know is it possible to connect to data warehouse using JSP and if it is how to do it? Thank you.

You can certainly connect to M$ SQL Server using the JDBC driver:
http://www.microsoft.com/downloads/details.aspx?FamilyID=4f8f2f01-1ed7-4c4d-8f7b-3d47969e66ae&displaylang=en
Connecting to a data warehouse is no different from any relational database. (My understanding is that a data warehouse usually means a star schema implemented in a relational database.) This will connect you.
If you're not familiar with JDBC, you might need the tutorial:
http://java.sun.com/docs/books/tutorial/jdbc/

Unread      Implementing heirarichal structure in data warehouse

I want to create a data warehouse for credit card application. Each user can have a credit card and multiple supplementary credit cards. Each credit card has a main limit, which can be sub-divided into sub-limits to supplementary credit cards as requested by the user. Let us consider the following example:
User “A” has a credit card “CC” with Limit “L” and its limit is $100,000.
User “A” requested for a supplementary credit card “CC1” which is assigned limit
“L1” = $50,000. He requests for another supplementary credit card “CC2” which is assigned limit “L2” = $100,000.
Source tables contain data like this:
1. src_client_card_trans: contains transaction data of client/user credit card usage (client_id, credit_card_number, balance_acquired)
Client_id     Credit_card_number     Balance_acquired
A     CC1     $20,000
A     CC2     $50,000
A     CC     $70,000
2. src_card_limits: contains client’s credit cards linked to credit limits.
Credit_card_number     Limit_id
CC1     L1
CC2     L2
CC     L
3. src_limit_structure: contains the relationship of limits and sub-limits.
Limit_id     Sub_Limit_id
L     L1
L     L2
I have designed two dimensions and one fact table. Dimensions are:
1. LIMITS: contains the limit_id.
2. CLIENTS: contains credit card user’s information.
And fact table is LIMIT_BALANCES_FACT, which have some fact columns with the above dimensions.
How can I implement the above scenario of limit hierarchy in data warehouse? Need your suggestions.
Thanks in advance

Much depends on how you want to analyze the data and there are a few options:
1) Use credit limit as an attribute of the customer dimension. This would allow you to create query filters that can just show those customers with a $100,000 credit limit. This would return a list of credit cards (since the attribute would be assigned to each credit card) and then you can simply add or just keep the parents of that result set.
However, this assumes you do not want to measure data specifically relating to credit card limit. For example it would not be possible to view a total amount spent by all customers who had a credit-limit of $100,000.
In this case the attribute, credit limit, is simply used to filter a result set
2) Create a separate dimension called Credit Limit and create three levels:
All
Range
Credit Limit
The level Range would contain groupings of credit limits such as 100-500, 501-1200, 1201-1,000 etc etc.
This would allow you to analyse your data by customer and by credit limit over time. Allowing you to slice and dice quickly and easily.
3) A second customer hierarchy could be added to the customer dimension. This would allow you to drill-down through different credit limits to customers to individual credit cards. It would be advisable to follow the same approach as option 2 and create some groupings for the credit limits to make the drill down easier for your business users to navigate:
All
Range
Credit Limit
Customer
Credit Card
Hope this helps
Keith Laker
Oracle EMEA Consulting
BI Blog: http://oraclebi.blogspot.com/
DM Blog: http://oracledmt.blogspot.com/
BI on Oracle: http://www.oracle.com/bi/
BI on OTN: http://www.oracle.com/technology/products/bi/
BI Samples: http://www.oracle.com/technology/products/bi/samples/

Implementing heirarichal structure in data warehouse

I want to create a data warehouse for credit card application. Each user can have a credit card and multiple supplementary credit cards. Each credit card has a main limit, which can be sub-divided into sub-limits to supplementary credit cards as requested by the user. Let us consider the following example:
User “A” has a credit card “CC” with Limit “L” and its limit is $100,000.
User “A” requested for a supplementary credit card “CC1” which is assigned limit
“L1” = $50,000. He requests for another supplementary credit card “CC2” which is assigned limit “L2” = $100,000.
Source tables contain data like this:
1. src_client_card_trans: contains transaction data of client/user credit card usage (client_id, credit_card_number, balance_acquired)
Client_id     Credit_card_number     Balance_acquired
A     CC1     $20,000
A     CC2     $50,000
A     CC     $70,000
2. src_card_limits: contains client’s credit cards linked to credit limits.
Credit_card_number     Limit_id
CC1     L1
CC2     L2
CC     L
3. src_limit_structure: contains the relationship of limits and sub-limits.
Limit_id     Sub_Limit_id
L     L1
L     L2
I have designed two dimensions and one fact table. Dimensions are:
1. LIMITS: contains the limit_id.
2. CLIENTS: contains credit card user’s information.
And fact table is LIMIT_BALANCES_FACT, which have some fact columns with the above dimensions.
How can I implement the above scenario of limit hierarchy in data warehouse? Need your suggestions.
Thanks in advance

Much depends on how you want to analyze the data and there are a few options:
1) Use credit limit as an attribute of the customer dimension. This would allow you to create query filters that can just show those customers with a $100,000 credit limit. This would return a list of credit cards (since the attribute would be assigned to each credit card) and then you can simply add or just keep the parents of that result set.
However, this assumes you do not want to measure data specifically relating to credit card limit. For example it would not be possible to view a total amount spent by all customers who had a credit-limit of $100,000.
In this case the attribute, credit limit, is simply used to filter a result set
2) Create a separate dimension called Credit Limit and create three levels:
All
Range
Credit Limit
The level Range would contain groupings of credit limits such as 100-500, 501-1200, 1201-1,000 etc etc.
This would allow you to analyse your data by customer and by credit limit over time. Allowing you to slice and dice quickly and easily.
3) A second customer hierarchy could be added to the customer dimension. This would allow you to drill-down through different credit limits to customers to individual credit cards. It would be advisable to follow the same approach as option 2 and create some groupings for the credit limits to make the drill down easier for your business users to navigate:
All
Range
Credit Limit
Customer
Credit Card
Hope this helps
Keith Laker
Oracle EMEA Consulting
BI Blog: http://oraclebi.blogspot.com/
DM Blog: http://oracledmt.blogspot.com/
BI on Oracle: http://www.oracle.com/bi/
BI on OTN: http://www.oracle.com/technology/products/bi/
BI Samples: http://www.oracle.com/technology/products/bi/samples/

Are there any timesten installation for data warehouse environment?

Hi,
I wonder if there is a way to install timesten as an in memory database for data warehouse environment?
The DW today consist of a large Oralcle database and I wonder if and how a timesten implementation can be done.
what kind of application changes involve with such an implementation and so on?
I know that the answer is probably complex but if anyone knows about such an implementation and some information about it , it would be great to learn from that experience.
Thanks,
Adi

Adi,
It depends on what you want to do with the data in the TimesTen database. If you know the "hot" dataset that you want to cache in TimesTen, you can use Cache Connect to Oracle to cache a subset of your Oracle tables into TimesTen. The key is to figure out what queries you want to run and see if the queries are supported in TimesTen.
Assuming you know the dataset you need to cache and you have control of your application code to change the connection to TimesTen (using ODBC or JDBC), you can give it a try. If you are using a third party tool, you need to see if the tool supports JDBC or ODBC access to the database and change the tool to point to your TimesTen database instead of the Oracle database.
If you are using the TimesTen Cache Connect to Oracle product option, data synchronization between Oracle and TimesTen is handled automatically by the product.
Without further details of what you'd like to do, it's difficult to provide more detailed recommendation.
-scheung

Why do we need SSIS and star schema of Data Warehouse?

If SSAS in MOLAP mode stores data, what is the application of SSIS and why do we need a Data Warehouse and the ETL process of SSIS?
I have a SQL Server OLTP database. I am using SSIS to transfer my SQL Server data from OLTP database to a Data Warehouse database that contains fact and dimension tables.
After that I want to create cubes using SSAS form Data Warehouse data.
I know that MOLAP stores data. Do I need any Data warehouse with Fact and Dimension tables?
Is not it better to avoid creating Data warehouse and create cubes directly from OLTP database?

Another thing to note is data stored in transactional system may not always be in end user consumable format for ex. we may use bit fields/flags to represent some details in OLTP as storage required ius minimum but presenting them as is would not make any
sense to user as they would not know what each bit value represents. In such cases we apply some transformations and convert data into useful information for users to understand. This is also in the warehouse so that information in warehouse can directly be
used for reporting. Also in many cases the report will merge data from multiple source systems so merging it on the fly in report would be tedious and would have hit on report server. In comparison bringing them onto common layer (warehouse) and prebuilding
aggregates would be benefitial for the report performance.
I think (not sure) we join tables in SSAS queries and calculate aggregations in it.
I think SSAS stores these values and joined tables and we do not need to evaluates those values again and this behavior is like a Data Warehouse.
Is not it?
So if I do not need historical data, Can I avoid creating Data Warehouse?
On the backend SSAS uses queries only to extract the data
B/w I was not explaining on SSAS. I was explaining on what happens inside datawarehouse which is a relational database by itself. SSAS is used to built cube (OLAP structures) on top of datawarehouse. star schema is easier for defining relationships
and buidling aggregations inside SSAS as its simple and requires minimal lookups to be performed. Also data would be held at lowest granularity level which can easily be aggregated to required levels inside OLAP cubes. Cube processing is very resource
intensive and using OLTP system would really have a huge impact on processing performance as its nnot denormalized and also doing tranformation etc on the fly adds up to complexity. Precreating a layer (data warehouse) having data in required format would
make cube processing easier and simpler as it has to just cross join tables and aggregate data based on relationships defined and level needed inside the cube.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Permanent Job Opportunity - Oracle BI Data Warehouse Developer Chicago, IL

Submit Resumes to [email protected]
The Business Intelligence Specialist will play a critical role in designing, developing, deploying, and supporting data warehouse/data mart applications. In this role, the person will be responsible for all BI aspects of a data warehouse/data mart application. Primary duties will be to create reporting standards, as well as coach and support power users with selected Oracle tool. The ideal candidate will have 3+ years demonstrated experience in data warehousing and Business Intelligence tools. Must also possess excellent communication skills and an outstanding track record with the user.
Principal Duties:
Participates with internal clients to define software requirements for development, maintenance and/or improvements
Maintains accuracy, integrity, and availability of the data warehouse
Tests, monitors, manages, and validates data warehouse activity, including data extraction, transformation, movement, loading, cleansing, and updating processes
Designs and optimizes data mart models for Oracle Business Intelligence Suite.
Translates the reporting requirements into data analysis and reporting solutions.
Reviews and sign off on project plan(s).
Reviews and sign off on technical design(s).
Defines and develops BI reports for accessing/analyzing data in warehouse.
Customizes BI tools and data sets for different types of users.
Designs and develop UAT (User Acceptance Testing).
Drives improvement of BI system architecture and development process.
Develops and maintains internal relationships. Actively champions teamwork. Uses internal resources to enhance knowledge and expertise of industry, research, products and services. Provides information and support to others in the company.
Required Skills:
Education and Experience:
BS/MS in Computer Science or equivalent.
3+ years of experience with Oracle, PL/SQL Development and Data Warehousing.
Experience Oracle Business Intelligence Suite and Crystal Reports is a plus.
2-3 years dimensional modeling experience.
Demonstrated hands on experience with Unix/Linux, SQL required.
Demonstrated hands on experience with Oracle reporting tools.
Demonstrated experience with translating business requirements into data analysis and reporting solutions.
Experience in training programs/teach users to use tools.
Expertise with software development process.
Effective mediator - able to facilitate constructive and productive discussions with internal customers, external clients, and development personnel pertaining to feature definition, project scope, and status
Problem solving*identifies and resolves problems in a timely manner, gathers and analyzes information skillfully and maintains confidentiality.
Planning/organizing*prioritizes and plans work activities and uses time efficiently. Work requires continual attention to detail in composing and proofing materials, establishing priorities and meeting deadlines. Must be able to work in a fast-paced environment with demonstrated ability to juggle multiple competing tasks and demands.
Quality control*demonstrates accuracy and thoroughness and monitors own work to ensure quality.
Adaptability*adapts to changes in the work environment, manages competing demands and is able to deal with frequent change, delays or unexpected events.
Benefits/Compensation:
Employees enjoy competitive compensation. We have a full benefits package including medical and dental insurance, long-term disability and life insurance and a 401(k) plan.
The client operates within the healthcare industry.
This is a permanent full-time position. After ensuring your availability and qualifications we will put you in direct contact with the client to move forward in the process.

FORWARD THE UPDATED RESUME AS SOON AS POSSIBLE.

Merant Driver is not available for Windows 7 to create data warehouse table

Hi All,
I had installed Database, OBIA, DAC and Informatica in windows 7 64 bit. Installation is successful.
To create data warehouse tables through DAC, we need merant driver. But Merant driver is not available in windows 64bit.
I followed the below procedue to create a merant driver.
"C:\Windows\SysWOW64\odbcad32.exe"
Executed odbcad32.exe, and I found the merant driver in the list, when i try to create DSN, I am getting the following error.
Specified Driver could not be loaded due to system error 193: (Oracle Merant Driver in DAC 10g_Oracle-OH1635326477, c:\orahome\10gR3_1\bifoundation\dac\utilities\DataDirectODBC\seor820.dll).
Can any one please provide me the solution.
Regards,
Kumar

I know the docs say you must use the merant driver however did you try a normal oracle odbc connnection. I am sure i did this in the past.
BTW I hope this is just your laptop and you are playing around. I do'nt think Windows 7 is a supported environment. You should consider a supported platform for production and maybe a virtual machine on your windows 7 machine in development so you can practice the install.
See
Oracle Fusion Middleware Supported System Configurations
http://www.oracle.com/technetwork/middleware/ias/downloads/fusion-certification-100350.html
System Requirements and Supported Platforms for Oracle Business Intelligence Applications 7.9.6.3 http://www.oracle.com/technetwork/middleware/bi/obia7963cert-matrix-395521.xls

Data Warehouse issue after deploaying Server 2012 MP in SCOM 2007R2

Hi all,
I have recently installed Server 2012 MP in SCOM 2007R2. But now I get DataWarehouse warnings that the specified schema name "sdk" either does not exist or I do not have permission to use it.
It's event 31565 with source Operations Manager Health Service Modules
Can anyone help me?
Failed to perform Data Warehouse component deployment operation:Install; Component: Script, Id: '79c157e5-9aa8-7045-d0b8-19e2d713d6c7',Management Pack Version-dependent Id: '01f04523-ee6c-1f3e-08a1-a108dda43507';Target: Database, Server name: 'SQL\SCOM,10100', Database name: 'OperationsManagerDW'. Batch ordinal: 0; Exception: The specified schema name "sdk" either does not exist or you do not have permission to use it.
Best regards,
Oliver
Gruss Oliver Scholl

Hi Gruss,
Was there any resent permission change on the SQL server or the Data warehouse ?
Has your Data reader and writer account Sys admin permissions on the SQL application ?
Also is the reporting tab working and are the reports related to this and other management packs working ?
Gautam.75801

ACI Setup - How to Configure Data Warehouse Database - Partitoning

After reading the ACI Install Guide & Data Warehouse documentation, I have some questions regarding how to setup the database:
- Should database partitioning be setup? If so, what tables should be partitioned and what should they be partitioned by?
- Are there any other best practices or tips for setting up & tuning the database?
We are trying to avoid the (painful) situation of having to add partitioning later on; it is much easier to add it up front (if done correctly up front).
Thanks in advance for any advice!

On the tables recommended for partitioning, the partition key is nullable. If ATG inserts a null value into the timestamp column of one of the partitioned tables, we'll receive an ORA-14300 or ORA-14440 error. Oracle isn't able to figure out what partition to map that record to.
Can the columns be changed to NOT NULL? Or, can the application guarantee a nullable value won't be inserted?
Here are some example columns:
ARF_SITE_VISIT.START_VISIT_TIMESTAMP --> TIMESTAMP(6) null
ARF_REGISTRATION.REGISTRATION_TIMESTAMP --> TIMESTAMP(6) null
ARF_LINE_ITEM.SUBMIT_TIMESTAMP --> TIMESTAMP(6) null
ARF_PROMOTION_USAGE.USAGE_TIMESTAMP --> TIMESTAMP(6) null
ARF_RETURN_ITEM.SUBMIT_TIMESTAMP --> TIMESTAMP(6) null
Thanks

Sort_area_size in data warehouse applications

Similar Messages

Maybe you are looking for