Aggregation, archiving and compression

Hello,
How important is it to begin the aggregation, archiving and compression setup before go-live of a BW-environment?
Is it possible to do this after the cubes have been filled?
We are going live after the weekend and haven't given this much consideration... The plan is to start thinking about this now, just after the go-live...
Need I worry?
Best regards,
Fredrik

hi Fredrik,
aggregate, compression and archiving are done to improve performance.
yes, we can do compression and rollup(after aggregate created) after cubes have been filled, and normally we will include this process in process chain, there are process type for this. compression and rollup are done after daily data filled in infocube.
normally you can set cube for compression.
aggregate is created based on query run examination.
check the aggregate on query performance doc.
Business Intelligence Performance Tuning [original link is broken]
archiving, after go live we won't need archiving so soon,
archiving is done to historical data that not used.
you can start include the compression in process chain,
and evaluate you most used queries and see if aggregate needed.
hope this helps.

Similar Messages

A question on different options for data archival and compression

Hi Experts,
I have production database of about 5 terabyte size in production and about 50 GB in development/QA. I am on Oracle 11.2.0.3 on Linux. We have RMAN backups configured. I have a question about data archival stretegy. To keep the OLTP database size optimum, what options can be suggested for data archival?
1) Should each table have a archival stretegy?
2) What is the best way to archive data - should it be sent to a seperate archival database?
In our environment, only for about 15 tables we have the archival stretegy defined. For these tables, we copy their data every night to a seperate schema meant to store this archived data and then transfer it eventualy to a diffent archival database. For all other tables, there is no data archival stretegy.
What are the different options and best practices about it that can be reviewed to put a practical data archival strategy in place? I will be most thankful for your inputs.
Also what are the different data compression stregegies? For example we have about 25 tables that are read-only. Should they be compressed using the default oracle 9i basic compression? (alter table compress).
Thanks,
OrauserN

You are using 11g and you can compress read only as well as read-write tables in 11g. Both read only and read write tables are the candidate for compression always. This will save space and could increase the performance but always test it before. This was not an option in 10g. Read the following docs.
http://www.oracle.com/technetwork/database/storage/advanced-compression-whitepaper-130502.pdf
http://www.oracle.com/technetwork/database/options/compression/faq-092157.html

Converting and compressing mp4 video for iBook Author

I'm working on an iBook on Propaganda in WW II and would like to include loads of videos.
The Internet Archive has a great collection:
Example: "Out of the Frying Pan Into the Firing Line (1942)"
http://archive.org/details/OutOfTheFryingPanIntoTheFiringLine
Most are in mp4 format. I have tried bringing the mp4 into QT or iMovie and both seem to do about the same job of converting. Plus they increase file size by about 50%
Primary question
I'm targeting my iBook as a free resource for schools. File size is an issue for school iPads that might be 16 GB. Is there anyway to reduce file size, without losing quality? I saw some discussions about Compressor, but that seems to be designed for Final Cut Pro and I'm not a video expert.
Secondary question:
Is there any way to improve the quality of these old videos. Perhaps there are some settings I'm missing in QT or iMovie??
I'm an instructional design guy - not a video guy. Your suggestions will be greatly appreciated ... especially in laymen's terms.
Thanks

I'm keeping a "Working with IBA" collection at Scoop.it.
http://www.scoop.it/t/publishing-with-ibooks-author
I posted a link to this discussion ... and I received a comment (at Scoop.it) from Frank Lowney that's interesting. I don't have Pages, but I may get it try this out. (cheaper than Compressor!)
Frank comments:
"Strange but apparently true. I downloaded a 320x240 *.mp4 file from the Internet Archive and compressed it with the latest version of Handbrake using the iPhone/iPod touch preset. The result was 2.6 MB smaller that what I had downloaded. As predicted, iBooks Author rejected it suggesting I use QuickTime X Player. I did that using the iPod touch/iPhone 3GS preset but the result was a whopping 21.9 MB larger!
Let's remember that the sole target of what iBooks Author outputs is iBooks on the iPad, nothing else.
Next, I use Pages to create an *.epub file with the Handbrake encoded movie in it. Guess what? It plays just fine in iBooks!
So something is fishy here."

How can I use my Mac G3 (OS 9.2.2) to make Windows disk images and compress them into Windows stuffed archives?

I have a collection of old Windows diskettes, 800 K and 1.4 MB. I want to make disk images (or whatever is the Windows equivalent) and compress them into Windows stuffed archives. I want Windows users to be able to download them, expand the stuffed archives, extract the disk images, and use the softsare on old Windows computers. I need to do the work on a Power Macintosh beige G3 Tower using OS 9.2.2. What software can I get to do it?

To Jan, Greetings
Thank you for your message and for your suggestions.
Is WinZip a Mac application that makes compressed files that can be opened on a Windows computer? If so, then do you have any notion where I could find a version old enough to run on OS 9.2.2?
I noticed that DropStuff 6.0, which I use on my G3 Tower, has an option to make compressed files that are self-extracting on Windows. Do you know if that works? I suppose that I could make such a file and then find somebody with a Windows computer to test if for me.
Yes, I could (shudder) get an old Windows computer somewhere, learn to use it, and do the project that way. Do you know if the old versions of Windows include disk image programs and file compression programs? Or, would I need to buy some old software?
Thank you for the information about the size of the diskettes. As you've probably noticed, I don't know very much about Windows things.
Sincerely,
Frontiersman

Aggregation,Roll up and Compression

please answer me the following question
what are the functions of aggregation,roll up aggregation and compression in an infocube....what is the relevance of zero factcount?

Hi Manoj kumar,
Aggregates.
Aggregates are for performance purposes. when u cretae an aggregate based on certain charecteristics u have some indices generated based on this set of charecteristics,
when u query based on this these set of charecteristics, the data retrival will be fast. mostly aggregates are created for set of charecteristics that would be often used for querying.
u can find out which are most used charecteristics from the transaction RSRD. this would suggest you on which charecters u can create aggregates.
if u unnecessarily create aggregates on less frequently used aggregates its gonna consume disk and unnecessary time.
Rollup.
Rollup is for filling the aggregates. its basically to recreate the indices for the aggregates after each time u load data to an infoobject.
rollup should be done after each time u load a target which has aggregates.
Compression
Compression is a process in which the data which is arranged on the basis of request number gets arranged on tha basis of keys.
its mainly to save disk space.
when u compress the requests , the records will be unlinked from their request numbers and and u can never delete any data based on request number.
Hope this helps you.
Regards,
Ranjith

RE: Cleaning and Compression Environment Repository

Mark,
I'm not sure what I did wrong the first time. I thought I
covered all the bases, but just to be certain, I did the
whole sequence again and now it works.
Before, I did have problems restarting the environment,
because the repository was locked. I had to kill a
hanging process to solve the problem. Most likely
it was this process that was interfering with my original
attempts.
With this script, you offered some very usefull infor-
mation. Where did you get it? Like I said in my
first posting, I couldn't find any documentation.
The "help" function of "envedit" doesn't show any of
the commands you used. No way, you stumbled
on this by accident.
Thanks,
Pascal Rottier
STP - MSS Support & Coordination Group
Philip Morris Europe
e-mail: [email protected]
Phone: +49 (0)89-72472530
+++++++++++++++++++++++++++++++++++
Origin IT-services
Desktop Business Solutions Rotterdam
e-mail: [email protected]
Phone: +31 (0)10-2428100
+++++++++++++++++++++++++++++++++++
/* Ever stop to think, and forget to start again? */
-----Original Message-----
From: [email protected]
[SMTP:[email protected]]
Sent: Wednesday, June 30, 1999 1:04 PM
To: [email protected]
Cc: [email protected]
Subject: Re: Cleaning and Compression Environment Repository
The script mentioned is the exact sequence of commands that should be
used.
Did you shutdown the environment first? You cannot clean an environment
repository while the environment is online. I know that these commands
work for both NT and Unix environment managers.
To unsubscribe, email '[email protected]' with
'unsubscribe forte-users' as the body of the message.
Searchable thread archive <URL:http://pinehurst.sageit.com/listarchive/>-
To unsubscribe, email '[email protected]' with
'unsubscribe forte-users' as the body of the message.
Searchable thread archive <URL:http://pinehurst.sageit.com/listarchive/>

Mark,
I'm not sure what I did wrong the first time. I thought I
covered all the bases, but just to be certain, I did the
whole sequence again and now it works.
Before, I did have problems restarting the environment,
because the repository was locked. I had to kill a
hanging process to solve the problem. Most likely
it was this process that was interfering with my original
attempts.
With this script, you offered some very usefull infor-
mation. Where did you get it? Like I said in my
first posting, I couldn't find any documentation.
The "help" function of "envedit" doesn't show any of
the commands you used. No way, you stumbled
on this by accident.
Thanks,
Pascal Rottier
STP - MSS Support & Coordination Group
Philip Morris Europe
e-mail: [email protected]
Phone: +49 (0)89-72472530
+++++++++++++++++++++++++++++++++++
Origin IT-services
Desktop Business Solutions Rotterdam
e-mail: [email protected]
Phone: +31 (0)10-2428100
+++++++++++++++++++++++++++++++++++
/* Ever stop to think, and forget to start again? */
-----Original Message-----
From: [email protected]
[SMTP:[email protected]]
Sent: Wednesday, June 30, 1999 1:04 PM
To: [email protected]
Cc: [email protected]
Subject: Re: Cleaning and Compression Environment Repository
The script mentioned is the exact sequence of commands that should be
used.
Did you shutdown the environment first? You cannot clean an environment
repository while the environment is online. I know that these commands
work for both NT and Unix environment managers.
To unsubscribe, email '[email protected]' with
'unsubscribe forte-users' as the body of the message.
Searchable thread archive <URL:http://pinehurst.sageit.com/listarchive/>-
To unsubscribe, email '[email protected]' with
'unsubscribe forte-users' as the body of the message.
Searchable thread archive <URL:http://pinehurst.sageit.com/listarchive/>

What is difference betwwen aggregates and compression in bi 7.0.

hi All,
will you please tell me difference between compression and aggregates in bi.

Dear Ganesh,
Aggregates allow you to improve the performance of BI queries when data is read from an InfoCube. The data of a BI InfoCube is saved in relational aggregates in an aggregated form. Relational aggregates are useful if you want to improve the performance of one or more specific BI queries, or specifically improve the performance of reporting on characteristic hierarchies
Compression is phenomenon where all the contents of ur info cube moves from F to E table,,This is called compression and data will be aggregated when this compression happens..
Why do we do compression..??
When you load data into the InfoCube, entire requests can be added at the same time. Each of these requests has its own request ID, which is included in the fact table in the package dimension. This makes it possible to pay particular attention to individual requests. One advantage of the request ID concept is that you can subsequently delete complete requests from the InfoCube.
However, the request ID concept can also cause the same data record (where all characteristics are the same except the request ID) to appear more than once in the fact table. This unnecessarily increases the volume of data and affects system performance when you analyze data, since each time you execute a query, the system has to perform aggregation using the request ID.
You can eliminate these disadvantages by compressing data and bringing data from different requests together into one single request (request ID 0).
For more info on aggregates
http://help.sap.com/search/highlightContent.jsp
and info on Compression
http://help.sap.com/search/highlightContent.jsp
Hope this helps u...
Best Regards,
VVenkat..

HTTP Headers - enabling caching and compression with the portal?

Has anyone configured their web server (IIS or Apache) or use a commercial product to flawlessly cache and compress all content generated by the portal?
Compression and caching is critical for making our portal based applictions work for overseas users. It should be doable, just taking advantage of standard HTTP protocols, but implementing this a complex system like the portal is tricky, we seem to be generating different values in the HTTP Headers for the same types of files (such as CSS).
We are running Apache so can't take advantage of the built in compression capabilities of the .net portal. We are running the java vervion. 6.1 mp1, sql server 2000 (portal, search, collab, publisher, studio, analytics, custom .net and java portlets on remote server).
Basically our strategy is to compress all outgoing static and dynamic text content (html, CSS, javascript), and to cache all static files (CSS, javascript, images) for 6 months to a year depending on file type.
Here are some links on the subjects of caching and compression that I have compiled:
Caching & Compression info and tools
http://www.webreference.com/internet/software/servers/http/compression/
http://www.ibm.com/developerworks/web/library/wa-httpcomp/
http://www.mnot.net/cache_docs/
http://www.codeproject.com/aspnet/HttpCompressionQnD.asp?df=100&forumid=322472&exp=0&select=1722189#xx1722189xx
http://en.wikipedia.org/wiki/Http_compression
http://perl.apache.org/docs/tutorials/client/compression/compression.html
https://secure.xcache.com/Page.aspx?c=60&p=590
http://www.codinghorror.com/blog/archives/000807.html
http://www.howtoforge.com/apache2_mod_deflate
http://www.ircache.net/cgi-bin/cacheability.py
http://betterexplained.com/articles/how-to-optimize-your-site-with-http-caching/
http://betterexplained.com/articles/speed-up-your-javascript-load-time/
http://betterexplained.com/articles/speed-up-your-javascript-load-time/
http://www.rubyrobot.org/article/5-tips-for-faster-loading-web-sites
http://betterexplained.com/articles/how-to-optimize-your-site-with-gzip-compression/
http://www.gidnetwork.com/tools/gzip-test.php
http://www.pipeboost.com/
http://www.schroepl.net/cgi-bin/http_trace.pl
http://leknor.com/code/gziped.php?url=http%3A%2F%2Fwww.google.com
http://www.port80software.com/surveys/top1000compression/
http://www.rexswain.com/httpview.html
http://www.15seconds.com/issue/020314.htm
http://www.devwebpro.com/devwebpro-39-20041117DevelopingYourSiteforPerformanceCompressionandOtherServerSideEnhancements.html
http://www.webpronews.com/topnews/2004/11/17/developing-your-site-for-performance-optimal-cache-control
http://www.sitepoint.com/print/effective-website-acceleration
http://nazish.blog.com/1007523/
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/IETechCol/dnwebgen/IE_Fiddler2.asp?frame=true
http://www.fiddlertool.com/fiddler/version.asp
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
http://www.web-caching.com/cacheability.html
http://www.edginet.org/techie/website/http.html
http://www.cmlenz.net/blog/2005/05/on_http_lastmod.html
http://www.websiteoptimization.com/speed/tweak/cache/
http://www.webperformance.org/caching//caching_for_performance.html
http://betterexplained.com/articles/how-to-debug-web-applications-with-firefox/
Edited by tkoenings at 06/18/2007 6:26 AM

Hi Scott,
Does Weblogic platform 8.1 supports netscape? We have developed a portal which
works perfectly on IE but it dies in netscape. Is netUI tags not supported in
Netscape?
Pls reply
manju
Scott Dunbar <[email protected]> wrote:
From a pure HTML perspective Portal does it's rendering with nested
tables.
Netscape 4.x and below have terrible performance with nested tables.
The
problem is not the Portal server but rather Netscape on the client machine.
If IE and/or a recent version of Netscape/Mozilla is not possible then
there are
really only two options:
1) Faster client hardware - not likely to be an acceptable solution.
2) Minimize the number of portlets and the complexity within the portlets.
Neither of these solutions are a great answer, but the 4.7 series of
Netscape is
getting pretty old. Having said that, we've got customers who want to
continue
to use IE 4 :)
Again, though, this problem is, I'm afraid out of our hands. It is the
client
rendering time that is the issue.
cg wrote:
Does anyone know of any known reasons why the 7.0 (did it also with4.0) portal
pages can take up to almost 30 seconds to load in Netscape 4.7? I knowit is a
very generic question but our customer still uses 4.7 and will notuse the portal
b/c it takes so long to load some of the webapps. What the pages willdo when
loading is that the headers will come up and when it gets to the bodyof the page
it seems to stall and then comes up all of a sudden. For some of thepages it
takes 6 seconds and for others it takes about 24-27 seconds.
We have suggested using IE only but that is not an option with allof the customers
and getting a newer version of Netscape is also out of the question.
Any suggestions would be greatly appreciated.--
scott dunbar
bea systems, inc.
boulder, co, usa

How to archive and Purge in Oracle 10g based on data

Hi,
Our requirement is that based on some rules stored in static tables we want to perform Archiving, purging and compression of data in Oracle 10g.
Can anyone guide me to proper information material for how to go about this?
Thanks!
Avinash.

Hi,
Thanks for the information.
But this traditional way of doing is archiving is for maintenance and backup operations.
We want to have this process online, without taking db offline. In this case will this approach work?
In our case, the rules can be like -
1. For table 'A', if rows exceed 10Million, then start archiving of the data for that table.
2. For table 'B', if data is older than 6 motnhs start archiving of the data for that table.
3. Archiving should be on for 15 minutes only after that should pause, and should resume whenever user wants to resume.
4. Archiving should start on specified days only... ETC...

Archiving only compressed InfoCubes?

Hello Colleges,
We have an InfoCube with 5 Mill Records. If I try to select a daterange for Archiving, I get allways the error the request is not compressed. I could not find in any document any record size when it has to be compressed. Is it a generall restriction, which is never mentioned in a document. Does anybody has experince with archiving and had also this problem?
Thanks for the help
Henning

Hi,
It is recommended to archive only fully compressed InfoCubes as you want to ensure that you do not archive data from requests while they are being loaded.
In addition, if you InfoCube has a non-cumulative key figure, there is a hard check to ensure all requests are compressed first before an archive job can execute.
I hope this helps,
Mike

Archive Log Compression

Hi
There is a feature that allows us to compress archive logs. This is set using "ALTER DATABASE ARCHIVELOG COMPRESS enable;"
What I need to know is
1) Which versions of oracle supports this.
2) Has anyone experienced any drawbacks using this.
Thanks

This is what I got back from ORACLE.
Generic Note
Hi,
The archive log compression feature was withdrawn from the release before the oracle 10g production release.
It may appear to function as expected, but it is not yet supported and is intentionally not documented.
Bug 3464260 - ALTER DATABASE ARCHIVELOG COMPRESS ENABLE/DISABLE IS MISSING IN 10G DOCS
Status: 92 - Closed, Not a Bug
- An Enhancement Request been created Bug 6780870 GENERATION OF COMPRESSED ARCHIVELOG FILES to ask
if this functionality will be offered in a future release which is yet not been update by development..
In other words, The SQL statement ALTER DATABASE ARCHIVELOG COMPRESS enable is possible, but indeed
currently unsupported and undocumented even in Oracle Database 11g
Archive compression was a planned new feature for 10G, but unfortunately it was withdrawn and it is still
not available in 11g .This feature is expected in future releases.
Please let me know if that answers your questions .
Regards,
One more question.
We are running windows, I know someone mentioned gzip for linux, can we use that for windows compresssion, or will winzip or winrar or any other compression tool work.
Thanks

Difference Between Aggregates and Compression

Hi,
Can you tell me what is the difference between Aggregates and Compression?
I know ,once compressed data is not available for deletion in request wise
and it moves from F table to E table .
Aggregates means ,data will move from cube to aggreagates(Baby cubes).
But my query is ,As both of them aggregates the data.which of them should be used. at what situation?
I hope you understood my Query.
Regards.
Naresh.

Hi,
An aggregate is a materialized, aggregated view of the data in an InfoCube. In an aggregate, the dataset of an InfoCube is saved redundantly and persistently in a consolidated form into the database.
Aggregates make it possible to access InfoCube data quickly in Reporting. Aggregates serve, in a similar way to database indexes, to improve performance.
especially in the following cases we create aggregates:
The execution and navigation of query data leads to delays with a group of queries.
You want to speed up the execution and navigation of a specific query.
You often use attributes in queries.
You want to speed up reporting with characteristic hierarchies by aggregating specific hierarchy levels.
For more info on aggregates go though the link below
https://help.sap.com/saphelp_sem320bw/helpdata/en/c5/40813b680c250fe10000000a114084/frameset.htm
Compression creates a new cube that has consolidated and summed duplicate information.
2. When you compress, BW does a group by on dimensions and a sum on measures... this eliminates redundent
information.
3. Compressed infocubes require less storage space and are faster for retrieval of information.
4. Once a cube is compressed, you cannot alter the information in it. This can be a big problem if there
is an error in some of the data that has been compressed.
For more info go though the below link
http://www.sap-img.com/business/infocube-compression.htm
Regards,
Marasa.

Hello All... Back after a brief absence, things look a little bit different. I'm trying to take a 16 minute mini dv video and compress it for use on the web. I'm interested in any suggestions you may have on settings for the video and audio tracks. I'v

Hello All...
Back after a brief absence, things look a little bit different.
I'm trying to take a 16 minute mini dv video and compress it for use on the web. I'm interested in any suggestions you may have on settings for the video and audio tracks. I've tried using Sorenson 3 (15 frames, key frames set to automatic, 320 x 240) for video and IMA 4:1 (mono) for audio. The resulting video looked great but the file size came in at about 255 Mb.
Thanks!
PowerMac G5 1.8 Dual Mac OS X (10.4.3)
Message was edited by: Dan Foley

Thank you for the replies. Everyone was correct about the jack, interface, and phasing problems. I have been unplugging my motu audio interface and then using headphones at work. I have not changed any detailed audio output settings in logic. When I read that the jack might be a problem I tried switching headphones. This actually helped. I am using dre-beats headphones and they seem to be having issues with the mac/jack-(the phasing/panning problems. I can use these headphones with other devices but not the mac. I have to use ipod ear buds and the phasing seems fixed. Hopefully this information is helpful to someone else.
If anyone knows how to correct this issue please let me know its difficult to know what my final mixes are going to sound like and I have had to keep bouncing everything into i-tunes- sync to ipod and then listen in my car radio.

PI 7.11 AAE- Archiving and deletion

Dear All,
I just want to get confirm on my understanding regarding AAE archiving and deletion.
We have 98% interfaces using only AAE in our landscape.
I had set up XMLDAS archiving for AAE ( After fixing so many issues ) which archives the AAE messages into the local file system. So no issues in that .
Now my requirements are,
1) I would like to keep only 7 days of data in the database and 100 days of data in the file system
2) I do not want to set up any rules for Archiving /Deletion. It means i would like to archive messages older than 7 days in the file system.
The steps are ( My understanding )
1) Set the persistence duration in NWA to 7 days . So only 7 days of data will be stored in table BC_MSG.
2) Set up the archival job in the background processing through RWB without ANY RULES. So all processed messages older than 7 days will be achieved in the file system. And Archive will come with default delete procedure as well hence Archived messages will be deleted from table BC_MSG.
3) Deactivate the delete job . If i have the active delete job in the background processing, it will delete all messages older than 7 days which i don want. The deletion of messages from BC_MSG will be taken care as part of archival job.
Is my understanding correct?
Thanks
Rajesh

Hello, any know what did they do to delete the files from the file system after 100 days?
We have the same problem, when we deleted manualy some messages from SO AIX but they still appear from the RWB, and when we clic over them we get an error: ...... java.lang.Exception: XML DAS GET command failed with ErrorCode: 598; ErrorText: 598 GET: Error while accessing resource; response from archive store: 598 I/O Error java.io.FileNotFoundException: /usr/sap/PID/javaarch/archive/usr/sap/pid/x
Is there any way to delete messages without generating errors / inconcistencias?

What is main diff b/w imported Archives and External Defination in IR

What is main diff b/w imported Archives and External Defination in IR

Hi Vijay,
ED-standard schema for describing the message structure.If you have the structure provided to you from some external application you can use it in PI directly without recreating in PI.you can import following formats which are used to describe message schema
WSDL (Web Service Description Language),
XSD (XML Schema Definition Language),
DTDs (Document Type Definitions)
Imported Archives: its used basically for java and xslt mapping where you create the required mapping externally using some tools
(java-eclipse/NWDS .XSLT-stylus studio).You can also implement XSLT and Java mappingsf and save them as archives in the Integration Repository.
read More details here
[ED|http://help.sap.com/saphelp_nwesrce/helpdata/en/26/9e97b0f525d743882936c2d6f375c7/content.htm]
[Imported Archives|http://help.sap.com/saphelp_nw04/helpdata/en/4c/b2ad3de2d76b3be10000000a114084/content.htm]
Regards,
Srinivas

Aggregation, archiving and compression

Similar Messages

Maybe you are looking for