Data archiving strategy

Hi,
Does anybody know of any good book/article about strategies to archive old data from an OLTP type Oracle database? If it has any hands-on examples it's even better. Since that's probably a common requirement, I hope there exists some material about this....
Maybe it's better if I describe the requirements I have and the solutions I thought of. We have an 11gR2 database where the growth of data is approximately 30GB/month. The largest part of it is due to some log activity tables which store XML data in CLOB columns (not XMLTYPE). None of the tables are partitioned. We still have some time till the disks will be full, but I'd like to establish a process to periodically archive old data to tape. For now we don't need data mining functionalities (i.e. a data warehouse) since I'd only like to archive log tables for legal compliance.
Roughly, I thought of something like the following:
1) Since XMLs are only inserted once and never updated again, I thought to convert them to SECUREFILE LOBs and apply HIGH compression on them. Now they are defined as BASICFILE CLOBs.
2) Turn log tables into range partitioned or interval partioned ones (partition by month). This should have been done before the application was put in production, but unfortunately it wasn't.
3) At the end of the month exchange/truncate the partition
4) Somehow export the old exchanged to tape (datapump export; transportable tablespaces if they are moved/created in its own tablespace; maybe using RMAN and READ-ONLY tablespaces?)
Anyway I have some questions/doubts, like:
1) How to archive data from multiple tables if data for a single entity is spread across many tables and some of them are partitioned and some are not. For example if there's a parent-child relationship and the child is partitioned but the parent is not.
2) Which technique is advisable for exporting/archiving data out the database? As I wrote above it could be expdp, transportable TS, RMAN
3) How to deal with table definitions changes in the OLTP database when an old partition has to be plugged again in the database?
So if anyone knows any good paper about questions I wrote about, I'll appreciate it very much.
Thanks in advance and regards,
Jure

Maybe this article can help; it uses XML for structural change flexiblity: http://www.oracle.com/technetwork/oramag/2006/06-jul/o46xml-097640.html

Similar Messages

Put Together A Data Archiving Strategy And Execute It Before Embarking On Sap Upgrade

A significant amount is invested by organizations in a SAP upgrade project. However few really know that data archiving before embarking on SAP upgrade yields significant benefits not only from a cost standpoint but also due to reduction in complexity during an upgrade. This article not only describes why this is a best practice but also details what benefits accrue to organizations as a result of data archiving before SAP upgrade. Avaali is a specialist in the area of Enterprise Information Management. Our consultants come with significant global experience implementing projects for the worlds largest corporations.
Archiving before Upgrade
It is recommended to undertake archiving before upgrading your SAP system in order to reduce the volume of transaction data that is migrated to the new system. This results in shorter upgrade projects and therefore less upgrade effort and costs. More importantly production downtime and the risks associated with the upgrade will be significantly reduced. Storage cost is another important consideration: database size typically increases by 5% to 10% with each new SAP software release – and by as much as 30% if a Unicode conversion is required. Archiving reduces the overall database size, so typically no additional storage costs are incurred when upgrading.
It is also important to ensure that data in the SAP system is cleaned before your embark on an upgrade. Most organizations tend to accumulate messy and unwanted data such as old material codes, technical data and subsequent posting data. Cleaning your data beforehand smoothens the upgrade process, ensure you only have what you need in the new version and helps reduce project duration. Consider archiving or even purging if needed to achieve this. Make full use of the upgrade and enjoy a new, more powerful and leaner system with enhanced functionality that can take your business to the next level.
Archiving also yields Long-term Cost Savings
By implementing SAP Data Archiving before your upgrade project you will also put in place a long term Archiving Strategy and Policy that will help you generate on-going cost savings for your organization. In addition to moving data from the production SAP database to less costly storage devices, archived data is also compressed by a factor of five relative to the space it would take up in the production database. Compression dramatically reduces space consumption on the archive storage media and based on average customer experience, can reduce hardware requirements by as much as 80% or 90%. In addition, backup time, administration time and associated costs are cut in half. Storing data on less costly long-term storage media reduces total cost of ownership while providing users with full, transparent access to archived information.

Maybe this article can help; it uses XML for structural change flexiblity: http://www.oracle.com/technetwork/oramag/2006/06-jul/o46xml-097640.html

A question on different options for data archival and compression

Hi Experts,
I have production database of about 5 terabyte size in production and about 50 GB in development/QA. I am on Oracle 11.2.0.3 on Linux. We have RMAN backups configured. I have a question about data archival stretegy. To keep the OLTP database size optimum, what options can be suggested for data archival?
1) Should each table have a archival stretegy?
2) What is the best way to archive data - should it be sent to a seperate archival database?
In our environment, only for about 15 tables we have the archival stretegy defined. For these tables, we copy their data every night to a seperate schema meant to store this archived data and then transfer it eventualy to a diffent archival database. For all other tables, there is no data archival stretegy.
What are the different options and best practices about it that can be reviewed to put a practical data archival strategy in place? I will be most thankful for your inputs.
Also what are the different data compression stregegies? For example we have about 25 tables that are read-only. Should they be compressed using the default oracle 9i basic compression? (alter table compress).
Thanks,
OrauserN

You are using 11g and you can compress read only as well as read-write tables in 11g. Both read only and read write tables are the candidate for compression always. This will save space and could increase the performance but always test it before. This was not an option in 10g. Read the following docs.
http://www.oracle.com/technetwork/database/storage/advanced-compression-whitepaper-130502.pdf
http://www.oracle.com/technetwork/database/options/compression/faq-092157.html

Best Practise for Data Archiving

Hi All,
I have query in SAP that when business will do the data archving in their R3 systems.
On which basis they will do data archiving?
Is there any minimum period that data need's to be archived?
before doing the data archiving what they will do ?
Regards
Srini

Hi Srini,
1. SAP suggestes to implement data archiving strategy as early as possible to manage database growth .
However pople think of archiving when they realise the problems like large data volumes,slow system resonse time,performance issues etc...
2. There is a proper way to implement Data Archiving . Database has to be anaylzed first for getting the top DB tables and Archiving objects.
3. Based on the DB analysis ,Data archiving plan has to be implenemted according to data management guide.
4. There is a minimum period known as residence time has to be completed before any data to be archived. Once the document is business completed and serverd its minimum required period in the Database ,it can be archived.
5, Before going for data archiving there are many steps to be followed like analysis,configuration etc that you can see in details at the link below :
http://help.sap.com/saphelp_47x200/helpdata/en/2e/9396345788c131e10000009b38f83b/frameset.htm
let me know if this helps you .
-Supriya
Edited by: Supriya Shrivastava on May 4, 2009 10:38 AM

Oracle Apps 11i - Data Archival

Hi,
Has anyone done data archival on Oracle Apps? I would like to know if there are any best practices or any guidelines for the data archival.
Kindly share your experience on data archival on Oracle Apps.
Regards
Sridhar M

Hi;
Please see:
Oracle E-Business Suite Data Archival Strategy
http://documents.club-oracle.com/downloads.php?do=file&id=1862
Can We archive the EBS r12 tables data?
Also see:
http://it.dspmanagedservices.co.uk/blog-1/bid/60253/Managing-data-growth-on-E-Business-Suite-with-an-archiving-strategy
Check this pdf
Regard
Helios

SAP Support for Data Archiving

Hello,
We just wanted to confirm if SAP will support issues if we encounter issues while restoring data archived via SARA. For example, if in the process of restoring archived data a defect is found, is this still supported by SAP?

Hi
Supported and the SAP Data Archiving Support Tools are SARA , DART and SARI
SARA --- SAP Archive Administration
All data archiving activities commence with Archive administration (transaction SARA). SARA provides the overall administration of archiving schedules and manages the archiving sessions. The process includes customization of objects, their conversion to archived sequential archived files and most importantly, their overall management. In addition to these, the archive administration process also retrieves the archived files and converts the data through an automated process, if there is a change in software, hardware or the data structure. The data archiving process is streamlined and simplified through the Central command of archiving administration
DART --- Data Retention Tool
A data retention strategy is required for retaining enterprise information for long periods of time. The Data Retention Tool (DART) provides this functionality. It is capable of extracting data, which is period-specific as well as any supporting information. DART transforms database objects into flat files that can be read by any third-party software designed for flat files. It is available as an add-on for older versions of SAP R/3, but is an integrated feature of the recent versions of SAP R/3.
SARI --- SAP Archive Information System
SARI provides much-needed retrieval capabilities against previously archived data. It requires the archive files be loaded into new tables in the database. In this approach, AS requires detailed, technical knowledge of the database table structure and field layouts. Database structure of an enterprise needs to be flexible and dynamic to be able to quickly adapt to organisational changes. It might happen that the current database structure is not compatible with the structure of database when archive files were initially generated. Archive Information System is a standard tool delivered by SAP that facilitates customized access to the archived data
Regards
Venkat

SAP Content Server Archiving strategy

Hi,
I have installed ECC 6.0 SR3 and SAP Content Server 6.40 recently. Content Server is contains only scanned copies. I want to propose archval strategy for the Content Server.
Any body has document for the same please let me know.
Thanks in advance

hi,
Archiving,
There are both technical and legal reasons for archiving application data. Data Archiving:
· Resolves memory space and performance problems caused by large volumes of transaction data
· Ensures that data growth remains moderate so that the database remains manageable in the long term
· Ensures that companies can meet the legal requirements for data storage in a cost-efficient manner
· Ensures that data can be reused at a later date, for example, in new product development
please refer this thread
http://help.sap.com/saphelp_erp60_sp/helpdata/en/8d/3e4c11462a11d189000000e8323d3a/frameset.htm
Benakaraja
??P
Edited by: benaka rajes on Sep 10, 2009 6:09 PM

Archival Strategy for IMAP?

I've been reading through the forums and I suppose my issue is different enough to warrant a new thread.
I have recently switched to IMAP. I have several accounts that I track, and dozens of subfolders into which I route and filter emails as they come in.
With POP, everything was stored locally, and I didn't have to worry about disk usage. This is no longer the case. So I would like to implement an auto-archival process that copies messages older than 90 days to a mailbox on my local machine.
Here's the catch: I want to preserve my folder structure.
Let's say I have this folder:
Inbox
|_account1
|_subfolder1
|_subfolder2
|_subfolder3
Basically I want to set up an automator action or a set of rules within Mail.app that periodically looks through the folders and moves any emails older than 90 days to it's corresponding folder on my local drive. In other words, mail in subfolder1 would go to a similarly named subfolder1 on my mac.
This is a different technique than simply synchronizing my IMAP directories in that the contents of the archived folders will be different than those of the ones on the server.
So far, I have been unable to find or create an automator action that looks for files older than a non-specified date (a pet peeve that I hope is addressed in Leopard). And even if I could, I have not found a way to have either automator or Mail.app itself apply rules to a specific folder (account, yes, but not a folder).
If anyone has developed an archival strategy that works in these conditions, please let me know.
Many thanks,
Steve

Hi Terry,
what we se usually is that customer run one company-wide SLT box. The monitoring and all operation aspects are easier to handle. For example, you can patch the SLT system independently from the source systems. The general recommendation for the SLT box is to have it close to your HANA -> why? The RFC compression is much better than the DB compression. From a performance perspective I do not see an advantage when you have SLT on the source system. In real life these are only figures, the overall network performance should not be a problem. We have customers with one central SLT box next to Hana and more than 80 ECC systems globally distributed.
Best, Tobias

Data Archiving suggestions

Hi- we have an HFM application with 8 years of data in it. Obviously this increases the size of our database and causes database copies with HFMcopyapp to take longer. We are trying to come up with an archiving strategy to move old data into a mirrored application or ?????. Just looking for some input on anyone's experience with this and how to keep the metadata in sync when changes are made to the current application that impact archived years..
Thanks
We are on 9.2.1
Jeff

assuming the historical data is static, why would you need to apply current metadata changes? Still if this is the case why not just load metadata in replace mode on the archived app? A move up to a later releases will give you more options from the EPM architect.

HR PA Data Archiving

Hi,
We are undergoing the archiving project for HR module. For PD data, we can use object BC_HROBJ, for PCL4 data, we can use PA_LDOC. What about 2000 series infotypes data, such as PA2001, PA2002, PA2003, etc.? Because all changes to these infotypes are stored in PCL4 cluster LA, we only need to purge the data from these tables. What is the best way to purge PA2xxx data? We cannot use transaction PA30/PA61 to delete records because user exit edits prevent any action against old data beyond certain time frame.
Thanks very much,
Li

This is not directly related to SAP NetWeaver MDM. You may find more information about data archiving on SAP Service Marketplace at http://www.service.sap.com/data-archiving or http://www.service.sap.com/slo.
Regards,
Markus

Data conversion strategy for new SOB

Dear Viewers
on 11.5.10
We are creating a new SOB with a change in currency from Feb-11 as this is the requirement
For the same, we need to do data conversion.
I have a confusion for Purchase Orders and Sales Orders
Purchase Orders:
Open purchase orders will be converted, means the unfulfilled PO`s i.e the ones not received and are fully open.
The PO`s which have been recieved but not delivered, Requested the users to clear the intransit receipts.
The PO's which are partially received, what to be done for them?
If a PO is fully received and Delivered will not me converted to the new SOB as its not an open PO
but If invoice comes after Feb-11, than how the matching will be done?
What if a return has to be made moving forward in FEB-11 under new SOB
Sales Orders:
Open sales orders will be converted, that is the ones that have been entered and not yet booked.
Users have been requested to clear off the Sales order lines which are already pick confirmed but not yet shipped, hence they will be shipped and interfaced to AR
For the Sales orders that have been booked, those lines that are not yet processed further will also be converted.
Now what if a receipt comes after feb 11, how to handle this as the sales order wiould not have been converted?
Please give your advise on the data migration strategy for PO`s and SO's.
Please do add any point that may have been missed by me
Appreciate your help
Thanks
Emm

Hi David,
for master data conversion you can use LSMW and the RE-FX BAPIs. (please refer to SAP note [782947|https://service.sap.com/sap/support/notes/782947] ).
Regards, Franz

Business Partner Data Archiving - Help Required

Hi,
Am New to Data Archiving and need help to Archive Business Partner in CRM. I tried to Archive some BP's but it was not archiving. Kindly throw some light on it.
Problem we face are:
1) When we try to write a Business Partner to an Archive File the Job Log Shows finished but no files are created in the System.
2) When we try to delete the BP from Data Base it doesn't show the Archived BP's that could be deleted (I guess this is because there are no archived file).
Archive Object Used is CA_BUPA. We have created a Variant and set the Time as Immediate and Spool to LOCL.
Kindly let us know if there is any step we are missing here and if we are on the wrong track.
All answers are appreciated.
Thanks,
Prashanth
Message was edited by:
        Prashanth KR

Hi,
To archive a BP following steps are to be followed.
A. Mark the BP to be archived in transaction BP >status tab
B. Run dacheck
FYI : Steps on how to perform dacheck :
1. In transaction DACONTROL change the following parameter to the
values : for CA_BUPA .
No. calls/packages   1
Number of Objects   50
Package SizeRFC     20
Number of Days       1 (if you use mobile client it should be more)
2.If a business partner should be archived directly after the
archiving note was set, this can be done by reseting the check
   date with report CRM_BUPA_ARCHIVING_FLAGS.
   here only check (set) the options :
   - Archiving Flag
   - Reset Check Date
Reset all options should be unchecked.
3. go to dacheck and run the process
4. This will change the status of the BPs to 'Archivable'
   *Only those bp's which are not involved in any business transactions,
    install base, Product Partner Range (PPR) will be set to archivable
The BP's with status 'Archivable' will be used by the archiving
     run.
Kindly refer note 646425 before goint to step C ***
C.Then run transaction CRM_BUPA_ARC.
- Make sure that the "selection program" in transaction "DACONTROL"
   is maintained as " CRM_BUPA_DA_SELECTIONS"
Also create a variant, which can done by excecuting
CRM_BUPA_DA_SELECTION and enter the variants name for the field
Variant. This variant is the buisness partner range.
- Also please check note 554460.
Regards, Gerhard

Data archiving for Write Optimized DSO

Hi Gurus,
I am trying to archive data in Write Optimized DSO.
Its allowing me to archive on request basis but it archives entire requests in the DSO(Means all data).
But i want to select to archive from this request to this request.(Selection of request by my own).
Please guide me.
I got the below details from SDN.Kindly check.
Archiving for Write optimized DSOs follow request based archiving as opposed to time slice archiving in standard DSO. This means that partial request activation is not possible; only complete requests have to be archived.
Characteristic for Time Slice can be a time characteristic present in the WDSO, or the request creation data/request loading date. You are not allowed to add additional infoobjects for semantic groups, default is 0REQUEST & 0DATAPAKID.
The actual process of archiving remains the same i.e
Create a Data Archiving Process
Create and schedule archiving requests
Restore archiving requests (optional)
Regards,
kiruthika

Hi,
Please check the below OSS Note :
http://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/com.sap.km.cm.docs/oss_notes/sdn_oss_bw_whm/~form/handler%7b5f4150503d3030323030363832353030303030303031393732265f4556454e543d444953504c4159265f4e4e554d3d31313338303830%7d
-Vikram

SAP Data Archiving in R/3 4.6C and ECC 6.0.

Hi Guys,
Need some suggestions,
We currently working on SAP R/3 4.6 and have plans to upgrade it to ECC 6.0,
In the mean time there is an requirement for SAP Data Archiving for reducing the database size and increase system performance.So wanted to know if its better to do Data Archiving before upgrade or after. Technically which will be comfortable, Also wanted to know if any advanced method available in ECC 6.0 compared to SAP R/3 4.6.
Please provide your valuable suggestion.
Thanks and Regards
Deepu

Hi Deepu,
With respect to archiving process there will be no major difference in 4.6 and ECC 6.0 system. However you may get more advantage in ECC6.0 system because of Archive routing and also upgraded Write, Delete programs (upgraded program will depend on your current program in 4.6 system). For example In 4.6 system for archive MM_EKKO write program RM06EW30, archiving of purchase document is based on company code in the selection criteria and there is no preprocessing functionality. In ECC 6.0 you can archive by purchase organization selection criteria and preprocessing functionality will additional help in archiving of PO.
In case if you archive documents in 4.6 and later upgrade it to ECC 6.0, SAP system will assure you to retrieve the archived data.
With this i can say that going with archiving after upgrade to ECC 6.0 system will be better with respect to archiving process.
-Thanks,
Ajay

Vendor Master Data Archiving

Hi,
I wanted to archive vendor master data. I got program - SAPF058V which gives FI,Vendor Master Data Archiving: Proposal list.
This program checks whether and which Vendor Master data can be archived or not. I am getting error message after running this program with selection of one of the Vendor saying that "Links stored incompletely". Can someone pls help on this
Thanks...Narendra

Hi,
Check if you have an entry in table FRUN for PRGID 'SAPF047'. Other option is set the parameter 'FI link validation off'' with the flag on (ie: value 'x'). Check the help for this flag, surely this vendor have a code of customer. Perhaps you must try to delete this link in customer and vendor master before.
I hope this helps you,
Regards,
Eduardo

Data archiving strategy

Similar Messages

Maybe you are looking for