Pros/Cons of replicating to files versus staging tables

I am new to GoldenGate and am trying to figure out pros/cons of replicating to flatfiles to be processed by an ETL tool versus replicating directly to staging tables. We are using GoldenGate to source data from multiple transaction systems to flatfiles and then using Informatica to load thousands of flatfiles to our ODS staging. Trying to figure out if it would be better just to push data directly to staging tables. I am not sure which is better in terms of recovery, reconcilliation, etc. Any advice or thoughts on this would be appreciated.

Hi,
My Suggestion would be to push the data from multiple source systems directly to staging table and then populate target system using ELT tool like ODI.
Oracle Data Integrator can be combined with Oracle Golden Gate (OGG) , that provides a cross-platform data replication and changed data capture. Oracle Golden Gate worked in a similar way to Oracle’s asynchronous change data capture but handles greater volumes and works across multiple database platforms.
Source -> Staging -> Target
ODI-EE supports all leading data warehousing platforms, including Oracle Database, Teradata, Netezza, and IBM DB2. This is complemented by the Oracle GoldenGate architecture, which decouples source and target systems, enabling heterogeneity of databases as well as operating systems and hardware platforms. Oracle GoldenGate supports a wide range of database versions for Oracle Database, SQL Server, DB2 z/Series and LUW, Sybase ASE, Enscribe, SQL/MP and SQL/MX, Teradata running on Linux, Solaris, UNIX, Windows, and HP NonStop platforms as well as many data warehousing appliances including Oracle Exadata, Teradata, Netezza, and Greenplum. Companies can quickly and easily involve new or different database sources and target systems to their configurations by simply adding new Capture and Delivery processes.
ODI-EE and Oracle GoldenGate combined enable you to rapidly move transactional data between enterprise systems:
Real-time data. - Immediately capture, transform, and deliver transactional data to other systems with subsecond latency. Improve organizational decision-making through enterprise-wide visibility into accurate, up-to-date information.
Heterogeneous. - Utilize heterogeneous databases, packaged or even custom applications to leverage existing IT infrastructure. Use Knowledge Modules to speed the time of implementation.
Reliability. - Deliver all committed records to the target, even in the event of network outages. Move data without requiring system interruption or batch windows. Ensure data consistency and referential integrity across multiple masters, back-up systems, and reporting databases.
High performance with low impact. - Move thousands of transactions per second with negligible impact on source and target systems. Transform data at high performance and efficiency using E-LT. Access critical information in real time without bogging down production systems.
Please refer to below links for more information on configuration of ODI-OGG.
http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_gg_integration/odi_gg_integration.htm
http://www.biblogs.com/2010/03/22/configuring-odi-10136-to-use-oracle-golden-gate-for-changed-data-capture/
Hope this information helps.
Thanks & Regards
SK

Similar Messages

How to pass parameters to the Control file while loading the data from flat file to staging table

Thanks in advance

Hi ,
LOADDATA statement is required at the beginning of the control file.
INFILE: INFILE keyword is used to specify location of the datafile or datafiles.
INFILE* specifies that the data is found in the control file and not in an external file. INFILE ‘$FILE’, can be used to send the filepath and filename as a parameter when registered as a concurrent program.
INFILE ‘/home/vision/kap/import2.csv’ specifies the filepath and the filename.
Hope this will help you......

License Setup file versus Serial Number Only Hosting

Could someone list the pros and cons of license setup file versus serial number only hosting. I understand that the latter requires the least effort to set up and manage, but what would make me choose the former?
Many thanks

SJW,
I'm think you're referring to the models where the license pool is hosted at Adobe. The two models are called "Serial Number-Only at Adobe" and "Adobe-hosted with custom options."
What's common between the two
- Pool of licenses is hosted at Adobe
- Customer obtains e-license reports on the Licensing Website (LWS)
- E-license is downloaded via the Internet (e-license is around 3 KB)
Differences - Serial No. Only
- Single pool of e-licenses, identified by the serial number, shared by the different groups in the organization that owns that serial number.
- Reports on LWS show the total licenses available/downloaded, overdraft licenses available/downloaded, secondary use licenses, and recovered licenses.
- Upon launching Acrobat, End-users see a dialog box indicating to download an e-license.
- No changes to the way Acrobat 8 is installed (just install the application with the serial number as usual).
Differences - Adobe-hosted with custom options
- Individual pool of e-licenses, identified by the identifier of the each individual Ship To in the order. Not shared across the the organization.
- Same as before, reports show the information about the customer's license pool. However, this option allows companies to have the request to download the e-license to capture the name of the machine so the reports will show the name of every machine that has downloaded (or returned) and e-license.
- Upon launching Acrobat, End-users *may* see a dialog box indicating to download an e-license.
- The installation of Acrobat 8 changes. In this case, Acrobat requires a Client Setup file that configures Acrobat 8 with the custom options the customer wants to setup
- Custom options: display or not the dialog to the end-user (silent e-license download) and include/not include machine name in LWS e-license reports.
In summary, the Serial Number-Only method is simpler but does not allow for customization (which in many cases that's just fine).
Regards,
Juan-Carlos

Help needed in Loading excel data to staging table from OAF Page

Hi All,
We have a requirement from the client on loading of a excel sheet data into staging table using OAF page.
We were able to load a CSV file into staging table via OAF. The approach we used is we created a item of style 'messageFileUpload', which would pick the CSV file from desktop and we wrote the logic on the controller to place the file into server and then sumit a concurrent program to load the data into the staging table.
But client wants data from the excel file to be loaded into staging table. Is there any way(approach) by which we can convert the excel file data into .CSV file using OAF?
Any help or pointers on this will be highly apperciated.
Thanks,
Chethana

Hi,
Read through this :
Need to upload a CSV/Excel to a table in OAF page
Thanks,
Gaurav

What are the pros/cons of having 1 large BI system versus 2 medium systems?

What are the pros/cons of having 1 large BI system versus 2 medium systems? Are there any known limits as to how large a BI system can be until it experiences performance problems?
Thanks!

Hi,
I dont think there is any limit as long as your DB and Apps servers support it.
There are case studies on systems from 20 TB to 60 TB.
Regarding having two meduim installations it may not be a right choice if you consider facts like routine maintenance, DB upgrades and reorganization, number of resources required, number of DEV and QUAL servers required (this will increase as each PROD requires one DEV and one QUAL atleast).
If you have a single PROD you can do away with this.
One DEV , one QUAL and a single team of dedicated resources are required to maintain the system.
Try this Help files.
Large Data Warehouse Implementations
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/c04aa1fe-0fa2-2a10-b78f-be514604d3eb
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/media/uuid/dcd29c7a-0701-0010-ceb3-8ac3918f1ac7
Experiences with SAP NetWeaver® Business Intelligence at 20 Terabytes
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/406a36d7-99b0-2a10-8b89-f91b4a020bd2
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/d56bbf90-0201-0010-53aa-923512127968
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/80d636a9-2dc1-2a10-42a3-ac0b8eac5dc0
Infrastructure Solutions Design, Manage and Optimize a 60 TB SAP NetWeaver Business Intelligence data
warehouse
http://www.redbooks.ibm.com/redpieces/pdfs/sg247289.pdf
http://www.sap.com/platform/esoa/pdf/CS_NetWeaver_Case_Studies.pdf
Hope this helps.
Thanks,
JituK

There are multiple MAC's in my house and I am considering buying Time Capsule to use as a file sharing device. Is anyone doing this? Pros/Cons?

I have multiple MAC's in my house. I would like to create a file server environmet. I am considering using Time Capsule. Is anyone doing this? Pros/Cons?

The Linksys should be about as fast as the Time Capsule. So you might as well connect the Time Capsule to the Linksys, disable sharing and wireless.
Solution two is to use the TC as the router, wireless and network drive. This would save the power the Linksys needs. What is the Linksys connected to? a separate modem?

How can you get the filter off if you can't remember the password? It's"pro con" and it wont even let me check out the rates for aflight.

How can you get the filter off if you can't remember the password? It's"pro con" and it wont even let me check out the rates for aflight.

You can check the file prefs.js in the [http://kb.mozillazine.org/Profile_folder_-_Firefox Firefox Profile Folder] and remove the line(s) related to that extension (procon.password).

RAID 5 Question - separate volume for OS/Apps: pros/cons

What are the pros & cons of having 2 volumes on a Mac Pro RAID 5 (instead of one big volume):
Vol 1: OS & Applications
Vol 2: Home Folders (basically everything else)
1) Is this a good idea/bad idea?
2) For what purposes would this setup be best used for?
3) At what point during the RAID 5 setup do you create multiple volumes and how is it done?
4) How/when do you set up the home directories to be on a separate volume from the OS/Apps volume and does the user have to do anything special during daily use to ensure they aren't saving/modifying anything on the OS/Apps volume?
5) I have four 250GB Barracudas sitting around and was wondering if they will work in a RAID 5 in my Mac Pro. I haven't heard of anyone using HDDs this small in a RAID 5 and Apple's website only talks about 500GB, 750GB, and 1TB in bays 2-4. Does anyone have this setup or know if it will work?
uryc hf bov-jna xrabov, lbh'er bhe bayl ubcr.

What are the pros & cons of having 2 volumes on a Mac Pro RAID 5 (instead of one big volume):
Vol 1: OS & Applications
Vol 2: Home Folders (basically everything else)
1) Is this a good idea/bad idea?
Bad. Using a separate drive (especially 10,000 rpm) for OS & Applications can be useful, but two volumes on the same drive (RAID behaves as one drive) means the heads have to continual move large distances, since home folders and system files are used a lot at the same time.
2) For what purposes would this setup be best used for?
If you often do things that destroy your system volume, you can quickly restore it from a backup copy, without having to worry if your user backup are completely up to date, but a separate system disk would still be better.
3) At what point during the RAID 5 setup do you create multiple volumes and how is it done?
After you create the RAID-5 with RAID Utility, create a volume and set its size to less than the default full size. You can then create more volumes. See page 8 of the RAID Userr Guide
<http://manuals.info.apple.com/en/RAIDUtility_UserGuide.pdf>
4) How/when do you set up the home directories to be on a separate volume from the OS/Apps volume and does the user have to do anything special during daily use to ensure they aren't saving/modifying anything on the OS/Apps volume?
For Tiger, use NetInfo Manager (Utilities folder). For Leopard, Right (or Control) click the user in "Accounts" system preference. Once set, the user doesn't have to do anything special.
5) I have four 250GB Barracudas sitting around and was wondering if they will work in a RAID 5 in my Mac Pro. I haven't heard of anyone using HDDs this small in a RAID 5 and Apple's website only talks about 500GB, 750GB, and 1TB in bays 2-4. Does anyone have this setup or know if it will work?
They should work. Apple only talks about drive sizes there were selling at the time the particular system was sold. If you have problems, though, Apple may blame the drives, and not provide much help.
uryc hf bov-jna xrabov, lbh'er bhe bayl ubcr.

RMAN backup with archivelog (pros/cons)

When performing a "backup database with archivelogs" where does rman store the info contained in the archive logs. If I do a backup like this do I still need to keep the archive logs out on disk? Does this increase time to do backups? I have some production databases that are close to 500gb. Wondering what the pros/cons are doing a backup this way.
Thanks All!

user10784896 wrote:
When performing a "backup database with archivelogs" where does rman store the info contained in the archive logs. The info stored in the archive logs (what you asked about, but probably not what you meant) is stored in the archive logs. The backups of those logs are stored in a backup set. The information about the logs is stored in the control file.
If I do a backup like this do I still need to keep the archive logs out on disk?
You have backed them up, right? Archivelogs are not actively used by the running database, right? If you don't delete them after they are backed up, let me know who you buy your disk storage solutions from, as I'd like to buy stock in that company.
Does this increase time to do backups? Anything you do "increases time to do" vs. not doing something. (The most efficient SELECT statement is the one not executed). The question is does this have a practical, measurable degradation of performance that is intolerable to my organization? The answer undoubtedly "no".
I have some production databases that are close to 500gb. Wondering what the pros/cons are doing a backup this way.
Pros are you can recover your database.
Cons are, you can't recover your database.
And it looks like we need to point out that the housekeeping of your archivelogs can and should be handled by rman, not outside of it. Look at "backup archivelog not backed up n times" and "delete archivelog backed up n times", and other variations of 'backup archivelog' and 'delete archivelog'.
>
Thanks All!

Pros & Cons of FileVault

I'm wondering what the group thinks about FileVault? I haven't used it. I have two files on my computer that, if it were ever lost or stolen, would put information out there that I would rather keep to myself.
An overall Pro/Con discussion would be great.
Thanks.
Chip

Hi, Chip.
You're wise to ask.
I recommend that one only implement FileVault if you really need that level of security:• You have a laptop and travel with it.
• You share a Mac with others or your Mac is in an office easily accessible by others and you have personal, confidential information on such that must be protected across your Home folder.In lieu of FileVault, you can always protect data selectively using an encrypted disk image.
The key risk with FileVault is that you put "all your eggs" in one encrypted disk image. If a bad sector or block develops on the hard drive where the encrypted Home folder resides, your data is toast. Having a comprehensive Backup and Recovery solution, and using it regularly, is critical for all users, and especially important if you use FileVault. Most backup and recovery utilities require that you backup a FileVault-protected Home folder separately from the rest of your hard drive. If you use FileVault, I recommend you backup your encrypted Home folder daily.
Some additional advice should you elect to use FileVault:• Before implementing FileVault:• Check your disk thoroughly. At a minimum, run Steps 1-3 of the procedure in my Resolving Disk, Permission, and Cache Corruption FAQ.
• You must have sufficient free space on your hard disk in order to enable FileVault. You need free disk space at least a bit larger than your Home folder in order for the encryption process to take place. That process copies your current Home folder into an encrypted, sparse disk image file (,sparseimage) — hence the need for the free space — and then securely erases your original, unencrypted Home folder when the copy has completed successfully.• Do not lose the Master Password for your Mac. This must be set in order to enable FileVault and is essential to recovering any encrypted Home folders on your Mac should a user forget their account password. The Master password should again be a "good" password: hard to guess or discern from publicly-available information about you.
• To avoid performance problems when using iMovie and either of Final Cut Pro or Express, save project-related files, scratch disks, and other linked media outside your Home folder. See:• “iMovie: Using FileVault Can Affect Performance.”
•“Final Cut Pro and Final Cut Express: About Using FileVault.”
• Repairing your encrypted Home folder requires special instructions.
• To disable FileVault, you again need free disk space greater than or equal to the size of your encrypted Home folder. In disabling FileVault, your Mac first copies the contents of your Home folder from the encrypted Sparse image to a new Home folder — hence why so much free space is required — and then securely erases the encrypted Home folder.Note that some of the information above is from the "Security" chapter of my book, Troubleshooting Mac® OS X, Tiger Edition.
Dr. Smoke
Author: Troubleshooting Mac® OS X
Note: The information provided in the link(s) above is freely available. However, because I own The X Lab™, a commercial Web site to which some of these links point, the Apple Discussions Terms of Use require I include the following disclosure statement with this post:
I may receive some form of compensation, financial or otherwise, from my recommendation or link.

DB Links Pros & Cons

Hello!
Some questions:
1) Can you explain me pros & cons using db links?
2) Are there alternatives to db links?
3) In which cases is opportune using db links or alternatives?

Hi,
Biggest advantage is, you can use remote database as if, it is part of your database.
i.e. there is one existing application and database. Now you are developing a new application which want to use 'some of' the existing data. Just create a DB links and you can get going.
NOTE: Different databases talking over a DB links can be on different operating systems.
You can create a view, which is based 3 tables sitting in 3 databases on 3 different operating systems.
You do not need any file transfer mechanisms etc etc etc
Why are you asking this question? You must be comparng DB links against something.....
Cheers

Currently my internal drive is full due to Aperture. I want to work with Aperture from a external drive verses my internal drive. In addition, I want to back-up my work to another external drive. Is this easily done? Does this make sense? Pros / Cons...

Currently my internal drive is out of space due to my Aperture files. I want to place Aperture on a external drive to work from verses in intnal drive. In addition, I want to back-up all my work to another external for safe keeping. Once I move aperture to the external, I want to delete it off my internal drive. Does this make sense to do? Pros / Cons....

It is common for people to move their Aperture, iPhoto, iTunes etc. files to an external drive when those files become too large to conveniently keep on the internal drive.
The best approach is probably to keep the applications (iPhoto, Aperture etc.) where they are now, but move the files (photos, music, videos etc.) to the external drive.
You should set up a process to regularly backup all the files from both the internal drive as well as the external drive you'll be using. Here are two approaches:
(1) Use Time Machine. You can configure Time Machine so that it backs up both the internal drive as well as the external drive, all to a third separate backup drive. This is done via the Options button in the Time Machine preference pane.
(2) You can use a "cloning" program, such as SuperDuper, Carbon Copy Cloner, to make complete copies of files on your internal drive and that external drive. One way to do this is to use a very large disk for the backups, and to partition it into two backup drives, making one dedicated to the internal, the other dedicated to the external.
Two separate types of backups are recommended. I use both (1) and (2) above, and I disconnect and shut down the clone type backups when not being used. That way a power failure or surge that might destroy the original disk(s) plus the Time Machine backup(s) would still not damage the backups made under (2).
Don't delete your original Aperture files until you have verified that they are working properly from the external drive AND you have made a backup of them.

Hola tengo una mac pro con pantalla de retina de 13 pulgadas y le quise conectar un proyector hacer serie c120 pero dise que no lo puede leer y el proyector lo provee con otras laos y si funciona me podrían ayudar porfavor'

hola tengo una mac pro con pantalla de retina de 13 pulgadas y le quise conectar un proyector hacer serie c120 pero dise que no lo puede leer y el proyector lo provee con otras laos y si funciona me podrían ayudar porfavor?

Yes in an STL file the subs have a comma after each In and Out Timecode (as a separator) and also they don't normally have line spaces between each entry. If you want the subs to be on two lines and "force" what goes on to the second line (rather than SP deciding for you) then where you want the return to be you need to add "bar" symbol (the one above the backslash on a Mac keyboard)
Try it on a couple of lines of your subtitles to see if it works before committing. I usually create my subs as STL files but it should work
Steve

Pros & Cons of Using SAP PI Interfaces for Report Generation

Hi Guru's
I have a Scenario's like
I have to generate a customized report in SAP with the main data's available in SAP ECC and some required data available in the Legacy System.
I want to know the Pros & cons of using SAP PI RFC/Proxy adapter interface to get the data from the legacy system for each time the user execute the report in SAP ECC.
Thanks in Advance

There are couple of "dimensions" to consider in your PI interface design. For exemple when you are running the sizing exercise (Since we are considering adding a net new interface), you will need to capture specific information about new interface.i.e. S/A, adapters, frequency, avg payload size. etc..Note that the last two attributes will be hard to size properly in this case since you can't predict how frequent the end user will run the report which will impact the latency time required to pull the data. Latency will affect the user experience as visible side affect and definitely the SLA for other interfaces running at the same time.
On the other hand, the data you are trying to retrieve from the legacy won't be used for transactional purpose but for the end user to pull KPIs from the system which can affect ECC as well. You may end up doing lot hot fixes for your report (Assuming that the report is a medium complexity code)
There are other factors to consider but let's consider these are the major one.
Cheers,
F

How to store data file name in one of the columns of staging table

My requirement is to load data from .dat file to oracle staging table. I have done following steps:
1. Created control file and stored in bin directory.
2. Created data file and stored in bin directory.
3. Registered a concurrent program with execution method as SQL*Loader.
4. Added the concurrent program to request group.
I am passing the file name as a parameter to concurrent program. When I am running the program, the data is getting loaded to the staging table correctly.
Now I want to store the filename (which is passed as a parameter) in one of the columns of staging table. I tried different ways found through Google, but none of them worked. I am using the below control file:
OPTIONS (SKIP = 1)
LOAD DATA
INFILE '&1'
APPEND INTO TABLE XXCISCO_SO_INTF_STG_TB
FIELDS TERMINATED BY ","
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
COUNTRY_NAME
,COUNTRY_CODE
,ORDER_CATEGORY
,ORDER_NUMBER
,RECORD_ID "XXCISCO_SO_INTF_STG_TB_S.NEXTVAL"
,FILE_NAME CONSTANT "&1"
,REQUEST_ID "fnd_global.conc_request_id"
,LAST_UPDATED_BY "FND_GLOBAL.USER_ID"
,LAST_UPDATE_DATE SYSDATE
,CREATED_BY "FND_GLOBAL.USER_ID"
,CREATION_DATE SYSDATE
,INTERFACE_STATUS CONSTANT "N"
,RECORD_STATUS CONSTANT "N"
I want to store file name in the column FILE_NAME stated above. I tried with and without constant using "$1", "&1", ":$1", ":&1", &1, $1..... but none of them worked. Please suggest me the solution for this.
Thanks,
Abhay

Pl post details of OS, database and EBS versions. There is no easy way to achieve this.
Pl see previous threads on this topic
SQL*Loader to insert data file name during load
Sql Loader with new column
HTH
Srini

Pros/Cons of replicating to files versus staging tables

Similar Messages

Maybe you are looking for