Best Practices on OWB/ODI when using Asynchronous Distributed HotLog Mode

Hello OWB/ODI:
I want to get some advice on best practices when implementing OWB/ODI mappings to handle Oracle Asynchronous Distributed HotLog CDC (change data capture), specifically for “updates”.
Under Asynchronous Distributed HotLog mode, if a record is changed in a given source table, only the column that has been changed is populated in the CDC table with the old and new value, and all other columns with the exception of the keys are populated with NULL values.
In order to process this update with an OWB or ODI mapping, I need to compare the old value (UO) against the new value (UN) in the CDC table. If both the old and the new value are NOT the same, then this is the updated column. If both the old and the new value are NULL, then this column was not updated.
Before I apply a row-update to my destination table, I need to figure out the current value of those columns that have not been changed, and replace the NULL values with its current value. Otherwise, my row-update will replace with nulls those columns that its value has not been changed. This is where I am looking for an advise on best practices. Here are the possible 2 solutions I can come up with, unless you guys have a better suggestion on how to handle “updates”:
About My Environment: My destination table(s) are part of a dimensional DW database. My only access to the source database is via Asynchronous Distributed HotLog mode. To build the datawarehouse, I will create initial mappings in OWB or ODI that will replicate the source tables into staging tables. Then, I will create another set of mappings to transform and load the data from the staging tables into the dimension tables.
Solution #1: Use the staging tables as lookup tables when working with “updates”:
1.     Create an exact copy of the source tables into a staging environment. This is going to be done with the initial mappings.
2.     Once the initial DW database is built, keep the staging tables.
3.     Create mappings to maintain the staging tables using as source the CDC tables.
4.     The staging tables will always be in sync with the source tables.
5.     In the dimension load mapping, “join” the staging tables, and identify “inserts”, “updates”, and “deletes”.
6.     For “updates”, use the staging tables as lookup tables to get the current value of the column(s) that have not been changed.
7.     Apply the updates in the dimension tables.
Solution #2: Use the dimension tables as lookup tables when working with “updates”:
1.     Delete the content of the staging tables once the initial datawarehouse database has been built.
2.     Use the empty staging tables as a place to process the CDC records
3.     Create mappings to insert CDC records into the staging tables.
4.     The staging tables will only contain CDC records (i.e. new records, updated records, and deleted records)
8.     In the dimension load mapping, “outer join” the staging tables, and identify “inserts”, “updates”, and “deletes”.
5.     For “updates”, use the dimension tables as lookup tables to get the current value of a column(s) that has not been changed.
6.     Apply the updates in the dimension tables.
Solution #1 uses staging tables as lookup tables. It requires extra space to store copies of source tables in a staging environment, and the dimension load mappings may take longer to run because the staging tables may contain many records that may never change.
Solution #2 uses the dimension tables as both the lookup tables as well as the destination tables for the “updates”. Notice that the dimension tables will be updated with the “updates” AFTER they are used as lookup tables.
Any other approach that you guys may suggest? Do you see any other advantage or disadvantage against any of the above solutions?
Any comments will be appreciated.
Thanks.

hi,
can you please tell me how to make the JDBC call. I triedit as:
1. TopicConnectionFactory tc_fact = AQjmsFactory.getTopicConnectionFactory(host, SID, Integer.parseInt(port), "jdbc:oracle:thin");
and
2. TopicConnectionFactory tc_fact = AQjmsFactory.getTopicConnectionFactory(host, SID, Integer.parseInt(port), "thin");
-as given in http://www.acs.ilstu.edu/docs/oracle/server.101/b10785/jm_opers.htm#CIHJHHAD
The 1st one is giving the error:
Caused by: oracle.jms.AQjmsException: JMS-135: Driver jdbc:oracle:thin not supported
at oracle.jms.AQjmsError.throwEx(AQjmsError.java:330)
at oracle.jms.AQjmsTopicConnectionFactory.<init>(AQjmsTopicConnectionFactory.java:96)
at oracle.jms.AQjmsFactory.getTopicConnectionFactory(AQjmsFactory.java:240)
at com.ivy.jms.JMSTopicDequeueHandler.init(JMSTopicDequeueHandler.java:57)
The 2nd one is erroring out:
oracle.jms.AQjmsException: JMS-225: Invalid JDBC driver - OCI driver must be used for this operation
at oracle.jms.AQjmsError.throwEx(AQjmsError.java:288)
at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:1307)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:1028)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:951)
at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:929)
at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:781)
at com.ivy.jms.JMSTopicDequeueHandler.receiveMessages(JMSTopicDequeueHandler.java:115)
at com.ivy.jms.JMSManager.run(JMSManager.java:90)
at java.lang.Thread.run(Thread.java:619)
Is anything else beyond this is required??? please help. :(
oracle: 10g R4
linux environment and java is trying to do AQjmsFactory.getTopicConnectionFactory(...); Java machine is diffarent from the database and no oracle client is to be installed on java machine.
The same code is working fine when i use oc8i instead of thin drivers and run it on db machine.
ravi

Similar Messages

  • Best practice for if/else when one outcome results in exit [Bash]

    I have a bash script with a lot of if/else constructs in the form of
    if <condition>
    then
    <do stuff>
    else
    <do other stuff>
    exit
    fi
    This could also be structured as
    if ! <condition>
    then
    <do other stuff>
    exit
    fi
    <do stuff>
    The first one seems more structured, because it explicitly associates <do stuff> with the condition.  But the second one seems more logical because it avoids explicitly making a choice (then/else) that doesn't really need to be made.
    Is one of the two more in line with "best practice" from pure bash or general programming perspectives?

    I'm not sure if there are 'formal' best practices, but I tend to use the latter form when (and only when) it is some sort of error checking.
    Essentially, this would be when <do stuff> was more of the main purpose of the script, or at least that neighborhood of the script, while <do other stuff> was mostly cleaning up before exiting.
    I suppose more generally, it could relate to the size of the code blocks.  You wouldn't want a long involved <do stuff> section after which a reader would see an "else" and think 'WTF, else what?'.  So, perhaps if there is a substantial disparity in the lengths of the two conditional blocks, put the short one first.
    But I'm just making this all up from my own preferences and intuition.
    When nested this becomes more obvious, and/or a bigger issue.  Consider two scripts:
    if [[ test1 ]]
    then
    if [[ test2 ]]
    then
    echo "All tests passed, continuing..."
    else
    echo "failed test 2"
    exit
    fi
    else
    echo "failed test 1"
    fi
    if [[ ! test1 ]]
    then
    echo "failed test 1"
    exit
    fi
    if [[ ! test2 ]]
    then
    echo "failed test 2"
    exit
    fi
    echo "passed all tests, continuing..."
    This just gets far worse with deeper levels of nesting.  The second seems much cleaner.  In reality though I'd go even further to
    [[ ! test1 ]] && echo "failed test 1" && exit
    [[ ! test2 ]] && echo "failed test 2" && exit
    echo "passed all tests, continuing..."
    edit: added test1/test2 examples.
    Last edited by Trilby (2012-06-19 02:27:48)

  • Is Adobe Connect part of Adobe Creative Cloud? Are there any best practices ideas from people who use Connect and Creative Cloud?

    Is Adobe Connect part of Adobe Creative Cloud? Are there any best practices ideas from people who use Connect and Creative Cloud?
    I have an Adobe Connect account and I'm are also in the early stages of developing a webinar. I am looking for any tips and advice from anyone who uses both of these services.

    As the £27, was an introductory offer. Upon the completion of one year, the price will change to the normal creative cloud cost which is at £46.88. However if you have the previous versions of the creative suites like CS 3, 4, 5, 5.5 or the CS 6. You can avail the offer at £27.34 per month incl. VAT. However this Requires annual commitment; billed monthly.

  • What is the best way to avoid latency when using the io plug in?

    What is the best way to avoid latency when using the io plug in?

    Hi colin a.
    Welcome to the Support Communities!
    The article below may be able to help you with this.  Click on the link to see more details and screenshots. 
    Logic: About I/O buffer size and monitoring latency
    http://support.apple.com/kb/ht1314
    Cheers,
    - Judy

  • Unable to open https sites. can open when using firefox in safe mode.

    Mozilla Firefox has been an exceptional browser, however in the last update I have experienced problems when trying to open https sites. They will not open and then time out. I do not have this problem when using Internet explorer. I have contacted customer service at the problem sites, and they have said other customers have also experienced this problem. I have gone into options and have changed various settings to see if this would solve the problem. It didn't, your response would be appreciated.. Please note if I use firefox in safe mode I can access these sites? My E-mail as follows: [email protected]

    Did you perhaps install an extension to open sites with https
    # Type '''about:config''' into the location bar and press "Enter"
    # Use Find ("Ctrl+F") to look for '''https'''
    One such extension is "HTTPS-Everywhere"
    As you probably know Firefox Safe Mode disables extensions and some preferences.
    http://kb.mozillazine.org/Standard_diagnostic_-_Firefox#Extension_issues
    You posted a question here: you should check for all of your answers at...
    :Unable to open https sites. can open when using firefox in safe mode. | Firefox Support Forum | Firefox Help
    :https://support.mozilla.com/en-US/questions/872957
    You are expected to pick up your answers where you post them, in the upper right corner is a "get email updates" that you can click on. For more information, see my avatar icon to the left of my reply where you posted.

  • Best practice for Video over IP using ISDN WAN

    I am looking for the best practice to ensure that the WAN has suffient active ISDN channels to support the video conference connection.
    Reliance on load threshold either -
    Takes to long for the ISDN calls to establish causing the problems for video setup
    - or is too fast to place additional ISDN calls when only data is using the line
    What I need is for the ISDN calls to be pre-established just prior to the video call. Have done this in the past with the "ppp multilink links minimum commmand but this manual intervention isn't the preferred option in this case
    thanks

    This method is as secure as the password: an attacker can see
    the hashed value, and you must assume that they know what has been
    hashed, with what algorithm. Therefore, the challenge in attacking
    this system is simply to hash lots of passwords until you get one
    that gives the same value. Rainbow tables may make this easier than
    you assume.
    Why not use SSL to send the login request? That encrypts the
    entire conversation, making snooping pointless.
    You should still MD5 the password so you don't have to store
    it unencrypted on the server, but that's a side issue.

  • Best Practice For Referencing JPEG Path Using Servlets To Prepare HTML IMG

    I am migrating a legacy app from Tomcat 5 to Weblogic 11g (10.3). In the legacy app, servlets write HTML that uses relative paths for <IMG src="../images/img.jpeg"> and <script src="../javascript/js.js">. The app is deployed as an exploded archive. Unfortunately, none of the images or script are being loaded. I've tried using http://serverIP:portNum/contextName/images/img.jpeg"> but it doesn't work. I've also checked to get the name of the context in the servlet and it's the root context. Could it have something to do with me having to append .war onto the application when I deploy it? Would it help if I deployed it as a war inside an ear? Basically, I want some best practices for doing this on Weblogic 11g. There are a lot of images and javascript and I'm really hoping they don't have to be inserted using ClassLoader.getResourceAsStream()... thank you.

    Best Practice:
    1. Move Static files like images, css, java scripts to a web server infrastructure if available.
    2. If this is not your case, then please send your directory information how you have packaged your EAR. I can advice :)

  • Best practices to include client libraries used at component level

    How to include component level resources, while following the best practices.
    Ex:
    I am looking at the geometrixx media site in CQ 5.6. In the some of components, ex: 2-col-article-summary we have a client library defined under the component.
    /apps/geometrixx-media/components/2-col-article-summary
    -2-col-article-summary.jsp
    -clientlibs
      -css
      -css.txt
    if i look at the categories of the clientlib, it is defined as follows
    categories String[] apps.geometrixx-media, apps.geometrixx-media.2-col-article-summary
    The only place this client library is included is in the head.jsp of main page level component.
        <cq:includeClientLib categories="apps.geometrixx-media.all"/> - this in turn embeds the apps.geometrixx-media
    my questions are as follows
    1) Why do we have two categories for the clientlib, if ithe second category name( apps.geometrixx-media.2-col-article-summary) is not being used. Is there is some other usage for this i am missing.
    2) Also these set of css is always included no matter whether a specific component is added to the page or not.
    3)  I could use the following to include the client at the component level, but this will cause unnecessary <script> and <link> elements at the component level mark up.
       <cq:includeClientLib categories="apps.geometrixx-media.2-col-article-summary"/>
    Essentially  i am trying to understand, how to include a specific component level resources while following the best practices

    Hi,
    I dont have CQ5.6 setup so could not see referring example but by looking at description i can say
    Ans 1. The client library can be invoked directly through tag lib as <cq:includeClientLib categories="category name"/> but it can also be invoked when you add "dependencies" property to client library folder and in that case it resolve all the dependencies first. So to answer your quesiton by looking at client library folder configuration you can not say that specific category has not been used any where or not invoked.
    Ans 2. Invoking client library folder completely depends on your code where you are placing call for client library using <cq:includeClientLib> tag and dependencies configuration. So you have to dig it more to trace out all the calls (also default css/js loads)
    Ans 3. Correct. So accomplish that best way to manage client library at component level and give it a unique name which can not be invoked any where neither through <cq:includeClientLib> call nor through dependenies configuration. This way you can avoide overridding of same library files. (Better to manage proper hierarchy of library)
    Hope it gives you some idea .
    Thanks,
    Pawan

  • Best Practices for SAP, connections to use without BW

    Hello,
    Could you help me to solve a problem with Best Practices 4.31?
    SAP Integration Kit XI 3.1 SP3 has 4 options  :
    u2022     SAP Infosets
    u2022     SAP BW MDX Query
    u2022     SAP BW Query
    u2022     Table, cluster ou function
    When I install SAP Integration Kit XI 3.1 SP3 for serveur, I have one additional service in my CCM,  BW Publisher 12. This service is from SAP BW.
    From the article /people/glen.spalding/blog/2010/08/04/the-full-montypart-13bobj-integration-kit-sp3-install-configure and installation guide for IIntegration Kit XI 3.1 SP3 I've learned that I need SAP BW to use Infosets dans Crystal Reports.
    Dans notre installation nous avons
    u2022     SAP ERP système without BW,
    u2022     BOBJ Server Setup,
    u2022     BOBJ Edge Integration Kit for SAP XI 3.1 SP3 server
    u2022     BOBJ Client Setup,
    u2022     BOBJ Edge Integration Kit for SAP XI 3.1 SP3 client
    u2022     Crystal Reports 2008
    u2022     Xcelsius 2008
    I've got Best Practices reports via « Table, cluster ou fonction » connection. There is no problem with this type of connection !
    I suppose that « SAP Infosets », « SAP BW MDX Query » et « SAP BW Query » connections should be used with SAP BW.
    Could you give me your opinion about this question?
    Thanks beforehand,
    Malika

    Hi,
    the InfoSets are a connection option for the classic InfoSets in an ERP system. Would suggest you take a look at the user guide for the SAP Integration Kit
    ingo

  • Best  practice: Fade out content when slide advances automatically?

    I have a course that uses the playbar to navigate and have the slides advancing (for the most part) automatically. Do you recommend setting the last content that's on the slide to fade out? I feel like it's jumpy if it doesn't but looks so sudden too.
    Please share your best practices!

    This one of those "it depends" questions.
    For the very first introductory module of a course I generally do not have the final slide fade out because I usually have a graphic on that final slide that shows what they should click next in the LMS interface to get to the next lesson.  So I want that image to remain visible until they shut down the browser window.  Doing this has avoided issues with newbie users getting lost in the LMS and not knowing how to proceed.
    However, this ends up being unnecessary for all subsequent modules, so for them I usually have the Project End option set to fade out.

  • VM Activation problem when using Internal Virtual Switch mode

    Hello,
    I have setup Windows 2012 R2 Standard Version host machine.  Hyper-V setup successful. Host machine already activate license.
    I have got a problem is when VM connecting with Internal virtual Switch.  It will not able to process Windows activation.  If it connected using External Virtual Switch, it can successful do windows activation.  However, using External Virtual
    Switch, one more WAN IP will be used.
    I am using ICS method to share the network interface to Internal Virtual Switch.
    Is it what problem and how to resolve it?
    Thanks!

    Hi Gyorgy,
    What is the version of your guest operating system? Why not we create an External Virtual switch to access Internet when using Hyper-V guest system? (Some reference if needed:Hyper-V Virtual Switch
    Overview )
    The network connection seems to be fine, if you mean the webpage can't be displayed through Internet Explorer, please take a look at the article below:
    "Internet Explorer cannot display the webpage" error
    For more information, please check:
    Fixing "Page cannot be displayed"
    Best regards
    Michael Shao
    TechNet Community Support

  • CVI under W2K crashes when using stop in debug mode and AOGenerateWaveforms

    When using AOGenerateWaveforms to generate a sine wave. If you hit stop in debug the PC restarts. If you're using the same channel to generate a DC signal, it's fine. Seems to be the card accessing the dynamic memory I've set the waveform up in after you hit stop and the memory has been free'd.

    This is sometimes the nature of the beast. When your program is running you are doing DMA transfers to locations in memory that have been setup by your operating system. Now if you stop your program, with out killing or stopping the DAQ device it will cause you problems in some cases. You need to call AOClearWaveforms before you stop your program.
    If you do not stop the DAQ device, before stopping your program it will cause the DAQ device to write to memory that is no longer allocated for that process. In most cases windows detects this as an error and reboots the system, because it thinks a program has crashed and is now writing to memory locations out of its designated space.
    I hope this helps.
    Joshua

  • Computer does not put itself to sleep when used in lid-closed mode

    Hi. I'm having a small problem when using my macbook pro with the LED display, lid closed and with external keyboard and mouse. If I leave the computer unattended, the display goes to sleep at the time specified in system settings under energy saver. The computer, however, does not go to sleep - the white power indicator remains solid and noises for new email etc can still be heard.
    I should probably just try to remember to put it to sleep manually but I keep forgetting. Does anybody know what settings I need to adjust?

    I tried cycling the old battery many times, and the computer would still shut down unexpectedly instead of going to sleep when it ran out of power. I got a free replacement from the supplier [http://myworld.ebay.com/cbkusaelectronics>, but I have cycled this new battery twice now, and it still does not go to sleep when it runs out of power. The capacity of the battery is fine (4.9 Ah), and the timing calibration seems fine as well. I have two older batteries from NewerTech and they allow sleep every time, so I think my computer is fine. Can anyone enlighten me about the cause of this problem? Does my battery lack some 'sleep battery' cell in it for use after the other 4 cells discharge? Or does my battery lack a chip that communicates with the computer? Is there any hope of getting this battery to support sleep when discharged?
    Has anyone else seen this problem consistently in a brand of battery? Does anyone have a recommendation for a cheaper battery that does not exhibit this problem? I have been happy with NewerTech, but they cost over $130.
    (I only paid $44 plus $10 shipping, so I probably got what I paid for. It is maybe still worth it to have a high capacity battery for such a low cost, but I can't rely upon it in those last few minutes or I may lose work.)

  • Screen goes dark (sleep) when using handset in phone mode

    When I am talking on the phone and remove the phone to hang up the phone goes dark (into sleep mode?) Is anyone else having this issue, does anyone have a solution? Thanks in advance for your help

    I had that problem when using a case that covered the proximity sensor. I don't know if you're using a case or not, but taking off the case solved the problem right away.

  • Best practice for JavaFX lifecycle when not used as a normal application

    Suppose there was a Java app that runs in a terminal with its own command line interface. Certain commands typed in would show a Window that contains a Stage/Scene. In order to accomplish this, I would call Application.launch( App.class, null );. The problem is this Window may be closed by the User and then a future command may bring up another Window. As we know Application.launch may only be called one. Also the launch method blocks the calling thread.
    Naively I would create an adapter that spawns a thread on the initial call to call Application.launch for the first window and bypass this for future windows.
    Though if someone has an understanding of the best way to go about spawning arbitrary Scenes from an app where the app lifecycle is at a larger scale than javafx.application.Application that would be most appreciated.

    In this situation, there isn't really a "primary" stage but an initial one. I can hook up a thread to call Application.launch and instrument it so the Stage it creates (and calls start on) is handled and additional stages will bypass Application.launch and create new Stages.
    But this feels like a kludge to me.
    I wonder if there is a different way to get the Toolkit operating in the background. This would be similar, I suppose, to how calling setScene on JFXPanel works.
    Ideas anyone?

Maybe you are looking for

  • VPRS cost and batch management

    Hi Gurus, Is there any link between the cost and batch management? Currently users has posted a billing document and the cost is 0. The VPRS value in the SO is correctly but it is not brought to the billing document. We have tested and it seems that

  • Document Update Error

    Hello Everyone,                               When i update my document form (UDO) i get the following error "Violation of primary key constraint  KPACK_PR. Cannot insert  duplicate key in object @PACK doument". Sometimes i get error as Corrupted Dat

  • Connecting audio speakers via external Bluetooth module

    I have an USB bluetooth module (manufactured by Broadcom) to connect to a bluetooth-enabled stereo, but it won't work. I think it must be a software problem. Can anyone help with this? The device is set up properly, and paired with the stereo. But in

  • How to turn off spell check

    What do I need to do to remove spell check when typing (turn off and on) as I type messages in different languages

  • . NET to call Oracle stored procedure, use an array of types of parameters

    . NET to call Oracle stored procedure, use an array of types of parameters Step1:(In the Oracle database define an array of types) CREATE OR REPLACE TYPE STRING_VARRAY AS VARRAY (1000) OF NVARCHAR2(255) Step2: CREATE OR REPLACE PROCEDURE Test (i_test