Getting notification back to java application with Goldengate

Question

Hello,
I am new to using Golden Gate. I have a java application running in an application server environment using Oracle database. We plan to use Goldengate to receive data directly in the database from external database. I have few questions in using goldengate:
1. Is there a way for my application to know that database change to a table has happened?
2. I came across goldengate and jms notification. Is there a non-jms way using java API for my application to know real-time that change has happened to database table?
3. Are there any samples for this type of feature?
Thank you.

Answer 1

There is a Java API that can be used directly, rather than just using the built-in JMS delivery.
The way it works: "extract" reads a trail, as the data is read, events are generated, and your custom event handler can receive these events. Types of basic events are "transaction begin", "new operation (insert, update, delete)", "transaction commit". These events are basically replaying what has happened on the source database; so, for example, if on the source DB you have "begin tx" + "insert" + "update" + "commit", then your custom Java event handler will receive an event for "begin tx", "insert", "update" and "commit".
One thing to consider early on: the event handlers can work in one of two "modes". Either an entire source database transaction can be cached in memory and processed at once, OR you can process one operation at a time. The latter is typically preferable, since any DBA or batch job can be executed on the source DB that updates a few million rows at once, which could cause your JVM to run out of memory in "transaction mode" as it tries to cache this in memory. On the other hand, in "operation mode" you only need enough memory to hold one operation at a time, but you don't know what operations have already happened or what will happen. In "operation mode" you are free to cache your own data in your custom event handler, though -- as much or as little as you need.
Note: GoldenGate only captures committed transactions, so you won't ever receive rolled back database transactions; however , there may be "interruptions" in the GG trail if/when a GG process abends & restarts. This puts a virtual "rollback" into the trail (technically, it's a restart/abend marker) which a GG replicat fully understands. The GG user-exit API prior to v11.2 does not pass RestartAbend events to any user-exits -- including the Java adapter. Long story short, your source "extract" (the one capturing changes from the source DB) must create trails with the "Format Release 9.5" qualifier, or (equivalently) specify "RecoveryOptions OverwriteMode". If you do this, then there will be no "restart/abend" markers in the trail. However, under certain (odd) circumstances, it may not be entirely possible to create trails in this format. The format 10+ trails (aka "RecoveryOptions AppendMode") are slightly more resilient to system failures, and do have additional file header info that is not available in "format 9.5" trails. Although GG 11.2 is available, the user-exits (the Java adapter & flat-file writer) that use this new API are presently not yet generally available.
Ok, with the caveats out of the way, here's a code sample:
package tst;
import com.goldengate.atg.datasource.*;
import com.goldengate.atg.datasource.GGDataSource.Status;
public class HelloWorldHandler extends AbstractHandler {
  private long numTxs = 0;
  private long numOps = 0;
  private long numCols = 0;
  @Override
  public Status operationAdded(DsEvent e, DsTransaction tx, DsOperation operation) {
    super.operationAdded(e, tx, operation);
    numOps++;
    numCols += operation.getNumColumns();
    return Status.OK;
  @Override
  public Status transactionCommit(DsEvent e, DsTransaction tx) {
    super.transactionCommit(e, tx);
    numTxs++;
    return Status.OK;
  @Override
  public String reportStatus() {
    String s = "Processed (mode='" + getMode() + "')" + " transactions=" + numTxs
              + ", operations=" + numOps + ", columns=" + numCols;
    return s;
}That's about as basic as it gets. It will print out number of operations processed to the report file (dirrpt/*.rpt). The Java user-exits typically are an "end point" for the data stream (that's what the "CuserExit... PassThru" option means), so this example isn't terribly useful by itself. Typically you would send the data somewhere else (like the JMS handler, or a file-writer) or update some other system with these events.
Btw, there are also a few helper classes to merge metadata (column/table names) and data (column "before" and "after" data). The "DsTransaction" and "DsOperation" and "DsColumn" classes are just "data". The following wrapper classes also provide metadata (Tx, Op, Col):
  import com.goldengate.atg.datasource.adapt.*;
  Tx tx = new Tx(dsTransaction, getMetaData(), getConfig());
  //or:  Op op = new Op(dsOperation, tableMeta, getConfig());
     for(Op op: tx) {
          for(Col c: op) { ... }
  }See the javadoc that comes with the software download for details (javadoc.zip).
To compile and use (preferably creating a jar; I'm assuming you put it in "dirprm" (see below)):
$ javac -d classes -cp {gg_home}/ggjava/ggjava.jar HelloWorldHandler.java
$ jar -cvf myCustom.jar ...etc...Your extract ("pump") parameter file that loads and runs your custom Java user-exit event handler:
Extract javaue
SourceDefs dirdef/tc.def
SetEnv ( GGS_USEREXIT_CONF = "dirprm/javaue.properties" )
GetEnv (JAVA_HOME)
GetEnv (LD_LIBRARY_PATH)
-- CUserExit ggjava_ue.dll CUSEREXIT PassThru IncludeUpdateBefores
CUserExit libggjava_ue.so CUSEREXIT PassThru IncludeUpdateBefores
GetUpdateBefores
-- must pass all data to user-exit, or
-- else tx indicators might be missed
TABLE GGS.*;
TABLE EXAMPLE.*;And the referenced properties file could look like the following. There are really two parts to this file; the first part is used to configure the Java application, the second is used to configure the JNI bridge between the JVM and "extract".
# Java application properties
gg.handlerlist=mytest
# your custom event handler.
# note: setting property foo=bar automatically calls your handler's method setFoo("bar")
gg.handler.mytest.type=tst.HelloWorldHandler
gg.handler.mytest.foo=bar
gg.handler.mytest.mode=op
# gg.handler.mytest.mode=tx
# Properties for native library ("C" User Exit)
# set to TRUE to *disable* the duplicates-checkpoint-file
goldengate.userexit.nochkpt=TRUE
# duplicates-checkpoint-file prefix
goldengate.userexit.chkptprefix=JAVAUE_
# tx timestamp datatbase local (default) or UTC timestamp
goldengate.userexit.timestamp=utc
# C-user-exit logging config for native library *only*.
# Java app uses log4j config in javawriter.bootoptions.
goldengate.log.modules=TXSTORE,JAVAWRITER,JAVAUSEREXIT
goldengate.log.level=INFO
goldengate.log.tostdout=false
goldengate.log.tofile=true
# prefix for native library logfile name
goldengate.log.logname=cuserexit
goldengate.userexit.writers=javawriter
# native lib statistics, defaults: time=3600, numrecs=10000
#   display=false (doesn't write to file)
#   full=false (would not include java report)
javawriter.stats.time=3600
javawriter.stats.numrecs=100
javawriter.stats.display=TRUE
javawriter.stats.full=TRUE
# Set classpath to required jars.
#    Use ':' path separator for Unix, ';' for windows.
# Set the log4j configuration -- note this found in the classpath.
#    See the example preconfigured log4j files in ggjava/resources/classes,
#    copy to dirprm and rename, then customize as desired. (Don't put or
#    modify files in the ggjava/* directory.)
javawriter.bootoptions=-Xmx64m -Xms32m -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/myCustom.jar -Dlog4j.configuration=log4j.propertiesNote that the property "bootoptions" includes "myCustom.jar" to find your class. You can use your custom log4j config as well (e.g., my-log4j.properties); put it in "dirprm" as well and it will be found in the classpath.
See also the GoldenGate Java docs for a little more on this topic:
* http://www.oracle.com/technetwork/middleware/goldengate/documentation/index.html => http://docs.oracle.com/cd/E18101_01/index.htm
Hope it helps,
-Michael
Edited by: MikeN on Jun 20, 2012 10:34 PM - notes on trail format compatibility