High Availability Of Service Replicated Across Domains

Question

Hi,
We have two Tuxedo application , one generate message and calls service of remote domain to send to another Tuxedo application(fix engine) which sends it to external world. There are two remote domains(individual means on separate nodes named SSGWBest and SSGWBoxt) which have the same service(OutFixEn) published of fix engine. We have done this for high availability scenario if one machine is not available or crashed then message can still be sent to external world.
We are using Oracle Tuxedo, Version 10.3.0.0, 64-bit, Patch Level 095 on AIX 6.1 Power 7 machine. Following is snippet of domain configuration to show how service is published in local domain. SSWBest and SSGWBoxt site both publish service OutFixEn as local and remote(to point to another) sections.
*DM_LOCAL_DOMAINS
DEFAULT: SECURITY = NONE
Dom1 GWGRP = LGWGRP
TYPE = TDOMAIN
DOMAINID = "PATDom1"
DMTLOGDEV = "/appl/aer/a01/data/tcs_bancs//DMLOGDEVICE"
DMTLOGNAME = "DMLOGDEVICE"
*DM_REMOTE_DOMAINS
Dom2 TYPE = TDOMAIN
DOMAINID = "PATDom2"
SSGWBest TYPE = TDOMAIN
DOMAINID = "SSGWBest"
SSGWBoxt TYPE = TDOMAIN
DOMAINID = "SSGWBoxt"
*DM_TDOMAIN
# Local network addresses
Dom1 NWADDR = "//uaix3017.unix.bank.nl:50708"
# Remote network addresses
Dom2 NWADDR = "//uaix3028.unix.bank.nl:50708"
#SSG Machine1 Network Address
SSGWBest NWADDR = "//uaix3021.unix.bank.nl:50708"
#SSG Machine2 Network Address
SSGWBoxt NWADDR = "//uaix3034.unix.bank.nl:50708"
*DM_LOCAL_SERVICES
sh_COETGETMESSG
sh_COETPICXML
sh_COETFLATFILE
sh_COBTRPAIRMSG
InpFixEnOC1
InpFixEnOC2
InpFixEn1
InpFixEn2
InpFixBrs
InpFixIon
InpFixRtrs
InpMmtpEnDrv
InpMmtpEnCash
*DM_REMOTE_SERVICES
sh_COETGETMESSG
RACCESSPOINT=Dom2
sh_COETPICXML
RACCESSPOINT=Dom2
sh_COETFLATFILE
RACCESSPOINT=Dom2
sh_COBTRPAIRMSG
RACCESSPOINT=Dom2
OutFixEn
RACCESSPOINT=SSGWBest
OutFixEn
RACCESSPOINT=SSGWBoxt
OutFixBrs
RACCESSPOINT=SSGWBest
OutFixIon
RACCESSPOINT=SSGWBest
OutFixRtrs
RACCESSPOINT=SSGWBoxt
OutMmtpEnDrv
RACCESSPOINT=SSGWBest
OutFixEnOC
RACCESSPOINT=SSGWBoxt
*DM_ROUTING
We tried to test this scenario and started calling service OutFixEn from local domain and during this run we shutdown the tuxedo application server on SSWBoxt site so that OutFixEn was not available (To create service un-availability scenario). Our understanding was that all service calls only land on SSWBest site as domain will suspends this site for service availbility but it did not happen as first few service call failed with TPETIME (my assumption was it will fail through TPENOENT)and then service landing on SSGWBoxt were routed on SSGBest site.
Based on this test scenario, i have following questions.
1/ How to achieve routing of services to available domain with minimal service failures (means my only first one or two services fail and then application adjust to route service to available domain) ?
2/ Is there any other better way to organize these services so that better load balancing and high availability can be ensured ?
Regards,
Ajeet Tewari

Answer 1

Hi,
It is possible to configure failover and load balancing as you suggest, but that won't solve the problem described. The issue is that the local domain gateway doesn't know the availability of the services in a remote domain. It decides to advertise imported services locally only based on the connection establishment policy (ON_STARTUP or ON_DEMAND), and not the actual state of the remote service. If the connection policy is ON_DEMAND, the service is always advertised locally and when a request arrives for that service, the domain gateway will establish a connection to the remote domain if one isn't present. If the connection policy is ON_STARTUP, the domain gateway won't start advertising the imported services locally until the connection is established. However, once the connection is established, it assumes the imported service is available at the remote domain.
You have a couple of options here. One is to make sure the service is highly available on the domain, such that when a connection exists to the domain, the service will be available. The other alternative is to switch to an MP single domain configuration as the availability of a service across machines is known.
Regards,
Todd Little
Oracle Tuxedo Chief Architect