Distributed polling

Hi ,
While configuring DB adapter there is an option of distributed polling. In an active-active setup, distributed polling
techniques can be used for inbound Database Adapters to ensure that the same data is not retrieved more than
once. Anyone knows how does it work internally is there any other configuration we need to do to achieve it?
Any help in this regard is highly appreciated.
With Regards,
Harsh

You can Emulate AQ functionality for any Non-Oracle Database where sometimes the *Clustered DB-Adapter[Poller]* polls the same record more than once.
Say, you need to poll 'status' column of table 'A' for a status 'POLL_IT'
1. Create a Table 'DUMMY_AQ' matching required column signatures from table 'A'
2. Create a Trigger on table 'A' for copying the row content on Insert or Update for status 'POLL_IT' in to 'DUMMY_AQ' table;
3. Poll DUMMY_AQ with Row Level Read lock & Delete On Read , that way it will is ensured that so that only One Instance of your Clustered DB Adapter can have visibility to any newly inserted row having status 'POLL_IT' .

Similar Messages

Duplicate processing by DBAdapter when using Distributed Polling with Logical Delete Strategy

We have DBAdapter based polling services in OSB running across two Active-Active clusters (total 20 managed service across 2 clusters),
listening to the same database table. (Both clusters read from the same source DB). We want to ensure distributed polling without duplication,
hence in the DBAdapter we have selected Distributed Polling option, meaning we are using "Select For Update Skip Locking".
But we see that sometimes, same rows are processed by two different nodes and transactions are processed twice.
How do we ensure that only one managed server processes a particular row using select for update? We do not want to use the markReservedValue option which was preferred in older version of DBAdapter.
We are using following values in DB Adapter configuration, the Jdev project for DBAdapter and the OSB proxy using DBAdapter are attached.
LogicalDeletePolling Strategy
MarkReadValue = Processed
MarkUnreadValue = Initiate
MarkReservedValue = <empty as we are using Skip Locking>
PollingFrequency = 1 second
maxRaiseSize = 1
MaxTransactionSize = 10
DistributionPolling = checked (adds lock-n-wait in properties file and changes the SQL to SELECT FOR UPDATE SKIP LOCKED)
Thanks and Regards

Hi All,
Actually I'm also facing the same problem.
Step I follwed:
1) Created a job_table in database
create talbe job_table(id, job_name, job_desc, job_status)
2)created a bpel process to test the Inbound distributed polling.
3)Configure the DBAdapter for polling.
a)update a field in the job_table with logical delete.
b)select the field name form the drop down
c) change the read value-->Inprogress and unRead value--->Ready
d) dont change the value for Reserved value
e) select the check box for "distributed polling".
f) the query will be appended with "For update NoWait."
g)click next and then finish.
4) Then i followed the below steps.
To enable pessimistic locking, run through the wizard once to create an inbound polling query. In the Applications Navigator window, expand Application Sources, then TopLink, and click TopLink Mappings. In the Structure window, click the table name. In Diagram View, click the following tabs: TopLink Mappings, Queries, Named Queries, Options; then the Advanced… button, and then Pessimistic Locking and Acquire Locks. You see the message, "Set Refresh Identity Map Results?" If a query uses pessimistic locking, it must refresh the identity map results. Click OK when you see the message, "Would you like us to set Refresh Identity Map Results and Refresh Remote Identity Map Results to true?Ó Run the wizard again to regenerate everything. In the new toplink_mappings.xml file, you see something like this for the query: <lock-mode>1</lock-mode>.
5) lock-mose is not changed to 1 in toplink_mappingss.xml
Can we edit the toplink_mappings.xml manually.
If yes, what allt he values Ineed to change in toplink_mappings.xml file, so that it will not pick the same record for the multiple times in clustered environment.
Please help me out this is urgent.
Thanking you in advance.

Distributed Polling Not working for Sybase Database poller Proxy

I am using a Proxy service which is polling records from a sybase database. When I tried to deploy this proxy in a clustered environment I noticed sometimes it is polling the same record twice. So I tried to enable distributed polling in my proxy by adding *<lock-mode>lock-no-wait</lock-mode>* in my topplink_mapping.xml file. But activating this change resulted error in the log file stating "Incorrect syntax near 'NOWAIT'".
Later on I found out that it is a known issue ...for reference check [http://www.oracle.com/technetwork/middleware/docs/aiasoarelnotesps5-1455925.html|http://www.oracle.com/technetwork/middleware/docs/aiasoarelnotesps5-1455925.html] with bug # 13405718.
So I am stuck here...and no solution yet.
Any suggestion??

You can Emulate AQ functionality for any Non-Oracle Database where sometimes the *Clustered DB-Adapter[Poller]* polls the same record more than once.
Say, you need to poll 'status' column of table 'A' for a status 'POLL_IT'
1. Create a Table 'DUMMY_AQ' matching required column signatures from table 'A'
2. Create a Trigger on table 'A' for copying the row content on Insert or Update for status 'POLL_IT' in to 'DUMMY_AQ' table;
3. Poll DUMMY_AQ with Row Level Read lock & Delete On Read , that way it will is ensured that so that only One Instance of your Clustered DB Adapter can have visibility to any newly inserted row having status 'POLL_IT' .

Select for update no wait distributed polling strategy

I have an esb process which currently uses 'logicaldeletepolling strategy'.
I need to change it to use the 'select for update no wait' distributed polling strategy.
What steps do I need to perform on the existing process ?

Hi,
Try: http://download-east.oracle.com/docs/cd/B31017_01/integrate.1013/b28994/adptr_db.htm#sthref558 (Oracle® Application Server Adapters for Files, FTP, Databases, and Enterprise Messaging User's Guide
10g Release 3 (10.1.3.1.0)
Part Number B28994-02).
Also you can reconfigure DB adapter (I think this's the best way): laucnch adapter configuration wizard in JDev and at step 9 (adapter options) check distributed polling checkbox (query now contains for update clause).

DB polling using custom SQL in SOA Suite 11g

Hi,
We are trying to poll records from a database using custom sql. But all the records in the table are being picked up at the same time and not as per the polling frequency set in the adapter.
Below is the configuration from the .jca file:
<endpoint-activation portType="II_ptt" operation="receive">
<activation-spec className="oracle.tip.adapter.db.DBActivationSpec">
<property name="DescriptorName" value="II.OrderRequest"/>
<property name="QueryName" value="IISelect"/>
<property name="MappingsMetaDataURL" value="II-or-mappings.xml"/>
<property name="PollingStrategy" value="DeletePollingStrategy"/>
<property name="PollingInterval" value="60"/>
<property name="MaxRaiseSize" value="10"/>
<property name="MaxTransactionSize" value="10"/>
<property name="SequencingColumn" value="REQUEST_SAK"/>
<property name="NumberOfThreads" value="1"/>
<property name="ReturnSingleResultSet" value="false"/>
<property name="DeleteDetailRows" value="false"/>
</activation-spec>
</endpoint-activation>
Please let us know if anything else needs to be set to enable the polling as per the frequency in the adapter.
Thanks.

As the link from Anuj said, you need to also configure 'Distributed Polling' in the wizard. This speed limit trick will no longer work out of the box in 11.1.1.3 on, you will also have to set usesSkipLocking="false" in your DbAdapter connection pool definition. Skip locking eliminates the locking contention issue, so this primitive kind of load balancing is no longer needed in that case.
Thanks
Steve

Issue in Clustered environment DB adapter Polling

hi all ,
i have a table and one BPEL process is polling records depending on status value .
Polled records are updated from N to P after polling .
but , In cluster environment for One Row Multiple Instances are creating .
Is there any way to ensure that in Cluster env. also for single Row , only one BPEL instance will be created ?
I am using SOA 10.1.3.4
i have used that "Distributed Polling" option in DB adapter .
In JCA file i am using below configaration : -
ActivationSpec="oracle.tip.adapter.db.DBActivationSpec"
DescriptorName="PollTable.InterfaceData"
QueryName="PollTable"
PollingStrategyName="LogicalDeletePollingStrategy"
MarkReadFieldName="STATUS"
MarkReadValue="P"
MarkUnreadValue="N"
SequencingFieldName="ENTRYTIME"
MaxRaiseSize="1"
MaxTransactionSize="1"
PollingInterval="10"
NumberOfThreads="1"
UseBatchDestroy="false"
ReturnSingleResultSet="false"
In that i am facing the Table Lock scenario .
Even if i having 10 Records in database Row .Its locking all other 9 row and no instance are generating .
Regards ...
Edited by: 807572 on Feb 28, 2012 9:36 AM

Use Reserved Value feature to avoid duplicate instances.
Even if i having 10 Records in database Row .Its locking all other 9 row and no instance are generating .Do you see any error in $ORACLE_HOME/opmn/logs/<BPEL/ESBRT container>.log file? If you are having 1-M relationship defined in adapter then possibly you are hitting a known issue.
Regards,
Anuj

DB Adapter Polling Issue

Hi All,
I am working on a scenario something like this:
I am polling an Oracle table and when certain flag is 'Y', I am selecting the record(s) and calling an external service. Then depending on the response from the external service, again updating the the same table which I am polling.
As I have clustered environment, I am using distributed polling and polling strategy is logical delete. But it seems DB adapter is keeping some lock on the table for distributed polling, that's why I am not able to update the polled table.
Please provide me some solution how to overcome the iisue.
Thanks,
Maitrayee

Hi Ram,
DbAdapter is transactional and it should not delete failed records...
Ref docs here...
http://docs.oracle.com/cd/E28280_01/integration.1111/e10231/adptr_db.htm#BABCIICF
Cheers,
Vlad

Polling DB Adapter is not working when "NumberOfThreads" value= 1

Hi All,
I am facing an issue with DB adapter in OSB.
My requirement is, i have one table name as "Header Table",for every 1 sec around 5 messages are inserted into this table.By using that records i need to call API.
For this i created one DB adapter with polling option.It is poll the Header table for every 3 seconds and fetch the 5 records.Suppose in first iteration at 3 seconds it poll and get 5 records
One proxy service is created for that Adapter jca file.This proxy will get the records and call the API.
But here issue is the thread is poll the table after first iteration(first 5) records are processing only.After All 5 records are processed only the next polling happening.
That means every 3 sec polling is not happening.
For this i increased NumberOfThreads value =3 in Adapter jca file.Then polling is Not working,it is completely stopped not even 3 sec also?
I tried with both XA , NonXA adapters and Distributed polling option also.But it is not working.This is not working in Cluster environment and as well as my local system?
When i use NumberOfThreads=1 it is working,More than 1 it is not working
These properties i used for JCA file.
<activation-spec className="oracle.tip.adapter.db.DBActivationSpec">
      <property name="DescriptorName" value="GOP_DB_FSL_HEADER_POLLING_APICALL.XxGilSvcOrderHeader"/>
      <property name="QueryName" value="GOP_DB_FSL_HEADER_POLLING_APICALLSelect"/>
      <property name="MappingsMetaDataURL" value="GOP_DB_FSL_HEADER_POLLING_APICALL-or-mappings.xml"/>
      <property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
      <property name="MarkReadColumn" value="SOA_POLL_STATUS"/>
      <property name="MarkReadValue" value="C"/>
      <property name="MarkUnreadValue" value="W"/>
      <property name="PollingInterval" value="3"/>
      <property name="MaxRaiseSize" value="1"/>
      <property name="MaxTransactionSize" value="5"/>
      <property name="NumberOfThreads" value="1"/>
      <property name="ReturnSingleResultSet" value="false"/>
      <property name="SequencingColumn" value="OFS_CREATEDDATE"/>
      <property name="MarkReservedValue" value="R${weblogic.Name-2}-${IP-2}"/>
    </activation-spec>
===========================================================
Professionals, Please Help me on the same
Thanks in Advance
AV

Hi All,
I am facing an issue with DB adapter in OSB.
My requirement is, i have one table name as "Header Table",for every 1 sec around 5 messages are inserted into this table.By using that records i need to call API.
For this i created one DB adapter with polling option.It is poll the Header table for every 3 seconds and fetch the 5 records.Suppose in first iteration at 3 seconds it poll and get 5 records
One proxy service is created for that Adapter jca file.This proxy will get the records and call the API.
But here issue is the thread is poll the table after first iteration(first 5) records are processing only.After All 5 records are processed only the next polling happening.
That means every 3 sec polling is not happening.
For this i increased NumberOfThreads value =3 in Adapter jca file.Then polling is Not working,it is completely stopped not even 3 sec also?
I tried with both XA , NonXA adapters and Distributed polling option also.But it is not working.This is not working in Cluster environment and as well as my local system?
When i use NumberOfThreads=1 it is working,More than 1 it is not working
These properties i used for JCA file.
<activation-spec className="oracle.tip.adapter.db.DBActivationSpec">
      <property name="DescriptorName" value="GOP_DB_FSL_HEADER_POLLING_APICALL.XxGilSvcOrderHeader"/>
      <property name="QueryName" value="GOP_DB_FSL_HEADER_POLLING_APICALLSelect"/>
      <property name="MappingsMetaDataURL" value="GOP_DB_FSL_HEADER_POLLING_APICALL-or-mappings.xml"/>
      <property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
      <property name="MarkReadColumn" value="SOA_POLL_STATUS"/>
      <property name="MarkReadValue" value="C"/>
      <property name="MarkUnreadValue" value="W"/>
      <property name="PollingInterval" value="3"/>
      <property name="MaxRaiseSize" value="1"/>
      <property name="MaxTransactionSize" value="5"/>
      <property name="NumberOfThreads" value="1"/>
      <property name="ReturnSingleResultSet" value="false"/>
      <property name="SequencingColumn" value="OFS_CREATEDDATE"/>
      <property name="MarkReservedValue" value="R${weblogic.Name-2}-${IP-2}"/>
    </activation-spec>
===========================================================
Professionals, Please Help me on the same
Thanks in Advance
AV

DB adapter polling in OSB

1. In a clustered environemnt, we need to use "Distributed polling" in DB adapter to make sure the multiple cluster instances are not polling for the database at the sametime. If "Distributed polling" can achieve that, what is the need to have singleton property set for the DB adapter in the composite.xml
2. For achieving DB adapter polling in osb, we need to create the adapter in bpel and then export them to osb project. In such OSB project, where do we set the singleton property ?

Thanka Ananth.
I assume active-passive cluster setup means,
there will be a cluster with only one osb or bpel managed server (active cluster)
There wil be another cluster with only one osb or bpel managed server (passive cluster).
so when there is some data inserted into the database, there will be only one managed server (available on active cluster) will be available to poll for the recored. Passive man server does not poll for this record. I assume this is the default behaviour. If such is the case, then what is the use of setting this singleton property ?

DBAdapter polling with logical delete x distrib polling x DB rows per trans

Hi all.
I'm trying to configure a DBAdapter with "logical delete" polling strategy, distributed polling (cluster environment) and a defined number of "Database Rows per Transaction".
When I check the box "Distributed Polling", the SQL generated gets appended by "FOR UPDATE NOWAIT"
However, when I set a value for "Database Rows per Transaction" the "FOR UPDATE NOWAIT" sql clause disappear.
Is this a bug, or some limitation related to the "logical delete" strategy???
Thanks
Denis

Hi All,
Actually I'm also facing the same problem.
Step I follwed:
1) Created a job_table in database
create talbe job_table(id, job_name, job_desc, job_status)
2)created a bpel process to test the Inbound distributed polling.
3)Configure the DBAdapter for polling.
a)update a field in the job_table with logical delete.
b)select the field name form the drop down
c) change the read value-->Inprogress and unRead value--->Ready
d) dont change the value for Reserved value
e) select the check box for "distributed polling".
f) the query will be appended with "For update NoWait."
g)click next and then finish.
4) Then i followed the below steps.
To enable pessimistic locking, run through the wizard once to create an inbound polling query. In the Applications Navigator window, expand Application Sources, then TopLink, and click TopLink Mappings. In the Structure window, click the table name. In Diagram View, click the following tabs: TopLink Mappings, Queries, Named Queries, Options; then the Advanced… button, and then Pessimistic Locking and Acquire Locks. You see the message, "Set Refresh Identity Map Results?" If a query uses pessimistic locking, it must refresh the identity map results. Click OK when you see the message, "Would you like us to set Refresh Identity Map Results and Refresh Remote Identity Map Results to true?Ó Run the wizard again to regenerate everything. In the new toplink_mappings.xml file, you see something like this for the query: <lock-mode>1</lock-mode>.
5) lock-mose is not changed to 1 in toplink_mappingss.xml
Can we edit the toplink_mappings.xml manually.
If yes, what allt he values Ineed to change in toplink_mappings.xml file, so that it will not pick the same record for the multiple times in clustered environment.
Please help me out this is urgent.
Thanking you in advance.

JCA Polling Adapter not picking up records

Hello All
I have been trying to configure a JCA polling adapter to pick up records from the Database, the first time I configured it, it read a whole bunch of records. Now I have configured the following in the database and am getting no records being polling.
1. Poll for new or changed records
2. Update a field in the table F40g02 for logical delete
3. Read value = READ, UNREAD value = UNREAD, Reserved value = RESERVED
4.Polling interval 5 seconds, Database XROWS per xml document 1, Database Rows per Transaction 10
I ran the SQL specified in the adapter and I get a record. Does anyone have any idea.
I have seen this warning in the EM
Database Adapter <oracle.tip.adapter.db.inbound.IPAddrPollingService getUniqueMarkReservedValue> The markReservedValue of distributed polling might not be a unique value. To make it so, you can add the last x digits (default = 2) of the host IP address using this format for the MarkReservedValue: [original_val]${IP[-x]}
Regards

Hello
The problem I am facing is I can't get the records that are processed (already READ) to not poll again despite the value being changed to the READ value configured, if I set the READ value in the configuration it will poll every record all the time despite the fact it is equal to the READ value. Then if I set the READ value and any other value no records will poll.
Consider 4 records in the DB (currently in test I have 431 but I have used distributed locking and ReturnSingleResultSet to limit it)
ID = 1, STATUS = PROCESSED
ID = 2, STATUS = UNPROCESSED
ID = 3, STATUS = UNPROCESSED
ID = 4, STATUS = OTHER
Config 1: If READ = PROCESSED records 1,2,3,4 will poll
Config 2: If READ = PROCESSED, UNREAD = UNPROCESSED no records will poll
Config 3: If READ = PROCESSED, UNREAD = UNPROCESSED, RESERVED = OTHER no records will poll
Config 4: If READ = PROCESSED and TOPLINK STATUS not equal string "PROCESSED" then records 1,2,3,4 will poll
What I want is Record 2 and 3 to poll which I believe is config 3 should do. However I believe there have been some changes that relate to distributed polling for clusters in 11.1.1.3 onwards which means in jdeveloper (11.1.1.6) when I go to re-config the db adapter after setting the RESERVED value it disappears from the editor .
JCA properties obviously with a MarkReservedValueand MarkUnreadValue column jumping in.depending on which config I am trying
      <property name="DescriptorName" value="dbPollBlockRecords.F40G02"/>
      <property name="QueryName" value="dbPollBlockRecordsSelect"/>
      <property name="MappingsMetaDataURL" value="dbPollBlockRecords-or-mappings.xml"/>
      <property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
      <property name="MarkReadColumn" value="STATUS"/>
      <property name="MarkReadValue" value="PROCESSED"/>
      <property name="PollingInterval" value="700"/>
      <property name="MaxRaiseSize" value="1"/>
      <property name="MaxTransactionSize" value="10"/>
      <property name="NumberOfThreads" value="1"/>
      <property name="ReturnSingleResultSet" value="true"/>   (This property has been manually configured, will unfortunately poll all records that match without it)
Regards

Database Poller issues in Clustered env

Hi,
I am running Oracle BPEL 10.1.3.4 on top of Weblogic version 9.x in a clustered env. I created a simple database poller BPEL process (for target table in Oracle database) with below parameters:
Max Raise Size: 1
Max Transaction Size: 2
Polling Interval:30
Distributed Polling is enabled
All other BPEL engine and domain level parameters have default values.
In my clustered env two instances of database poller BPEL process are running on diff nodes. Now I populated my table with 1000 records in one go. According to parameters I expect:
- In each polling interval max 2 record will be processed by database poller BPEL process running on single node. And there will be 2 instances of this bpel process as max raise size is 1. Same I expect for database poller BPEL process running on another node. So overall 4 records should be processed in each polling interval and 4 instances of BPEL processes should be visible in BPEL console. But in actual its always 2.
Is I am missing something here? Won't the load balancer distributes records to BPEL processes on both nodes equally?
Also then I raise Max transaction size to higher value e.g. 20. But in this case after processing nearly 1/3 rd of records, the BPEL process stopped picking up any further records. Is there any known issue where Adapter stops picking any further database records if transaction size limit is higher?
thanks
Ankit
Edited by: AnkitAggarwal on 22-Feb-2010 03:36

Hi Ankit,
I have Oracle BPEL env11.1.1.2.0 clustered over 2 nodes.
I am facing a similar issue. In my scenario I have Bpel (with DB poller) and its deployed on the cluster. And I am connected to DB via Multi data source (MDS-1) with only one datasource (DS-1) configured in it.
So whenever I update the table which is being polled, in some cases I have TWO instances and in some cases I have ONE Instance running. My requirement is to have only one instance running every time the DB poller is initiatied.
Kindly help me out.
Thank You
Best Regards
Prasanth

DB Adapter Polling with Reserved Value in SOA 11.1.1.6

Hi Experts,
I am currently working on DB Adapter Project in SOA 11.1.1.6. The project I am working on is deployed on SOA Cluster environment and accesses a RAC (Multi) datasource
Due to volume of data, I have set the NumberofThreads as 10 for the Polling DB Adapter.
Because of this configuration, its currently sending the duplicates to Target system, I have set the Distributed Polling Flag and Singleton property for the adapter, still its posting duplicates.
We planned to use the RESERVED VALUE concept for this approach.
I am using a connection pool where the UsesSkipLocking is set to FALSE and pointing to the same RAC DataSource. The Algorithm Type for Multi DataSource in LOAD-BALANCING
In DB Adapter, I have set the RESERVED VALUE string R${weblogic.Name-2}-${IP-2} and unchecked the Distributed Polling Flag.
Its updating the polling column from UNREAD VALUE to RESERVED VALUE and after that its not polling the records. Instances are not getting created in SOA.
Please can you help me in this regards, Thanks very much.
RK.

Hi Experts,
Please can you provide your suggestions / inputs regarding the above post.
Appreciate your help Thanks in Advance.
Thanks - RK.

Using Custom SQL in DBAdapter Polling

Hi All,
I am using DB Adapter Polling Service!
The expression builder allows to use only nested AND or OR conditions. Can I write customized SQL queries for Polling?
Thanks,
CD

As the link from Anuj said, you need to also configure 'Distributed Polling' in the wizard. This speed limit trick will no longer work out of the box in 11.1.1.3 on, you will also have to set usesSkipLocking="false" in your DbAdapter connection pool definition. Skip locking eliminates the locking contention issue, so this primitive kind of load balancing is no longer needed in that case.
Thanks
Steve

DB Adapter Polling Issues - causing DB Locks

Hi Experts,
I am working on project in SOA Suite 11.1.1.6 environment.
In my project I have used DB Polling which has following settings
<property name="DescriptorName" value="XXXXXXXXXXX"/>
<property name="QueryName" value="XXXXXXXXXXXXXXXXXXXXX"/>
<property name="MappingsMetaDataURL" value="XXXXXXXXXXXXXXXXXXXX.xml"/>
<property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
<property name="MarkReadColumn" value="PROCESS_FLAG"/>
<property name="MarkReadValue" value="P"/>
<property name="MarkUnreadValue" value="Y"/>
<property name="PollingInterval" value="20"/>
<property name="MaxRaiseSize" value="1"/>
<property name="MaxTransactionSize" value="10"/>
<property name="NumberOfThreads" value="1"/>
<property name="ReturnSingleResultSet" value="false"/>
The problem is when I update 25 records, the last 5 records are not getting processed and its just stuck there. It just shows DB Adapter Instance in the Composite Instance and the record is locked.
I am using Multi DataSource configuration with 3 Load Balancing Database Instances on SOA Clustered Environment. I have DISTRIBUTED POLLED flag checked in the DB Poller Adapter
If I update records less than 20, the polling frequency then its working fine. If I update more than 20 then its causing issue of locks.
Thanks in Advance

Hi All,
Issue resolved by setting the <property name="NumberOfThreads" value="1"/> value to 10
Thanks

Distributed polling

Similar Messages

Maybe you are looking for