Database Adapter Polling 100 large size messages at one time.
We have a BPEL process which polls a header & detail table on a Oracle DB.
Typically we are having a scenario that we have around 1000 detail line items for each header.In the night when we start the process we have around 100 such headers each having 1000 detail lines.
We have configured the adapter in following way.Hence according to us having MaxRaiseSize="1" will create individual instace for each row that adapter fetches.is that correct?
We do get one instace per header in console.
Currently when we run the interface with 3 headers + each having 3000 details BPEL works fine.
But when i increase the load to have 50 headers + each having 3000 details , i dont see messages coming to BPEL console & also BPEL goes down & comes up on its own.
While i was monitoring it i saw that invoke_message table in dehydration store was fetching the records from messages from DB & gradually creating rows in the table & after like 30 mins it stopped we were getting internal server error on BPEL console and then it comes up on its own after some time & again rows are added in the invoke_message.
Althoug out of 100 instances that i think i should get, i see just 2 created in spam of almost 2 hrs.
My question is how can make this processing faster ? can configure something in DB adapter liek increase numberofthreads or something wich willmake the processing faster?
Any idea how BPEL works in this scenario?
Appreciate if someone can help me in this regard.
Thanks
**<binding name="DBRMSTransferPoll_binding" type="tns:DBRMSTransferPoll_ptt">*
*<pc:inbound_binding/>*
*<operation name="receive">*
*<jca:operation*
ActivationSpec="oracle.tip.adapter.db.DBActivationSpec"
DescriptorName="DBRMSTransferPoll.XxlcbTsfHdrV"
QueryName="DBRMSTransferPoll"
PollingStrategyName="SequencingPollingStrategy"
SequencingFieldName="LAST_UPDATE_DATETIME"
SequencingFieldType="java.sql.Timestamp"
SequencingTableNameFieldValue="TSF_HDR_V"
SequencingTableName="SEQUENCING_HELPER"
SequencingTableNameFieldName="TABLE_NAME"
SequencingTableValueFieldName="LAST_READ_DATE"
MaxRaiseSize="1"
MaxTransactionSize="unlimited"
PollingInterval="5400"
NumberOfThreads="1"
DelayCommit="false"
PollForChildUpdates="true"
ReturnSingleResultSet="false"
MappingsMetaDataURL="DBRMSTransferPoll_toplink_mappings.xml" />
*<input/>*
*</operation>*
*</binding>**
The issue you are facing is memory. SOA Suite is not really designed to handle large payloads because of this issue.
What I would do in the first instance is shorten the frequency so you are only selecting a small number of rows at a time. This may not have an impact because the data maybe produced via a batch program so you will always get these large volumes.
What you could do is just select the header still using the MaxRaiseSize="1" so you get just one instance per header but then you select the lines via a db adapter in the BPEL process.
The other option is to have some other mechanism that invokes the process with the full payload, e.g. setup a DB schedule that runs PL/SQL selects the data and places it on a AQ.
cheers
James
Similar Messages
-
How do you remove multiple email messages at one time
I have a iphone 4 and today it started piling up over 4000 undeliverable messages for no apparent reason. I somehow managed to stop the process but I am left with over 4000 messages that I need to delete. How do you remove multiple email messages at one time. I can "mark" all the messages at once but I cannot find the move or delete button to trash the messages. Can some one help me out.
Thanking you in advance.
PLJust figured it out - I can select multiple messages as you suggested when I am not in full screen mode.
So once I come out of full screen mode - then the multiple windows open.
QED -
How to delete my 500 messages at one time without selecting one by one?
How to delete my 500 messages at one time without selecting one by one?
Could you break this down for me more specifically? Hit menu after I've gotten into my email? I don't see delete in the menu (only the box to the left of the actual email itself). Thanks for any help!!
-
I am using Jdeveloper 10.1.3.1.0 to do the database polling. The adapter polls two tables joined by the foreign key.Then it will update the status code as logic deletion. However, when I tried to set the database rows per xml document to 1 on step 10 of 11 of the polling options. However, when I tried to set database row per transaction to 50, it won't allow me. My understanding is that this 50 will set the database commit every 50 records. The wizard basically stopped there. Only after I changed it to unlimited, it proceeded.Can anybody help me?
Hi,
The triggering instances will not depend on the Database rows/transaction but will only depend on the Database rows/xml document.
Database rows/transaction is internal to the DB and the DB adapter.
If u keep Database rows/transaction as 200 and no. of rows/ xml document as 100 and 1000 rows are to be processed, then it will create 10 instances.
However if you have 1000 rows for processing DB adapter will run a cursor in a loop to pick up 200 rows from the 1000
rows 5 times (this is a batch setting between DB and the DB adapter and will distribute the load of picking 1000 rows at a time into 5 transactions) and then these 200 rows will be distributed into 2 BPEL instances of 100 rows (this is a batch setting between DB adapter and BPEL).
Hope this clarifies.
With Regards,
Harshwardhan -
Database Adapter Polling, Instances in Running State
Hello All,
I have a composite in which the database adapter is a service (polling a table in SQL Server). Sometimes, when instances gets created, like 2 instances are being in a running state and not being completed at all. we terminated the instances last 2 times, and this scenario is being repeated again. If i click the instance which is running, i can only see the service which says it is completed, no BPEL component is being seen, can anyone faced this issue before ? why this is happening ?
Please help....
NareshCould you provide more insight on how you wired the adapter and BPEL process inside the composite? What does the BPEL look like? You could add a BPEL component that persists some stuff just for test purposes. That could give you more feedback from the console.
hth,
Peter Paul -
Oracle database Adapter polling in OSB Cluster
Hi,
I have OSB 11g with two nodes. I have a database adapter that polls a table after every 30 seconds and perform logical delete on the rows read. I came across interesting situation. When one node is up and PollingInterval=30, it works okay and polling is done right after 30 seconds but when both nodes are up the PollingInterval property doesnt work as expected.
First node will poll after 24 seconds and then second node will again poll after 6 seconds (node1+node2=30 seconds in total).
and After i restart the servers the first node is polling at 18 sec and second node is polling at 12 sec (node1 +node2=30 sec), so different polling intervals.
How does it determines the first server should poll at 24 or 18 sec and second node should poll at 6 or 12 sec?
I would like to know is there any property or configuration required to poll the table for each node for every 15 seconds (node1(15sec)+node2(15sec)=30 sec)?
Please help me, It is very urgent?
Thanks
Rajesh.Hi,
If any one has the same issue, please let me know.
Also if you know the solution, can you please update here, it's very helpful for me.
Thanks,
Rejesh. -
Jdev Database Adapter - polling updated rows
Hello, I have a question reguarding the polling strategy available in the database adapter.
I set it up and it works great with new inserted rows in the table.
However, it doesn´t capture the updated rows!
For instance, i have the following table:
ID - NAME - AGE
1 -John- 21
2 -Mary- 25
When I insert a new row, it is captured by comparing the last captured ID in the sequencing file.
ID - NAME - AGE
1 -John- 21
2 -Mary- 25
3 -Cindy- 20 <--------- New row
But when i UPDATE an already existing row, it doesnt load the changed row!
ID - NAME - AGE
1 -John- 26 <----------- Age changed, but polling doesnt recognize it!
2 -Mary- 25
3 -Cindy- 20
Is there a way to get this to work? Should I set an special option? Thank you very much.
Im using jdeveloper 11g.Hi John, it depends on which type of polling strategy you are using to poll the new/updated records. You must have the necessary previleges to add the special field and creat triggers or your db team must have.
i.Physical delete polling strategy: Cannot capture the UPDATE operations on the table.This because when the adapter listens to the table, when ever a record is polled, that record is deleted after the polling process. After polling the records, if it is not deleted, then the adapter knows that it’s a updated one but here when a record is polled, it is deleted. So when an adapter encounters a record, it’s a new record to the adapter though the record is updated before the polling cycle). So physical delete cannot capture UPDATED records.
ii. Logical delete polling strategy: The logical delete polling strategy updates a special field of the table after processing each row (updates the where clause at runtime to filter out processed rows).The status column must be marked as processed after polling and the read value must be provided while configuring the adapter. Modified where clause and post-read values are handled automatically by the db adapter.
Usage:
<operation name="receive">
<jca:operation
ActivationSpec="oracle.tip.adapter.db.DBActivationSpec"
PollingStrategyName="LogicalDeletePollingStrategy"
MarkReadField="STATUS"
MarkReadValue="PROCESSED"
This polling strategy captures the updated records only if triggers are added.This is because when the record is polled, its status is updated to 'PROCESSED'. To capture this record if its updated, its status has to be 'UNPROCESSED'. So a trigger has to be added to update the status field to 'UNPROCESSED' whenever the record is updated. Below is the example.
Ex:
CREATE OR REPLACE TRIGGER nameOftrigger_modified
BEFORE UPDATE ON table_name
REFERENCING NEW AS modifiedRow
FOR EACH ROW
BEGIN
*:modifiedRow.STATUS :='UNPROCESSED';*
END;
In this example, STATUS is the special field(of the polling table). When the record is updated, the trigger gets fired and updates the STATUS field to 'UNPROCESSED'. So when the table is polled, as this record's status is Unprocessed, this record ll be captured during polling.
Other polling strategeis like "Sequencing Table Last Updated","Sequencing Table Last-Read Id" Polling strategy,etc also can be used to capture the updated records. In these strategies also, you need to add the triggers like the above. These need an extra helping table also to poll.
Logical delete polling strategy is good enough to poll the updated records.
Hope this helps.
Thank you. -
My requirement is, when ever a new recored inserted to CREDITCARDINFOPARENT table need to insert the same record to CREDITCARDINFORESULT;
desc CREDITCARDINFOPARENT;
Name Null? Type
SNO NUMBER
SSN NOT NULL VARCHAR2(15)
FIRST_NAME VARCHAR2(30)
LAST_NAME VARCHAR2(30)
CCNUMBER NOT NULL VARCHAR2(20)
CREDITRATING NUMBER
STATUS NOT NULL VARCHAR2(20)
CCNumber is my primary key
desc CREDITCARDINFORESULT;
Name Null? Type
NO NUMBER
SN NOT NULL VARCHAR2(15)
FIRSTNAME VARCHAR2(30)
LASTNAME VARCHAR2(30)
CCNUMBER VARCHAR2(20)
CRATING NUMBER
CCSTATUS NOT NULL VARCHAR2(20)
CCNUMBER is Foreign Key.
I tried Poll DB adapter example in the following way
Created a project with "Composite with Mediator"
under exposed services I dragged DB Adapter
Service Name-> Example Service
Step 4 of 13 -> selected Poll or Changed Records in a table
step 5 of 13 -> imported 2 tables (CREDITCARDINFOPARENT, CREDITCARDINFORESULT)
step 6 of 13 -> displayed the Relationships
step 8 of 13 -> After Read -> selected Update a Sequencing Table
step 9 of 13 -> Sequencing Table: CREDITCARDINFORESULT
Sequence Name Field: CCNUMBER
Sequence Value Field: STATUS
Sequence ID Field: CCNUMBER
step 10 of 13 -> Polling Options: SQL
Pre Polling SQL:
SELECT STATUS FROM CREDITCARDINFORESULT WHERE (CCNUMBER = 'CREDITCARDINFOPARENT')
Polling SQL:
SELECT SNO, SSN, FIRST_NAME, LAST_NAME, CCNUMBER, CREDITRATING, STATUS FROM CREDITCARDINFOPARENT WHERE ((STATUS = 'VALID') AND (CCNUMBER > #STATUS)) ORDER BY CCNUMBER ASC
After Read SQL:
UPDATE CREDITCARDINFORESULT SET STATUS = #STATUS WHERE (CCNUMBER = 'CREDITCARDINFOPARENT')
step 11 of 13 ->Define Selection Criteria:
Parameters:ccno
SQL: SELECT SNO, SSN, FIRST_NAME, LAST_NAME, CCNUMBER, CREDITRATING, STATUS FROM CREDITCARDINFOPARENT WHERE ((STATUS = 'VALID') AND (CCNUMBER > #STATUS)) ORDER BY CCNUMBER ASC
Under External Services : I dragged DB Adapter and i selected Perform an Operation on a table as Insert Only. and I imported CREDITCARDINFORESULT
I deployed the the Project. Project depolyed successfully;
under EnterpriseManager command prompt I got
<Jun 29, 2010 2:25:46 PM IST> <Warning> <oracle.soa.adapter> <BEA-000000> <JCABinding=> ExampleProject JNDI lookup of 'eis/DB/soademodatabase' f
ailed due to: Unable to resolve 'eis.DB.soademodatabase'. Resolved 'eis.DB'>
<Jun 29, 2010 2:25:46 PM IST> <Error> <oracle.soa.adapter> <BEA-000000> <JCABinding=> ExampleProject Error while performing endpoint Activation:
BINDING.JCA-12561
JCA Resource Adapter location error (WebLogic).
Unable to locate the JCA Resource Adapter via .jca binding file element <connection-factory/>
The JCA Binding Component is unable to startup the Resource Adapter specified in the <connection-factory/> element: location='eis/DB/soademodata
base'.
The reason for this is most likely that either
1) the Resource Adapters RAR file has not been deployed successfully to the WebLogic J2EE Application server or
2) the JNDI <jndi-name> setting in the WebLogic JCA deployment descriptor has not been set to eis/DB/soademodatabase. In the last case you might
have to add a new 'connector-factory' entry (connection) to the deployment descriptor.
Please correct this and then restart the WebLogic Application Server
>
I confugired the Connection factory properly; I checked at DB Adapter -> Configuration -> Outbound Connection Pools -> xADataSourceName - jdbc/soadeoDatabase
when I checked at JNDI tree I dont have any value under it.
in Firefox EM, when I clicked on Example Project Test button is disabled. and I inserted one record to CREDITCARDINFOPARENT; but CREDITCARDINFORESULT table is not updated.
can any body help me in this?
Please let me know whether I follwed correct procedure or not?
If I am wrong, please provide me the correct steps.
Thanks in Advance.This error is usually raised if there is no outbound pool connections with the "eis/db/soademodata" name in the database adapter.
First create a data source to the database in the JDBC Resources.
Next in Weblogic server console --> Deployments --> DBAdapter --> Configurations --> Outbound Pool Connections --> create a new outbound pool connection with the name "eis/db/soademodata".
Now update the "xADataSourceName" property of this connection pool with the JNDI name of the datasource created in the first step.
Regards,
KSN Pavan
Edited by: KSN Pavan on Jun 30, 2010 2:51 AM -
Database Adapter - Polling updated rows
Hello, I have a question reguarding the polling strategy available in the database adapter.
I set it up and it works great with new inserted rows in the table.
However, it doesn´t capture the updated rows!
For instance, i have the following table:
ID - NAME - AGE
1 -John- 21
2 -Mary- 25
When I insert a new row, it is captured by comparing the last captured ID in the sequencing file.
ID - NAME - AGE
1 -John- 21
2 -Mary- 25
3 -Cindy- 20 <--------- New row
But when i UPDATE an already existing row, it doesnt load the changed row!
ID - NAME - AGE
1 -John- 26 <----------- Age changed, but polling doesnt recognize it!
2 -Mary- 25
3 -Cindy- 20
Is there a way to get this to work? Should I set an special option? Thank you very much.
Im using jdeveloper 11g.Hi John,
I guess you're talking about BPEL? If so, try the BPEL forum for a better response.
Best,
John -
Database Adapter Polling problem
Hi All,
The polling option is processing all the rows in a table at a time instead of using polling frequency. I have used an option of update a field (logical delete) where I have initialized unread value='N' and read value='Y'. Now ,the problem is i have to poll a single row from the table for every 60 seconds ( second row should be processed only after 60 seconds), but all the rows are processing at a same time. Can somebody let me know what could be the problem?
The options that I set in db adapter are:
Polling Frequency: 60 seconds
Database &rows per XML Document: 1
Database Rows per transaction: 1
JDBC options , Query & Timeout: 60
Thanks,Hi Vlad,
please find the jca file
<endpoint-activation portType="PollEvent_ptt" operation="receive">
<activation-spec className="oracle.tip.adapter.db.DBActivationSpec">
<property name="DescriptorName" value="PollEvent.HolxSrEventData"/>
<property name="QueryName" value="PollEventSelect"/>
<property name="MappingsMetaDataURL" value="PollEvent-or-mappings.xml"/>
<property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
<property name="MarkReadColumn" value="PROCESSED_FLAG"/>
<property name="MarkReadValue" value="Y"/>
<property name="MarkUnreadValue" value="Yes"/>
<property name="PollingInterval" value="60"/>
<property name="MaxRaiseSize" value="1"/>
<property name="MaxTransactionSize" value="1"/>
<property name="NumberOfThreads" value="1"/>
<property name="ReturnSingleResultSet" value="false"/>
<property name="SequencingColumn" value="INCIDENT_ID"/>
<property name="DelayCommit" value="true"/>
</activation-spec>
</endpoint-activation> -
Database Adapter Polling - Setting arguments in runtime
Hi!
I have a bpel-process with a DB-adapter that polls a table for new records every 10 second. I have a Where clause for the Select query, that reacts on all new records where the first argument (a field in the table) is equal to the second argument that is a literal value that will be hardcoded in the adapter.
But in the future it might be possible that the second argument might change and I want to be able to edit that without redeploying the process. Is there anyway I could set the second argument as a parameter and change it in run-time, as a property in the BPEL console for example?DB Adapter does not have possibility to change toplink mappings in runtime :-(
But maybe you can move this adapter to separate process (or even to ESB flow)which will be responsible just for receiving the data from DB and this process will call your main process. In future you will change just this small process and not the main one. Could be a workaround. -
JDBC sender adapter polling data more than once within one polling interval
Hi,
Our sender communciation channel is polling data twice in one polling interval, hence triggering the corresponding BPM twice.
The polling interval for the channel is set to 30 mins.
We dont have an update query and use <test> instead. Currently there is no provision to use an update query in the source system.
Intermitently the polling happens within an interval of few milliseconds, because of which we are in doubt whether the use of Update query will solve our problem
Additionaly in the BPM, we have one channel which deletes the data from source system after the processing is complete by the BPM, but the second BPM is getting triggered before the deletion step is being executed in the first BPM.
Please Advise how we can stop the channel from polling data twice.
Thanks,
Merrilly.Hi Merrilly,
Please ty to set an advanced mode option in the table(for sender
channel configurations) as follows:
Key name is "clusterSyncMode" and its values are either "scheduler" or
"none".(both are without quotes and case-sensitive)
Default value is "scheduler", this means it prevents 2
messages being generated regardless of 2 server nodes. This is true,
even if you do not add anything in advanced options table.
Also are more information about this parameter of file adapter, what
is the same for jdbc:
#801926 - XI 3.0 File Adapter: Additional Parameters
Regards
SK -
When sending text messags from my iphone4 it will only allow me to send to no more that 10 contacrs at a time. On my 3GS I was able to sendto well over 20 contacts at once. When I try to send on this iphone 4 all I get is the red exclamation mark and a message saying message not send and to try again. Is there something wrong with the way I have my setting? I use this feature alot so I am really trying to figure it out. Any insight would be greatly appreciated. Thanks.
I understand that the limit is 10 people when sending MMS messages. However there is a much higher limit for SMS messages.
If they would suit your needs, try disable the switch for "MMS messaging" on the Messages page of the Settings app.
Peter Johnson
Developer of TextShot Pro
I may receive some form of compensation, financial or otherwise, from my recommendation or link.
<Edited by Host> -
Sending a max of 8 SMS messages at one time?
Is it just me or is this happening to anyone else? You used to be able to send texts to much more than 8 contacts at one time. Is there an explanation for why this is happening? Please give me your thoughts and feedback.
Thanks.I think I'm seeing 10 as a max. Just assumed it was system wide.
-
Rollback in database adapter with delete polling strategy
Hi All,
We have designed a database adapter with "Delete the Rows That Were Read" after read strategy with auto-retry attempts as 5. In BPEL process, where we are receiving the DB records, we are throwing a rollback fault in case of any fault.
Database adapter polling is being re-tried 5 times in case of faults but the data is being deleted from the tables after 5 retries. Is this is the expected behavior of DB adapter? Doesn't the rollback fault rollback the complete transaction and leave the failed data back in the tables?
Can any one provide more information on this polling strategy after the number of auto-retries are completed?.
Thanks.
-PavanYou need to include your bpel process in the same DB adapter transaction
Use the following properties in the bpel component to do this
<property name="bpel.config.transaction" type="xs:string" many="false">requiresNew</property>
<property name="bpel.config.oneWayDeliveryPolicy" type="xs:string"
many="false">sync</property>
Make sure the connection factory that you are using in the DB adapter is XA transaction enabled
Maybe you are looking for
-
Programmatically closing a popup from a managed bean
Hi, in the following blog [http://thepeninsulasedge.com/frank_nimphius/2008/02/14/adf-faces-rc-programmatically-closing-a-popup-from-a-managed-bean/|http://thepeninsulasedge.com/frank_nimphius/2008/02/14/adf-faces-rc-programmatically-closing-a-popup-
-
How do i get a print spooler for office jet 6500 E709n
somehow my print spooler is gone how do i get another, i have run the install, diagnostic and utility and none fix the problem. I can see the printer in my wireless network but can't print to it This question was solved. View Solution.
-
Printer problems with Officejet Pro 8500 A910
Dear all, I have been given as a present an Officejet Pro 8500 A910 (serie number CN16LDM020 and product number CM756A). The people whom I have got it from already had some problems with the printer heads. I have accepted the printer, because I want
-
I have selected that links should open in new tabs, but they keep opening in my current tab.
-
Getting error for opening shopping cart after deleting user fron org struct
Hi Experts I am getting dump user attribute not maintained after deleting user assignment from org structure. what can i do to retain old status. or atleas wants to avoid getting this dump ..while displaying this SC. pl provide any solution Regards N