How to Process Files in Cluster Environment

Hi all,
We are facing the below situation, and would like to know your opinions on how to proceed.
We have a cluster environment ( server a and server b). A ESP Job is picking the files from a windows location and placing it in the unix location( server a or server b).
The problem is ESP job can place the file in only one server. This will affect the basic purpose of the cluster environment (then the file will always be processed from that particular server only).
If we place the file in both the servers, then there are chances that the same file can be processed mutilple times.
Is there a way that the load balancer can direct the file to either one of the server based on the load of the servers(just like how it does with the BPEL processes).
Or are there any other suggestions/solutions for this.
Thanks in Advance !!
Regards
Mohan

Hi,
wch version of SOA are you using? ...have a luk at this:Re: Duplicate instance created in BPEL

Similar Messages

How to process files sequentially in PI using bpm

Hi Folks,
I am really looking for one requirement like, Sender file adapter has to pick multiple files by file name based with some time gap? Can we?normally i am getting 40 files in the source directory with some time gap like 1hr and 2 hours.but my in some situations like system got down and if the server stopped for some refresh work then 2 days files will come to source directory and after system is sap pi try to process files at a time but the messages not going in order.
I have got one BPM in this, I have tried with Process mode : Name and Date, with wait step on BPM, but no use. The way how PI behaving, if 40 files in file directory, it is picking all files in one shot. Start processing but not in order. if it is process also the SNC system can't process 4 at time.it will process files with some time gap.
The problem is on Receiver system side. The receiver system is SNC system, if old data receives later than earlier date data; we get data obsolete application error.
Ex: If I receive 25th and 26th files, first I need to process 25th first on PI sends to SNC, I need to give some time gap and pick another file or even PI picks and process 26th file, no problem but I need to give some time gap to send SNC to this 26th date file?
Please how guys, throw me your great ideas
Step1: i configured the sender file adapter with by name property to sort the files but some times pi picking new date file first and old one later.here my question is how to configure adapter to pick files in sorted by name.the filename i given like xml_0809008998_*.xml
step2: after pi picking the files in order but the messages not sending order to target system.i was configure the bpm like first recive step then transformation step then i was used wait step to process files with time gap.after that block step mode is default inside i was used 2 bblock steps.
here my question how to configure bpm process messages in order?
Thanks in advance!!
Regards
SG

Hi,
In the sender file comm channel use Processing Sequence = By Date. After that use Quality of service as EOIO and provide one queue name. Use same queue name in the receiver comm channel as well. So files will be picked by the date of the file and messages will be passed to SNC system in "first in first out" basis.
Reagrds,
Nayan

OSB 10.3g file proxy and how to process files sequentially

Hi all,
I've a folder that contains some files that I need to process sequentially by a file proxy:
/tmp/osb/msg > ls -l
-rw-r--r-- 1 weblogic weblogic 904 Jul 22 12:02 yyy_C0001279792949924_20100722_120229_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 917 Jul 22 12:02 yyy_C0001279792949955_20100722_120229_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 932 Jul 22 12:02 yyy_C0001279792949986_20100722_120229_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 858 Jul 22 12:02 yyy_C0001279792950017_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 890 Jul 22 12:02 yyy_C0001279792950033_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 790 Jul 22 12:02 yyy_C0001279792950064_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 844 Jul 22 12:02 yyy_C0001279792950096_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 937 Jul 22 12:02 yyy_C0001279792950111_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 813 Jul 22 12:02 yyy_C0001279792950142_20100722_120230_BU.yyy.CH_ASS_xxx.xml
-rw-r--r-- 1 weblogic weblogic 963 Jul 22 12:02 yyy_C0001279792950158_20100722_120230_BU.yyy.CH_ASS_xxx.xml
this is the list of the files sorted alphabetically but also is the list sorted by creation time (each filename (after yyy_C) contains a timestamp and also a date).
The file proxy is configured as belows:
Service Type : Any XML Service
Protocol : file
Endpoint URI : file:///tmp/osb/msg
Get All Headers : No
File Mask : yyy*.xml
Polling Interval: 30
Read Limit : 0
Sort By Arrival : true
Scan SubDirectories: false
Pass By Reference : false
Post Read Action : archive
Stage Directory : /tmp/osb/stage
Archive Directory : /tmp/osb/archive
Error Directory : /tmp/osb/error
Request encoding : utf-8
Content Streaming : Disabled
I used "Sort By Arrival : true" because from doc (http://download.oracle.com/docs/cd/E13159_01/osb/docs10gr3/httppollertransport/index.html) I read:
"This parameter indicates the sequence of events raised in the order of the arrival of files. The default value for this parameter is False"
It is not clear if 'order of the arrival' means by time of the arrival or by 'alphabetical order'.
I did a test using the above settings but I see from OSB log file that the files are processed neither by time order nor by alphabetical order:
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '4' ... <BEA1-07857BB52A53B9043886> <> <1279794804450> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792949955_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '0' ... <BEA1-078F7BB52A53B9043886> <> <1279794804454> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792949924_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '1' ... <BEA1-07907BB52A53B9043886> <> <1279794804457> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792949986_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '0' ... <BEA1-07917BB52A53B9043886> <> <1279794804462> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950064_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '1' ... <BEA1-07927BB52A53B9043886> <> <1279794804467> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950017_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '0' ... <BEA1-07937BB52A53B9043886> <> <1279794804607> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950142_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '4' ... <BEA1-07947BB52A53B9043886> <> <1279794804611> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950111_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '4' ... <BEA1-07957BB52A53B9043886> <> <1279794804615> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950096_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '4' ... <BEA1-07967BB52A53B9043886> <> <1279794804618> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950033_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 12:33:24 PM CEST> <... ExecuteThread: '1' ... <BEA1-07977BB52A53B9043886> <> <1279794804630> ...########## ready to process filename: .../tmp/osb/msg/yyy_C0001279792950158_20100722_120230_BU.yyy.CH_ASS_xxx.xml
How could I process files sequentially ?
I think using also Read Limit = 1 and Polling Interval = 1 sec doesn't work, because
- if the proxy instance P1 reading file A takes more than 1 sec
- after 1 sec a second proxy instance P2 starts to process file B
- but P2 can completes its processing before P1 ...
What do you suggest to do ?
Thanks in advance
ferp

Hi Anuj,
even trying with
- Read Limit 1
- Polling Interval 1
- Sort By Arrival true
I see from OSB log file that the files are processed neither by time order nor by alphabetical order:
####<Jul 22, 2010 2:35:56 PM CEST> ... ExecuteThread: '0' ...<BEA1-07F27BB52A53B9043886> <> <1279802156519> .../tmp/osb/msg/yyy_C0001279792949955_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:35:58 PM CEST> ... ExecuteThread: '0' ...<BEA1-07F47BB52A53B9043886> <> <1279802158508> .../tmp/osb/msg/yyy_C0001279792949924_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:00 PM CEST> ... ExecuteThread: '1' ...<BEA1-07F67BB52A53B9043886> <> <1279802160525> .../tmp/osb/msg/yyy_C0001279792949986_20100722_120229_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:02 PM CEST> ... ExecuteThread: '4' ...<BEA1-07F87BB52A53B9043886> <> <1279802162544> .../tmp/osb/msg/yyy_C0001279792950064_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:04 PM CEST> ... ExecuteThread: '0' ...<BEA1-07FA7BB52A53B9043886> <> <1279802164564> .../tmp/osb/msg/yyy_C0001279792950017_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:06 PM CEST> ... ExecuteThread: '1' ...<BEA1-07FC7BB52A53B9043886> <> <1279802166584> .../tmp/osb/msg/yyy_C0001279792950142_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:08 PM CEST> ... ExecuteThread: '0' ...<BEA1-07FE7BB52A53B9043886> <> <1279802168606> .../tmp/osb/msg/yyy_C0001279792950111_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:10 PM CEST> ... ExecuteThread: '4' ...<BEA1-08007BB52A53B9043886> <> <1279802170624> .../tmp/osb/msg/yyy_C0001279792950096_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:12 PM CEST> ... ExecuteThread: '0' ...<BEA1-08027BB52A53B9043886> <> <1279802172644> .../tmp/osb/msg/yyy_C0001279792950033_20100722_120230_BU.yyy.CH_ASS_xxx.xml
####<Jul 22, 2010 2:36:14 PM CEST> ... ExecuteThread: '4' ...<BEA1-08047BB52A53B9043886> <> <1279802174664> .../tmp/osb/msg/yyy_C0001279792950158_20100722_120230_BU.yyy.CH_ASS_xxx.xml
As said I think that using Read Limit = 1 and Polling Interval = 1 sec doesn't work, because
- if the proxy instance P1 reading file A takes more than 1 sec for processing the file
- after 1 sec a second proxy instance P2 starts to process file B
- but P2 can completes its processing before P1 ...
Do you know if OSB when polling a file, takes it only when OSB recognizes the file has been written completely ?
I mean what happens if OSB wakes up while the file producer is still writing the file ?
Regards
ferp

How to set property for Cluster Environment for JMS Adapter

Hi All,
I am moving from DEV to Prod environment which is cluster.
Can you Please explain me what property I need to Set for Cluster Environment for JMS Adapter, so that I could avoid race condition for Dequeue/enqueue.
I am using soa suite 10.1..3.4
Thanks in Advance.
Edited by: vikky123 on Jul 12, 2010 7:03 AM

put something like this
<activationAgents>
      <activationAgent className="oracle.tip.adapter.fw.agent.jca.JCAActivationAgent" partnerLink="PARNERLINKNAME">
        <property name="clusterGroupId">SOMEUNIQUEVALUE</property>
        <property name="portType">PARTNERLINK_PORTTYPE</property>
      </activationAgent>
    </activationAgents>

How to process files one by one using sender file adapter

Hi,
I have to process file one by one using my file adapter (sender), because while doing the G/R materials get locked, if the same materal exist in other files.
I have maintained QOS as "Exactly Once in Order" & processing Sequence as "by date".
Processing sequence come in which tab? What will be the Queue name?
Thanks,
krishna

It will come under Processing tab
Here you need to select the first parameter QOS as EOIO
Then QUEUE name and processing sequence will be enabled.
NOTE: This will work only for NFS protocol not for FTP.

How to upload files in cluster without bringing down the cluster.

Hi,
Here is my problem:
My weblogic portal 7.0 application should run in 24x7 environment. The production
system is running in a cluster mode with external staging. Both the managed servers
are pointing to the same application sync database.
So to give a solution to this problem, we are trying to bring down one managed
server down and apply the patches into that managed server without bringing down
the other. Since webflow files and entitlements needs to be synchronized with
the portal database. If we perform a sync on the portal database, there might
be a chance that it might corrupt the existing portal database state. Our deployment
would not be complete unless we perform a sync on the other managed servers. So
we could not able to update the latest pathces into the cluster without bringing
down the cluster.
What I would like to know is "Is there any work around to get rid of this problem?".
I would be delighted if any one provides the solution.
Thanks & Regards,
Ratnakar

Hi Ratnakar,
When you perform sync you do it against the admin server which in turn does it
to the managed server. If you want to apply any changes to webflow you can do
it on the live system itself. I mean apply the new webflow and entitlements and
do the sync on the live system. Also while doing sync there are two modes. One
mode will sync only the modified changes and the other will apply all of it. You
can opt for the first option by which you can sync only the modified changes.
This may bring a small disturbance but it will be unnoticed.
The other option is to have two separate domains(each having one admin & one or
more managed servers) which may be having two separate portal databases. In this
way you can apply patches to one domain while other domain may provide the service.
Regards
Shaan
"Ratnakar Sonti" <[email protected]> wrote:
>
Hi,
Here is my problem:
My weblogic portal 7.0 application should run in 24x7 environment. The
production
system is running in a cluster mode with external staging. Both the managed
servers
are pointing to the same application sync database.
So to give a solution to this problem, we are trying to bring down one
managed
server down and apply the patches into that managed server without bringing
down
the other. Since webflow files and entitlements needs to be synchronized
with
the portal database. If we perform a sync on the portal database, there
might
be a chance that it might corrupt the existing portal database state.
Our deployment
would not be complete unless we perform a sync on the other managed servers.
So
we could not able to update the latest pathces into the cluster without
bringing
down the cluster.
What I would like to know is "Is there any work around to get rid of
this problem?".
I would be delighted if any one provides the solution.
Thanks & Regards,
Ratnakar

How to process files on the fly instead of uploading them to server ?

Hello,
We are using upload package of oreilly. is there a way to read files on the fly ? We don't want to upload files to server, open them and process them. But processing them on the fly will be faster for us.
thanks in advance...

Hello,
If we can upload a file to server by using oreilly's package, there should be a way to read the file after it's POSTED by client's browsers..
If we can do that, we'll not have to do followings:
- Saving file to our server
- Openning this file
- Reading this file sequentially
We want to read file on the fly. isn't it possible ? If not possible, how can oreilly package save files to a directory on our server.
thanks in advance...

Polling file on cluster environment in weblogic

Hi All,
i am getting one issue while reading a file and deleting at end.
while running it on one server it is working fine but while running on cluster it read same file more than nce.
how could we restrict that one file is pollled by only one server.
Thanks &Regardsa
Ajay Pratap

Can you try to add this property in your adapter and see if it works -
<binding.jca config="file_adapter.jca">
<property name="singleton">true</property>
</binding.jca>

How to process files sequentially ?

Hi Folks
i have a requirement in which i have three files placed on the server in random order and i have to bring them and process sequentially in the order a-b-c i cannot use processing sequence due to nature of filenames
my source structure is same for all the files and for target proxy i need data in the same order a-b-c
please provide any solution to this .. Can we do it without BPM.. and IF BPM how ?

Hi Ninad,
According to my knowledge its not possible because we don't know when the files come to the server.
Based on the poll interval the XI/PI picks the file, In that interval suppose b,c,d,a,... files come to the server randomly at that time you need to pick the file in a,b,c,d,... order then you have to write the user defined module.
Check this links for module development
http://help.sap.com/saphelp_nw04/helpdata/en/96/f04142099eb76be10000000a155106/frameset.htm
http://help.sap.com/saphelp_nw04/helpdata/en/cd/5af7c0c994e24fb0d0088443513de2/content.htm
Regards
Ramesh

How to process file with fixed length using MFL in OSB

Hi Team,
I need to translate this file using MFL. its fixed lenght[each line is having 81 chars] file in below format.
Please help me to process this in OSB using MFL.
==================================================================
START
[email protected]
[email protected]
END
Subject : CALIFORNIA STEEL IND Renege #00424523 Hse580 CTH580
BODY:
User_ID: LARRY014
Xpd2 Hse/Customer # : 580/1196310
Xpd Order Number: 580094624
Customer E-Mail: [email protected]
Customer E-Mail 2:[email protected]
Customer Phone : 9093506458
Dear Salesperson,
gexprosupply.com Order # : 00424523
gexprosupply.com User ID : LARRY014
Customer CALIFORNIA STEEL IND has entered order 00424523
through gexprosupply.com.
THIS ORDER HAS RENEGED for the following reason(S):
I. ORDER LEVEL
NOTE SEGMENTS FOUND IN INPUT - SENTRY
CDF REQUIRED CUSTOMER - ORDER RENEGED
II. ITEM/LINE LEVEL
LINE # ECOM LINE NAED QTY STATUS ALLOW SUBS
Please resolve the renege and release the order in Sentry
01 as soon as possible. Thank you.
EMAIL-END
==================================================================
Thanks
Reddy
Edited by: 913248 on Feb 8, 2012 2:07 PM
Edited by: 913248 on Feb 8, 2012 2:08 PM

Hi Prabu,
thanks a lor fro you replay.......
the above file is having mail content[from id, to id, subject and body] i need to read this file using OSB and need to sent to the respective email id.
the file format is :
it will have
0- its a fixed length in each line [81 chars]
1--"START" tag....starting of the email
2--after START, the email ID is "FROM" email ID
3-after "FROM" emailid, 3rd line onwards....will have "TO email IDs
4-"END"---end of the email IDs
5- "Subject " ...will have subject of the email
6-"BODY".... the email body will start in this tag
7-"EMAIL-END"..end of the email....
so my requirement is ..i need to read above file in OSB and send notification emai.
Thanks
Reddy

Question about how XI process files in FTP

Hi experts,
I have a file-FTP to JDBC scenario.
XI scans a folder every minute to see if any file has appear.
My question is, does XI process the files in their order of creation, or randomly ?
If not, is there a way to make XI process them in order of creation ?
Thanks a lot, any help would be appreciated.
Kind regards,
Jamal

Hi,
from SAP Help:
If you select files from an FTP server and not from the file system, you need to make the following additional specifications:
○       ftp.host=<ftp-server>
The host name or IP address of the FTP server. If this specification is made, it is assumed that you are accessing an FTP server. The specifications file.sourceDir and file.sourceFileName then refer to the FTP server. The following are then not possible specifications:
■       setAttribute for parameter file.processingMode.
■       byDate for parameter file.processingOrder.
Regards
Patrick

How to setup the cluster environment for BPM using weblogic

want to setup the cluster environment for BPM using weblogic....
i have installed the oracle weblogic server 10gr3 and oracle BPM enterprise for weblogic 10gR3
i have used the Admin tools from the "oracle BPM enterprise for weblogic" to setup the configuration and create the weblogic domain servers.
i can launch the process administrator and import the project exp file to domain server.
but what should i do to setup cluster environment using weblogic?
what i want to do is :
setup one admin machine..
setup two product machine..
enable the cluster so the admin machine can monitor the status of the product machine..
thanks a lot ...

The install guide at http://download-llnw.oracle.com/docs/cd/E13154_01/bpm/docs65/config_guide/index.html gives a reasonable amount of info on how to do this.
Personally I have not used the OBPM option to configure WebLogic instead I've used the information in the above install guide to create the weblogic domain in advance of configuring OBPM.
Once you've setup WebLogic configure OBPM using the values I mention in the following thread: How to set the JMX Engine parameter in Process Administation?
Let me know any specific config questions and I'll do my best to answer them for you.
Thanks,
Mike

Deploying Java Web Application (WAR-File) into a cluster environment

Hi,
we have a web application which has to read from and write to the file system.
Since a short time we have a cluster environment (2 parallel servers) and since thisa time we have the problem, that files are worked double in the cluster. The application is working on both servers now and so we have this problem.
Does anybody know how we have to deploy the application correctly in a cluster environment or do we have to change anything in our source code of the application?
I didn't find any documentation about this.
At the moment we have deployed the application on one of the two servers only, but I think there must be a better way to solve this problem.
Thanks for your replies.
Regards
Thorsten

Hi,
I think first you need to wrap it into an EAR file, then you can deploy it.
As far as I know standalone deployment of WAR is deprecated as of 640.
similar threads:
How to deploy .war on NWDI
Deploying an existing WAR file into the Portal
Hopefully this tutorial also gives some idea:
http://help.sap.com/saphelp_nw70ehp1/helpdata/en/70/13353094af154a91cbe982d7dd0118/frameset.htm
Regards,
Ervin

How to display XML generated dynamically, as TREE in cluster environment

hai guys,
we are generating a tree.xml file in server side as follows.
path = getServletConfig().getServletContext().getRealPath("/QBE/jsp/tree.xml");
BufferedWriter out1 = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(path),"UTF8"));
out1.write(xmlString + "</tree>");
out1.close();
"xmlString" holds the data in xml tag format.
in order to display the xml file in TREE structure, we are using a function like
function createTree()
var tree = new WebFXLoadTree("SMQ/AMQ List", "tree.xml");
document.write(tree);
ALL THIS MECHANISM IS WORKING FINE IN DEVELOPMENT ENVIRONMENT ANS AS WELL AS WHEN WE DEPLOY THE RELEASE IN LOCAL MACHINE. BUT IT IS NOT WORKING WHEN THE RELEASE IS DEPLOYED IN CLUSTER ENVIRONMENT.
Please help me out how to solve this...
thanks in advance,
Ranga
Edited by: Ranganatha on Jun 5, 2008 5:18 AM

Hey if you want any more information regarding this problem, i can provide.

File polling in cluster environment

We have a cluster environment with two servers(say, osb_server1 and osb_server2) and a proxy service is configured with file polling using HAfileAdapter.
When executing this service immediately after activation, for first time, the server “osb_server2” throws error as,
Caused By: BINDING.JCA-11042
File deletion failed.
File deletion failed.
File : /**Path**/filename.txt as it does not exist. could not be deleted.
But for consecutive attempts, the osb proxy service is processed by “osb1_server1” and we are not getting the above mentioned error.
In short, We face the issue as below only when executing first time after “Activate” session but the consecutive attempts doesn’t throw error.
Why such error occurs when executing in cluster environment?
Thanks in advance.

try to use the file protocol provided by osb.And select the polling server as on eof the managed server.

How to Process Files in Cluster Environment

Similar Messages

Maybe you are looking for