Performance Tuning in File Adapter
Hi,
I have an issue with InMemory Translation and Serialize Translation.
For Inmemory Translation, I set the UseStaging Property as false in jca.file then only its applicablle, after this and for In Memory Translation, I set the property in the composite inside the binding with this
<property name="inMemoryTranslation" type="xs:string" many="false" source=""override="may">false</property> </reference>.
my project, I am reading a file from source Directory and sending this file to target directory by setting this properties. Once I deployed and test the Instance but there is no change in the behaviour, just it written file to the target directory. And for the Serialize Translation also there is no change in the behaviour after setting Seralize properties.
Can anyone tell me, what is InMemory Translation and Serialize Translation will do and when exactly its useful,
I will appreciate any comments in order to help to fix the problem.
Thanks & Regards,
G.Satish
Hi,
Thanks for you reply,
I selected the check box "use file streaming" and I deployed to check the instance without setting any below properties
"use staging properties":-
<property name="UseStaging" value="false"/>
and "In memory translation properties":-
<property name="inMemoryTranslation" type="xs:string" many="false" source=""override="may">false</property> </reference>
by giving multiple Inputs from my source Directory to write in target directory, but in instance its showing below Errors.
faultName: {{http://schemas.oracle.com/bpel/extension}subLanguageExecutionFault} messageType: {{http://schemas.oracle.com/bpel/extension}RuntimeFaultMessage} parts: {{ summary=<summary></summary> ,code=<code>XPathExecutionError</code>} cause: {}
Fault ID default/InmemoryTranslation!1.0*soa_c02360b9-94e4-43f8-9042-1c24766df3c5/TranslationBPELProcess/931935-BpAss0-BpSeq0.3-2
Fault Time Jun 6, 2013 4:11:37 PM
Non Recoverable System Fault :
<bpelFault><faultType>0</faultType><subLanguageExecutionFault xmlns="http://schemas.oracle.com/bpel/extension"><part name="summary"><summary></summary></part><part name="code"><code>XPathExecutionError</code></part></subLanguageExecutionFault></bpelFault>
Kindly help me in this.
Thanks & Regards,
G.Satish
Similar Messages
-
Performance problems with File Adapter and XI freeze
Hi NetWeaver XI geeks,
We are deploying a XI based product and encounter some huge performance problems. Here after the scenario and the issues:
- NetWeaver XI 2004
- SAP 4.6c
- Outbound Channel
- No mapping used and only the iDocs Adapter is involved in the pipeline processing
- File Adapter
- message file size < 2Ko
We have zeroed down the problem to Idoc adapters performance.
We are using a file channel and every 15 seconds a file in a valid Idoc format is placed in a folder, Idoc adapter picks up the file from this folder and sends it to the SAP R/3 instance.
For few minutes (approx 5 mins) it works (the CPU usage is less then 20% even if processing time seems huge : <b>5sec/msg</b>) but after this time the application gets blocked and the CPU gets overloaded at 100% (2 processes disp_worker.exe at 50% each).
If we inject several files in the source folder at the same time or if we decrease the time gap (from 15 seconds to 10 seconds) between creation of 2 Idoc files , the process blocks after posting 2-3 docs to SAP R/3.
Could you point us some reasons that could provoke that behavior?
Basically looking for some help in improving performance of the Idoc adapter.
Thanks in advance for your help and regards,
AdalbertHi Bhavesh,
Thanks for your suggestions. We will test...
We wonder if the hardware is not the problem of this extremely poor performance.
Our XI server is:
Windows 2003 Server
Processors: 2x3GHZ
RAM: 4GB (the memory do not soak)
The messages are well formed iDocs = single line INVOICES.
Some posts are talking 2000 messages processed in some seconds... whereas we got 5 sec per message.
Tnanks for your help.
Adalbert -
Performance Problem with File Adapter using FTP Conection
Hi All,
I have a pool of 19 interfaces that send data from R/3 using RFC Adpater, and these interfaces generate 30 TXT files in a target Server. I'm using File Adapters as Receiver Comunication Channel. It's generating a serious perfomance problem. In File Adpater I'm using FTP Conection with Permanently Conection, Somebody knows if PERMANENTLY CONECTION is the cause of performances problem ?
These interfaces will run once a day with total of 600 messages.
We still using a Test Server with few messages.Hi Regis,
We also faced teh same porblem. Whats happening is that when the FTP session is initiated by the file adapter, then its getting done from teh XI server. Hence the memory of the server is also eaten up. Why dont you give a try by using 'per file transfer'.
If this folder to which you are connecting is within your XI server network then you can mount(or map) that drive to the XI server and use it with a NFS protocol of the file adapter and thereby increasing the performance.
Cheers
JK -
XSd-validation isn't performed when invoking file Adapter
I've configured an ESB where data needs to be transformed using a file adapter and a db adapter.
The deleimted files first need to be validated using xsd (native transformation) and if validation was performed correctly they can be transformed and loaded into db tables.
The xsd-validation isn't performed in my use case where some fields are required, have a decimal-format etc., if the files don't comply they're still picked up and being transformed to the db adapter, where the insertion will fail then because the file had errored records in it.
I'm using esb 10.1.3.3 in this case.
kind regards,
NathalieHi Nathalie,
I'm not able to define properties on my esb routing
service in release 10.1.3.3, the definition tab isn't
showing any detailed information for my routing
service.Strange, we're also using 10.1.3.3. Make sure in the ESB Control when you select the routing service, that the operation for which you want to validate the payload is selected. If it is selected, the operation details section should show the validation option. Also see page 3-14 of the ESB Developers Guide. You can also change the esbsv-file:
<operations>
<operationInfo guid="xyz" qname="Test.Insert.insert" wsdlOperation="insert" mepDisplayName="One Way" mep="OneWay">
<request validate="true" xmlns:tns="http://xmlns.oracle.com/pcbpel/adapter/d
b/top/Insert" element="tns:TestCollection"/>
</operationInfo>
</operations>
</service>
using the validate="true" option.
No logging is being generated for this matter, so I'm
not sure which problem I'm facing wright now. I will
bounce the service and try again.We had such an issue when we didn't validate payload at runtime and inserted data using DB adapter. Our TopLink mappings didn't correspond to the XML data to be inserted. That caused only 2 tables out of 4 to be updated, without any error logged. Enabling validation, resulted in an error being thrown; the behaviour we wanted.
The only thing that's of importance here, is that I
need to define this functionality within BPEL and not
ESB, the customer has only acquired bpel for this
matter because human task integration was necessary.One of the advantages of 11g fortuntely is that separation of concerns will be better. It will be much clearer where to use adapters. Not that this helps in your current case right now :-s
I should be able to accomplish the xsd validation
inside a bpel configuration.That should work, good luck!
Regards, Ronald -
Tuning the file adapter to parse subnodes
I have been looking for a solution, that how to handle Hierarchies in receiver file content conversion but everyone's suggestion is to make it as flat structure, were subnodes cant be handled by File Adapter. But in my case, I should handle the hierarchies. So I traced the File adapter behaviour and now I'm able to achieve it in my case.
My source data will be as follows,
<Root>
<Lineitem>
<no>101</no>
<name>pen</name>
<Comment>
<color>red</color>
<price>20</price>
</Comment>
<Comment>
<color>blu</color>
<price>50</price>
</Comment>
</LineItem>
</Root>
My Receiver FCC parameters are
LineItem.fieldFixedLengths = 3,3
LineItem.absoluteRowWidth = 6
Comment.fieldFixedLength = 3,2,3,2
Comment.endSeparator = 'nl'
Output:
101pen
red20
blu50
The reason I used 3,2,3,2 for Comment.fieldFixedLengths though it has only 2 fields because the parser considers the 'comment' as a field of 'lineitem' but for the 'comment' I had specified 'fieldFixedLength' parameters so it gets the fieldlength of 3rd field and omits the first two were it already passed.But for the other occurrances it going to take first two field lengths.
The absolute row width is to make the subnode to be on next line.
Though I achieved my need through this method, it has some drawbacks like
>The number of elements in parent node must be greater or equal to child node.
>Applicable only for fixedlength files.
>wont applicable if it has more then two levels
It works well, with many occurrences of both lineitem and comment.
I would like to receive the suggestions, since its not SAP recommended method.
<<<soon to be blog'd in detail>>>Hi,
Have a look at this.
/people/karthiknarayan.kesavan2/blog/2007/08/22/file-content-conversion-for-multi-hierarchical-structure
Regards,
Akshay Jamgoankar.
Reward Points if find useful. -
Hi Experts,
Many of the Messages are failed in File Adapter that is in the Module. We found it is due to heavy load.
How to improve performance of File Adapter.
How to check the Thread Count for the File Adapter.
What are the parameters i need to change to improve performance.
Regards
SankarHi
Look this blog to inhance performace of File Adapter.
XI/PI File and JDBC Receiver Adapter performance and availability improvements
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c0ed3b8c-698c-2a10-fbaf-b1df0c82fc8a
Re: Performance Problem with File Adapter using FTP Conection -
File Adapter or File Transport which one will give more performance
Hi all,
File Adapter or File Transport which one will give more performance ? in OSB?
Which one to select any one did performance analysis?
Thanks
PhaniWhy don't you just go read some benchmarks?
http://www.barefeats.com/mbpp18.html -
"Performance" problems with the File adapter on Plain J2SE Adapter Engine
Hi,
At the moment I'm on a customer side to solve some XI issues for a few days. One of the issues is the performance of the Plain J2SE Adapter Engine, using the file adapter to transfer XML messages(already XI message format) from the legacy system to the Integration Engine. The File adapter has to deal with "large" XML messages(max at the moment is 65 Mb) and the engine fails with the following error when transferring the big XML file: "ERROR: Finished sending to Integration Engine with error "java.lang.OutOfMemoryError". Skip confirmation and quit this loop".
As far I got the information from the customer the memory use of the Plain adapter engine is set to 512Mb. This is maybe to low. But I don't know where to look for this, I only have the adapter web interface in front of me, no access to the OS it self via for example remote connection.
On the Integration Engine I know there is the ability to split large message with the file adapter(File Content Conversion), but I don't know this for the Plain Adapter Engine. Is there a possibility to do this also on the Plain Adapter Engine?
Thanks in advance for any input.
Greetings,
PatrickHi Sameer,
Thanks for your answers.
On the first solution, yes that is possible, we first decided to see if the legacy system can do the splitting, before starting developing a Java program.
On the second solution, as far as I know is this solution possible on the Integration Engine. But we are facing the problems on the Plain J2SE Adapter Engine. I went trough that documentation(link:
http://help.sap.com/saphelp_nw04/helpdata/en/6f/246b3de666930fe10000000a114084/frameset.htm ), to look for I similiar solution in the Plain Adapter Engine. So my question is, is this possible with the Plain Adapter? And if so, what kind of parameters I need to use to achieve this.
Regards,
Patrick -
File Adapter - Check file exist before performing read
Is there a feature in the file adapter or invoke activity for SOA 11.1.1.5 that performs a check to see if a specific file exists before it performs a read function? If file does not exist then move onto to next step, if it does then read it.
I am trying to do this in 1 step instead of performing a listing of the directory first before perfomring the read.Hi,
When you read a file, using the synchronous read file, you have to point to a file (file name, directory...). That way, when you are executing the invoke activity, followed by a FileAdapter, it's either the file exists, and you read it, or it's not exist and you don't read it.
Arik -
How to handle large data in file adapter
We have a scenario Proxy -> PI -> File Sever using File adapter.
File adapter is using FCC for conversion.
recently we had wave 2 products live and suddenly for this interface we have increase in volume of messages, due to which File adapter is not performing well, PI goes slow or frequent disconnect from file server problem. Due to which either we will have duplicate records in file or file format created is wrong.
File size is somewhere around 4.07 GB which I also think quite high for PI to handle.
Can anybody suggest how we can handle such large data.
Regards,
VikrantCheck this Blog for Huge File Processing:
Night Mare-Processing huge files in SAP XI
However, you can take a look also to this Blog, about High Volume Messages:
Step-by-Step Guide in Processing High-Volume Messages Using PI 7.1's Message Packaging
PI Performance Tuning Best Practice:
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac62214c7?QuickLink=index&overridelayout=true&45896020746271 -
Any issues pulling files from external partners using PI File adapter?
I would like to know if there are any issues with pulling files directly from external partners using PI File adapter.
We are estimating 60 communication channels to pull files from the external vendors (25 vendors). Polling for these communication channels will be done for every 5 minutes. We have a load of around 500 files per day with peak size as 50MB and average file size as 2MB. We are currently using PI 7.0 SP12.
Did any one face any problems with respect to performance or any other issues?The volume as described would be handled with no issues.
Memory tuning and threads tuning may be required - see this guide:
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac62214c7?quicklink=index&overridelayout=true -
Date and Time in Sender File Adapter Target Directory
Hi there,
is there a way of using Date and Time from XI to the Target Directory naming? To use variable substitution some fields of the message had to contain that date and time information, which is not our case.
For filename you can use "add time stamp" option, but and for target directory?
Thanks and regards,
Henrique.> <i>Do you have further information on how to perform
> this with shell commands?</i>
> >> You need to write a shell script. This script will
> add the date/time stamp into the folder in the target
> system. This shell script is executed from the File
> adapter. So once the file is written into the target
> directory, then it will rename the directory.
Hi there, Moorthy
We've tried to write that shell script that you mentioned, but now we have a few doubts on how to make it create the proper directory.
In the file adapter, my target directory is "/%var1%/%var2%/", where %var1% and %var2% are variable substitutions, referencing data which comes from the payload. Now, I need the shell script to append "/<Year>/<Month>/" in the target directory. But where to archive the script? If it stays in the root directory, than how to make it create "/<Year>/<Month>/" folders inside a directory which is variable (/%var1%/%var2%/)?
Is there a way of passing %var1% and %var2% as parameters for the shell script?
Thanks a lot,
Henrique. -
Hi
I am using a File Adapter created to read a file using OSB 11.1.1.3.0. But after i publish that osb project, i am getting the below warning and file is not been polled by the proxy.
####<Feb 23, 2011 10:08:30 AM IST> <Warning> <JCA_FRAMEWORK_AND_ADAPTER> <wipro-e16351a68> <AdminServer> <[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <dcc2d8a908de630a:59887477:12e50cc23d3:-7ff2-0000000000000084> <1298435910015> <BEA-000000> <PollWork::run exiting, Worker thread will die>
After 600 secs i could see StuckThread error in server.
####<Feb 23, 2011 10:15:01 AM IST> <Error> <WebLogicServer> <wipro-e16351a68> <AdminServer> <[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <dcc2d8a908de630a:59887477:12e50cc23d3:-7ff2-0000000000000092> <1298436301203> <BEA-000337> <[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "601" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@dd20cd", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.lang.Object.wait(Native Method)
oracle.tip.adapter.file.inbound.FilesToProcess.dequeueToProcess(FilesToProcess.java:101)
oracle.tip.adapter.file.inbound.ProcessWork.run(ProcessWork.java:269)
weblogic.work.ContextWrap.run(ContextWrap.java:41)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Please help what is going wrong in my approach.
Thanks,
SeshaI even tired creating a new WM thread and assigned to this proxy.
I have chosen MaxThreadsConstraint option with Maximum of 5 concurrent thread.
Also i have enabled Ignore Stuck Threads in that WM thread.
Now i dont see any thread stuck in the server log, but sametime the file is not been picked by the Proxy which means the error is suppressed alone from displaying in logs..
Thanks,
Sesha -
Dynamic file name (Date) in Receiver File Adapter
Hi,
can any one tell me how to create outbound file name with receiver file adapter in format like Name_22-08-2006.dat. I need from a current Date in the name of the message (without any other information). In message payload I dont have that date. Is it possible to use variable from user defined function in message mapping for that name?
Regards,
NaychoNaycho,
that's exactly what I've managed to perform using DynamicConfiguration. Check these links:
http://help.sap.com/saphelp_nw2004s/helpdata/en/43/03612cdecc6e76e10000000a422035/frameset.htm
http://help.sap.com/saphelp_nw2004s/helpdata/en/43/09b16006526e72e10000000a422035/frameset.htm
http://help.sap.com/saphelp_nw2004s/helpdata/en/43/03fe1bdc7821ade10000000a1553f6/frameset.htm
Particularly, I've created a Java API for using setDirectory and setFileName methods through mappings. I use XSLT with java enhancement because it's a million times easier to parse and get node values with XSL than pure java.
I'm writing a blog on the subject, if you don't resolve the problem until it's out, let me know.
Regards,
Henrique. -
Adapter module to enhance File adapter
Hey guys
i need a small clarification regarding adapter modules.
is it possible via adapter modules to enhance the File adapter to such an extent that it can deliver files to two separate systems in separate landscapes.
i know i can design one more receiver determination and can also use Run OS command options,but modules in general give better performance compared to others so just wan't to explore a little bit more about them.
Thanx
AamirHey Aamir,
yeah, they do take a lot of resources, but that's how XI was built to work.
What you're trying to do is a workaround and may not get support from SAP.
If I have understood right, you still wanna maintain the endpoint connection data in the communication channels but want to acces them directly without going through the technical routing pipeline, is that so?
If that's it, I'm sorry but I cannot help you any further.
The only way I know of accessing adapters without going throught technical routing is by using lookup API (RFC, JDBC, Soap) but it is not intended for endpoint scenarios (insert, update etc), mainly for simple read (lookups).
If maintaining the connection data outside adapters is not a concern, you could try to create the connections directly in Java code, inside java proxies.
Regards,
Henrique.
Maybe you are looking for
-
I bought a itunes card and it says its invaild
i can not buy anything with my iturnes card help what to do it says invalid card/code
-
I purchased an album, and all but two songs downloaded successfully. It says files being proccessed. It has been 1.5 hours.
-
Possibility of Entering Discounts in Quick Sale Order Screen
client wants the below mentioned scenario When sales person creates the order, He should be able give the Discount Percentage. Based on that Unit price needs to be calculated and Displayed. For example: Item A with Price 100 USD at the point of creat
-
HT1222 How do I download ios702 it's come up in my setting but won't let me down load it
How do I download ios702 it's come up in my setting but won't let me down load it
-
Passing values as parameters from forms
We are trying to use the On successful submission of a form area to forward to a report based on a value entered on the form. Is there a way to use a value from the form in the url?