File Adapter-Problem Reading Huge Files

Hi,
Here is the issue that i am facing
When reading huge file(csv file upto 6MB-8MB) the communication channel configured as File Adapter with a polling interval of 7 min(420 sec) is inconsistent in reading the complete file.Sometimes it reads the the complete file of 6 MB and sometimes it reads a part of the file say 3MB/6MB.Can this inconsistent behaviour be resolved.??
Your suggestions highly appreciated.
Regards
Pradeep

Hi Pradeep !
8mb is not a huge file for XI, I think it is a small one. Maybe your problem is not the size..please check if XI is not starting to read the file before it is completely written to the source folder. If you are creating that csv file from another application directly to the poll source directory of the XI scenario specified in the file adapter, and your poll interval is small, XI could start reading the file while you are still writing it. If this is the case, try to put the file with a different extension or filename than the specified in file adapter comm channel and when the file is completely written, rename it to its final filename and check if you are still having that misbehavior.
You can write the file to a temp directory and the move it to the XI directory once finished.
Regards,
Matias.

Similar Messages

  • File Adapter to read Zip file and send it as input to another webservice

    Hi,
    I have the below requirement:
    1. A service will generate 3 attachments and place it in a particular directory.
    2. SOA service has to pick those 3 files and send those files as input to another custom application which will email.
    Design :
    1. First SOA will create an archive file of those 3 attachements and then file adapter will poll for that zip file in that location and send that file as a whole to the custom application.
    Query:
    Now my question, is the above design feasible? If so, how to configure the file adapter to pass the file as input to that custom application?
    Kindly do the needful
    Thanks,
    Priya

    You can accomplish this via java embedding activity...Create a java embedding, which will create a zip file.. this java code is easy to implement..
    You can also do away with un-necessary polling file adapter.. and you can use "Synchronous File Read" operation of File Adapter.. For Sync Read, you'll have to pass the zip file name, which you can easily fetch from java embedding activity..
    Let me know, if this doesn't work.

  • File Adapter Not Reading Large Files

    Dear Experts,
    Enviornment :-
    OS:-Linux
    Jdeveloper:- 11.1.1.6
    SOA:-11.1.1.6
    Weblogic:-10.3.6
    JDK:-SUN
    Allocated RAM:-16GB
    Currently we are in UAT Phase and now we are facingan  issue in reading large file.Below is the Design details of the service
    FileAdapter(Read)-->Bpel(Business login,Using FlowN)-->FileAdapter(Write CSV),JMS Adapter(AQ JMS topics)
    In this case at the time of read itself we are facing issue. File adapter reading xml file but in receive activity receives input data as
    xmlDocKey:1C135990067411E3BFA6B5087B629F9DI
    I really couldn;t understand about the error. Even i tried reading as Opaque format and still end up with same error.
    In order to make sure i have create mediator and tried reading the file, in case i was able to read file upto 15mb with out any error.Also i tried as  "read as Attachment" in Bpel component and able to read the attachment upto 7 mb file, but this is hitting the performance side.
    I request some one please let me know why the file adapter is giving XmlDOCKey rather then XMLContent to the inputvariable
    Regards,
    Tarak

    Can you check your BPEL Properties in EM?
    Go to Soa-infra > right Click > SOA Administration > BPEL properties
    increase the Dispatcher Engine Thread = 10, Dispatcher Invoke Threads = 60 and Dispatcher Engine Threads to 90
    Click on "More BPEL Configuration Properties"
    Increase the  DispatcherMaxRequestDepth from 600 to 1000.
    Bounce the server and see if works..
    Bounce the server and try again.
    If this fails, try get a threshold by increasing the file size until it fails again. 

  • Invoke File Adapter to read just once.

    Hello all,
    I want to invoke a partnerlink which is using File Adapter to poll for a file every fixed interval. I want the File adapter to read the file just once. In other words, what would be the behaviour of the File Adapter when i invoke it syn or asyn way? Would the file adapter read the file only once? Is there anyway to do this? I read in some post about it but i cannot find or recall the thread.
    Thanks for your help.

    I think I need to explain the scenario in order to give you some more idea as to why i need to poll file just once.
    I am trying to create a schedular using Quartz and If i have this BPEL process scheduled to trigger at say some time, the process would keep on polling. Whereas i just need to poll when the trigger is fired that is once per every trigger fired.
    Can anybody throw some ideas on this?

  • FILE ADAPTER NOT READING FILE.

    HI, GUYS.
    I'M USING A FILE ADAPTER TO READ A FILE THAT HAVE 4 REGISTERS AND EACH REGISTER ONLY HAS ONE FIELD THAT IS NUMBER TYPE.
    I CREATE THE FILE ADAPTER, AND I LET BPEL TO CREATE THE SCHEMA. THE IDE CALLS THES FIELD "C1". I ALSO CONFIGURE THAT THE FILE THAT MUST RAD IS THE PATTERN "*.TXT" AND EVERY 5 SECONDS.
    THEN A CREATE A RECEIVE ACTIVITY THAT RECEIVES WHAT WAS READ FROM THE FILE ADAPTER IN A VARIABLE.
    THEN I CREATE AN ASSIGN ACTIVITY AND I ASSIGN THE VALUE FROM THE RECEIVE ACTIVITY VARIABLE TO A NUMBER VARIABLE THAT IS THE INPUT OF A WEB SERVICE.
    THEN I DEPLOY THE BPEL, AND I INITIATE IT, BUT I GOT AN ERROR THAT SAYS THAT THE FIELD "C1" IS EMPTY. SO LOOKS LIKE THE FILE ADAPTER IS NOT READING THE FILE. I WOULD THANK ANY HELP THAT YOU CAN BRING ME.

    caps off
    You're sure the interface of the fileadapter (client partnerlink) is the same as your data-file which needs to be picked up?
    can you paste the content of the file and the content of the xsd of the process.

  • File Adapter and reading all XML files from direcotry

    Problem occurs on PI 7.1
    I defined sender file adapter. File name mask is: "*.xml" to read all XML messages from directory.
    Quality of service is: Exactly One.
    Poll Interval: 30
    Retry interval: 30
    Processing mode: Archive with option "Add Timestamp".
    Processing sequence: by name.
    I though that with above configuration my File Adapter will be reading folder for all coming XML files. But  somehow it is reading XMLs only when I'm activating it in Integration Builder.
    Any idea what can cause such strange problem?

    Hi Tomasz,
    As per my understanding, you need to activate the file adapter for reading the XML files on your directory. Right?
    If that is the case, then the issue might be with the Cache.
    1. Clear the cache from the Integration Builder.
    2. Check in SXI_CACHE whether there are any issues. Click on Delta Cache refresh to find out if there are any cache related issues.
    Thanks,

  • File Adapter to read from remote file server query

    Hi all,
    I want a SOA 11g application to read files on remote server using File Adapter. Is it possible to configure the File Adapter to read from remote location instead of locally. Is it possible. Is yes where can I find the associated remote file configuration documentation.
    Thanks
    Edited by: user5108636 on 19/09/2010 19:49

    I also changed one more properties the FTP connection factory called serverType=win. I get bits and pieces in each document. Is there any working example to connect to an FTP server running on Windows using Ftp Adapter. Still not working.
    These are the error logs related to it. Please suggest
    ####<Oct 12, 2010 1:17:22 PM EST> <Info> <Common> <server_name> <WLS_SOA> <[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <46e7c258a0e5cad6:-145e5cdb:12b99475c2b:-7ffd-0000000000046afa> <1286849842995> <BEA-000628> <Created "1" resources for pool "eis/Ftp/FtpAdapter", out of which "0" are available and "1" are unavailable.>\
    [2010-10-12T12:56:07.142+11:00] [WLS_SOA] [NOTIFICATION] [] [oracle.soa.adapter] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@170e13a9] [userId: weblogic] [ecid: 0000IiTsVSi9xWD_n91FiZ1CfLfc00006s,1:24709] [APP: soa-infra] [dcid: 46e7c258a0e5cad6:f9faea1:12b852b7f0f:-8000-00000000000005ed] FTP Adapter FetchAvlFiles Recovery still not possible after 1500 attempts due to javax.resource.spi.IllegalStateException: [Connector:199176]Unable to execute allocateConnection(...) on ConnectionManager. A stale Connection Factory or Connection Handle may be used. The connection pool associated with it has already been destroyed. Try to re-lookup Connection Factory eis/Ftp/FtpAdapter from JNDI and get a new Connection Handle.
    Thanks
    Edited by: user5108636 on 11/10/2010 19:05
    Edited by: user5108636 on 11/10/2010 19:22

  • How to get file name using File adapter Sync read

    Hi All,
    I am using SOA 10.1.3.3 and JDEV 10.1.3.3.
    I have an async bpel process.
    I have to read file name in this process... so i have used file adapter sync read operation.
    How can we get the file name with out payload using sync read.
    For normal read (Inbound Spec)we have UseHeaders="true" property.
    Is there any property for sync read to read the file name.
    Please help me
    Regards
    PavanKumar.M
    Edited by: [email protected] on Oct 27, 2009 11:23 PM

    Hi Eric,
    The info in the link provided by you is for a normal read.
    I need to read he file name using Sync read operation.
    Regards
    PavanKumar.M

  • Reading huge files

    I have a simple java file which reads huge files and writes them to database , I am getting out of memory error frequently , please tell me what memory arguments do I have to add to make this work ?

    Without seeing your code to produce the blob I have no idea what options you have. For all I know you could use the setBinaryStream method of PreparedStatement where you just pass it an InputStream pointing to your data. Which obviously wouldn't have to be in memory.

  • Is anybody having problems reading PDF files

    Having a problem reading PDF files. Is there a workaround?  Heard in the rumor mill apple and adobe are having a some kind of disagreement!!!

    What exactly is your problem?
    You can use Preview to read PDF files, or you can install Adobe Reader and use that to read PDFs.

  • File Adapter is not polling File.

    Hi All,
    I am using file adapter to poll a file. When I am restarting server then it is picking file. After then when I am dropping a file then File Adapter is not picking File.
    Can any one guide me what si problem?
    I am using SOA SUITE 10.1.3.4
    Thanks in Advance.

    First clusterGroupId is a unique value. Do not use the same clusterGroupId across different adapters even in different process.
    Once clusterGroupId is defined, the multiple activations of the same adapter Activation Agent is to be detected implicitly and automatically by all the instances of the adapter framework active in that cluster. The cluster will allow onlye one node activation to start the reading the files/messages. The adapter framework instances chooses one among multiple nodes, randomly to assume the Primary Activation responsibility. The other activations (instances) in the cluster will initiate to a hot stand-by state, without actually invoking EndpointActivation on the JCA resource adapter.
    If a primary activation at some point becomes unresponsive, is deactivated manually or if it crashes/exits, then any one of the remaining adapter framework members of the cluster group will immediately detect this, and reassign the primary activation responsibility to one of activation agents standing by.
    This feature uses JGroups underneath for the implementation, hence the clusterGroupId property. These features require cluster and JGroups configuration in collaxa-config.xml and jgroups-protocol.xml. All BPEL instances participating in the cluster must share the same cluster name in collaxa-config.xml,and the same multicast address and port in jgroups-protocol.xml.
    As far as for this issue, I too face the same across diff environments and searching for the exact reason. I believe this happens when primary activation agent crashes, the hand-over to the other node is not happening properly through jgroups.
    Nirav

  • File Adapter polls all the files from the directory when deployed again

    Hi,
    File Adapter polls all the files from the inbound directory irrespective of the timestamp when deployed again.
    For ex. when the BPEL process with file adapter is deployed, it starts polling the directory based on the timestamp as and when the directory is written with the files. I did not opt for deleting the files after read by the File Adapter.
    But, when BPEL process is modified and deployed again, this time, the file adapter picks up all the files in the inbound directory and doesn't consider the timestamp.
    My guess is that it should not pick all the files but based on timestamp only.
    I guess I can observe the same behavior if I bounce the Application Server also
    Do I have an option to achieve this?
    Thanks,
    Sasi Bhushan.

    try this:
    1- create a File object for the directory. (look at isDirectory() )
    2- create a File[] that represents the files in that directory. (.listFiles())
    3- if (file[x].isDirectory()) {recursive method call;}
    4- else {
    process file;
    If you want more help--do some work on it, and come back with specific questions relating to specific lines in your code and specific error messages you've been receiving.

  • Jar files required to read excel file in SAP PI 7.3.1 sp09 dualstack

    Hi experts,
    I need to read excel file (.xls) using SAP PI and process it to target system. I have read blogs
    and found that there are 2 ways to read an excel file in PI using file adapter.
    1) Developing a custom adapter module
    2) Using XSLT code.
    So in order to develop a custom adapter module, i have followed the following blogs
    **************** - XI - Step-by-step guide to develop Adapter Module to read Excel file
    and
    Excel Files - How to handle them in SAP XI/PI (The Alternatives)
    and
    http://wiki.scn.sap.com/wiki/display/ABAP/Adapter+Module+To+Read+Excel+File+with+Multiple+Rows+and+Multiple+Columns
    I am unable to find the jar files in SAP PI at OS level as per the first blog(think they were obsolete).
    Please let me know
    1) What are the required jar files needed to read excel file and their location
    2) Even if i use the old jar files as mentioned in the first blog can i achieve my requirement
    3) Following this blog Convert incoming XML to Excel or Excel XML – Part 1 - XSLT Way if i apply the same logic at sender side, will it work? Because through case studies i came to know that we cannot read a .xls file using XSLT code. Correct me if i am wrong.
    Looking for your valuable suggestions.
    Regards
    Shilpa

    Hi Shilpa
    Welcome to SCN!
    The blog you refered to might be for previous versions of PI. You can refer to the following two wikis to find out what are the relevant JAR files for PI 7.3 and also how to get them.
    XI libraries for development - Process Integration - SCN Wiki
    Where to get the libraries for XI development - Process Integration - SCN Wiki
    It also looks like for newer versions, you might not need to manually get and add those JAR files into your NWDS project - please refer to the first comment on the blog below. I have not tried it personally as I'm not using the latest NWDS, but you can try that first, and if it does not work, then go get them manually.
    PI 7.4 - Adapter Module Creation using EJB 3.0
    Do note that you should be using the JAR files that is corresponding to your PI server version.
    As for your third question, that does not apply to you. XLS is the older non-XML format, and therefore cannot be read by XLST since it is in binary format.
    Rgds
    Eng Swee

  • Sender File adapter not picking the file ABCD.PRN extension file

    Hi
    Am doing File to Abap Proxy scenario. My source will be in text format of fixed length of fileds 7.
    My souce file will be generated by a third party machine with the extension EEE150809.PRN. In my scenario my file adapter should pick the file and update the same in ECC via a Proxy.
    I have configured the Sender File adapter with Message Protocol -  Content Conversion but file adapter is not picking the file. I have checked the Communication channel and status is fine. But the same Communication Channel works for .XML file.
    What are the parameters that I have to consider with the file extension .PRN using File adapter?
    Thanks.
    S.

    Hi Swarna,
    You dont need to worry about the extension when picking the file. You can try using EEE* so that is picks all the files starting with EEE. If you have the fixed name then you can try giving EEE150809.*. Also EEE150809.PRN should not have any issue. Try to see in sender commu ication cahnnel for anyerrors. Probably there might be some network issues or authorisation issues. If you are reading with NFS then ask them to give necessary permissions. If you are reading through FTP check the user id and pwd you are using.
    Regards,
    ---Satish

  • Sender File Adapter stop processing all files

    Hello all,
    the file adapter pick up all files in the directory by default.
    if  a large number of files are in the directory then this could slow down the pi processing.
    is there any way to process only one file per polling??
    regards

    >
    Ralf Zimmerningkat wrote:
    > Hello all,
    > the file adapter pick up all files in the directory by default.
    > if  a large number of files are in the directory then this could slow down the pi processing.
    > is there any way to process only one file per polling??
    >
    > regards
    I do not have any proble if you have got the answer. BUT this blog says how to exclude the other files from the same folder.
    Your Case: For example, Your Sender CC wants to pick up file ABC.txt from /xyz dir, now suppose there are 10 thousand files of same name in the dir and you want ABC.txt should be picked up one by one, So how this blog is going to help you. Can you Plz explain to me and others too?
    @Sachin may be you can throw some light on this... may be I am missing something.

Maybe you are looking for