How to handle Big FIles in SAP PI Sender file adapter

Question

Hi all ,
I have developed a interface , where it is File to Proxy, it is fine when i do with small and normal files
The structure contain one  Header unbounded  detail and one  Trailer, how to handle when the file size is more than 40 MB
Thanking you
Sridhar

Answer 1

Hi Sridhar Gautham,
We can set a limit on the request body message length that can be accepted by the HTTP Provider Service on the Java dispatcher. The system controls this limit by inspecting the Content-Length header of the request or monitoring the chunked request body (in case chunked encoding is applied to the message). If the value of the Content-Length header exceeds the maximum request body length, then the HTTP Provider Service will reject the request with a 413 u201CRequest Entity Too Largeu201D error response. You can limit the length of the request body using the tting MaxRequestContentLength property of the HTTP Provider Service running on the Java dispatcher. By default, the maximum permitted value is 131072 KB (or 128MB).You can configure the MaxRequestContentLength property using the Visual Administrator tool. Proceed as follows:
       1.      Go to the Properties tab of the HTTP Provider Service running on the dispatcher.
       2.      Choose MaxRequestContentLength property and enter a value in the Value field. The length is specified in KB.
       3.      Choose Update to add it to the list of properties.
       4.      To apply these changes, choose  (Save Properties).
The value of the parameter MaxRequestContentLength has to be set to a high value.
The Visual administartor tool may be accessed using this link
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/40a08db9-59b6-2c10-5e8f-a4b2a9aaa3d2?quicklink=index&overridelayout=true
In short  ICM parameters to reset values for this case are
icm/HTTP/max_request_size_KB
icm/server_port_ TIMEOUT
rdisp/max_wprun_time
zttp/max_memreq_MB
Please look into this thread to know more about ICM parameters
http://help.sap.com/saphelp_nw04/helpdata/en/61/f5183a3bef2669e10000000a114084/frameset.htm
Second solution is that you must split the source file, so that each file is less than 5MB in size, then PI would not cause problem for file size between 1MB-5MB. you can insert header and trailer for individual smaller file obtained after split. All this can be done using scripts or conventional programing provided individual records within file are independent of each other. Finally you have to rename each new file created and put them in PI folder in sequential manner. All this can be achieved by simple shell script/batch file, a C code or java code. If you are going for a C or Java code you need a script to call them from PI communication channel parameter  "run operating system command before message processing".
regards
Anupam