Is 1mb a large xml payload?

What size is considered a large xml payload? we are expecting something about 500k to 1mb with few concurrent instances for a given bpel process.

Hi
Where can i find reference which tells which version supporst how much size. I need to know approx figure for 11G.
Bikash
[email protected]

Similar Messages

PI process Large Xml payload?

HI experts:
in my scenario, i call a RFC and the RFC have a response ,the response payload have 10M, and the my java crushed with the error:out of memory,is there any way to expand the pi java memory?
thanks ,best regards

Hi Raja Sekhar Reddy T :
thanks for your response. my scenario is abap proxy<-> PI<->RFC. when i call the Proxy use the abap code . the RFC's response ,message payload is tremendous. so pi crashed with the error out of memory. the response message structrue is like folllwing:
<Persons>
<Person>
<personname></personname>
<***></***>
</Person>
<Person>
<personname></personname>
<***></***>
</Person>
</Persons>
the Person element is 0~unbounded. so i can split the message into small piece . but i have some doubt:
1.where can i split the message . if i split message in the mapping.because the original message is tremendous,the jvm will be crash when response message load into the jvm memory before starting message splitting.
2. if the splitting is success.how the RFC which call the abapproxy receiver the response muti-message.
thanks very much.

Reading large XML file using a file event generator and a JPD process

I am using a FileEventGenerator and a JPD Subscription process to read a large XML file. The large XML file basically contains repeated XML elements. My understanding is that the file subscription method reads the whole file in memory which causes lots of problem for huge file size like 1MB. Is there a way to read the file size-wise or is there a way to read chunks of data from a large size file..or any other alternative. I would like to process the file in a loop iteration by iteration.

Hitejain,
Here are a couple of pointers you could try. One is that the file event generator has a pass by reference (filename) functionality which you could use so that you could do the following inside of your process.
1) Read file name from the reference
2) Move the file to a processed directory (so it doesn't get picked up again. Note: I don't know how the embedded archive methods of the file event generator plays with pass by reference.
3) Open a stream to the file.
4) Use a SAX or SAX - DOM combined approach to parse your XML while managing the memory usage inside of your process
There is another possibility which might fit your needs and it is related to the RawData object that BEA provides. If I understand it correctly provides wrapping functionality around a stream object, but depending on your parsing methods might just postpone the problem.
Hope this helps
Chris Falling
Stormforge Software

In PI how do I convert the XML payload (input) to a text file and send out

I am extracting data from SAP and sending thru XI / PI to a file (non-sap system).
On the receiving end, I need the file to be a text file, with one row for each "FTZ_Row concatenated_data" tag in the input. It needs to be just a plain text file with no XML tags.
Currently the file gets one record which matches exactly to the XML payload below.
I want it to create a file that looks like this :
           CONCATENATED DATA 1
           CONCATENATED DATA 2
            012345678912345678MAKTX-TEST LB 123123123123 01/02/2009 Z001WCHA9876543210.12
here is the payload going thru XI
<?xml version="1.0" encoding="utf-8" ?>
- <n0:FTZ_Concatenated_Row xmlns:n0="http://intel.com/xi/INTEL_PROCUREMENT/ForeignTradeZone" xmlns:prx="urn:sap.com:proxy:FI0:/1SAI/TAS0C0412CDBC3E782D0219:700:2008/06/25">
<FTZ_Row concatenated_data="CONCATENATED DATA 1" />
<FTZ_Row concatenated_data="CONCATENATED DATA 2" />
<FTZ_Row concatenated_data="012345678912345678MAKTX-TEST LB 123123123123 01/02/2009 Z001WCHA9876543210.12" />
</n0:FTZ_Concatenated_Row>
thanks
jay

thanks for the input but I am still having issues - I get an empty file when I use the content conversion option.
My payload coming into XI is :
        <?xml version="1.0" encoding="utf-8" ?>
- <n0:FTZ_Concatenated_Row xmlns:n0="http://intel.com/xi/INTEL_PROCUREMENT/ForeignTradeZone" xmlns:prx="urn:sap.com:proxy:FI0:/1SAI/TAS0C0412CDBC3E782D0219:700:2008/06/25">
<FTZ_Row concatenated_data="CONCATENATED DATA 1" />
<FTZ_Row concatenated_data="CONCATENATED DATA 2" />
<FTZ_Row concatenated_data="012345678912345678MAKTX-TEST LB 123123123123 01/02/2009 Z001WCHA9876543210.12" />
</n0:FTZ_Concatenated_Row>
Here is the layout of my data type - I am using the same for input & output
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://intel.com/xi/INTEL_PROCUREMENT/ForeignTradeZone" targetNamespace="http://intel.com/xi/INTEL_PROCUREMENT/ForeignTradeZone">
   <xsd:complexType name="FTZ_Concatenated_Row">
      <xsd:sequence>
         <xsd:element name="FTZ_Row" maxOccurs="unbounded">
            <xsd:complexType>
               <xsd:attribute name="concatenated_data" type="xsd:string" />
            </xsd:complexType>
         </xsd:element>
      </xsd:sequence>
   </xsd:complexType>
</xsd:schema>
I mapped these items : FTZ_Concatenated_Row & FTZ_Row & concatenated_data
I set up my content convesion with :
recordset structure   FTZ_Row
FTZ_Row.fieldSeparator     #
FTZ_Row.endSeparator     'nl'
I am not sure why the following row looks like this ??
    <FTZ_Row concatenated_data="CONCATENATED DATA 1" />
I thought FTZ_Row would be one tag and "concatenated_data" would be another one.
any help would be appreciated.
I have read many entries/blogs/etc in SDN but and tried many different setups, but nothing is working.
thanks

"Error while parsing SOAP XML payload: no element found" received when invoking Web Service

Running PB 12.1 Build 7000. Using Easysoap. Error ""Error while parsing SOAP XML payload: no element found" received when invoking Web Service". This error does not appear to be coming from the application code. Noticed that there were some erroneous characters showing up within the header portion of the XML ("&Quot;"). Not sure where these are coming from. When I do a find within the PB code for """" it gets located within two objects, whereas they both reference a "temp_xml_letter". Not sure where or what temp_xml_letter resides???   The developer of this is no longer with us and my exposure to WSDL and Web Services is rather limited. Need to get this resolved...please.
This is the result of the search. Notice the extraneous characters ("""):
dar1main.pbl(d_as400_mq_xml)
darlettr.pbl(d_email_xml)
---------- Search: Searching Target darwin for 'temp_xml'    (9:52:41 AM)
---------- 2 Matches Found On "temp_xml":
dar1main.pbl(d_as400_mq_xml).d_as400_mq_xml: export.xml(usetemplate="temp_xml_letter" headgroups="1" includewhitespace="0" metadatatype=0 savemetadata=0 template=(comment="" encoding="UTF-8" name="temp_xml_letter" xml="<?xml version=~"1.0~" encoding=~"UTF-16LE~" standalone=~"yes~"?><EmailServiceTransaction xmlns=~"http://xml.xxnamespace.com/Utility/Email/EmailService" ~" xmlns:imc=~"http://xml.xxnamespace.com/IMC~" xmlns:xsi=~"http://www.w3.org/2001/XMLSchema-instance~" xmlns:root=~"http://xml.xxnamespace.com/RootTypes~" xmlns:email=~"http://xml.xxnamespace.com/Utility/Email~" xsi:schemaLocation=~"http://xml.xxnamespace.com/Utility/Email/EmailService http://dev.xxnamespace.com/Utility/Email/EmailService/V10-TRX-EmailService.xsd~"><EmailServiceInformation><EmailServiceDetail __pbband=~"detail~"><ApplicationIdentifier> applicationidentifier </ApplicationIdentifier><AddresseeInformation><AddresseeDetail><Number> number </Number></AddresseeDetail></AddresseeInformation><EmailMessageInformation><Ema
darlettr.pbl(d_email_xml).d_email_xml: export.xml(usetemplate="temp_xml_letter" headgroups="1" includewhitespace="0" metadatatype=0 savemetadata=0 template=(comment="" encoding="UTF-8" name="temp_xml_letter" xml="<?xml version=~"1.0~" encoding=~"UTF-16LE~" standalone=~"yes~"?><EmailServiceTransaction xmlns=~"http://xml.xxnamespace.com/Utility/Email/EmailService" ~" xmlns:imc=~"http://xml.xxnamespace.com/IMC~" xmlns:xsi=~"http://www.w3.org/2001/XMLSchema-instance~" xmlns:root=~"http://xml.xxnamespace.com/RootTypes~" xmlns:email=~"http://xml.xxnamespace.com/Utility/Email~" xsi:schemaLocation=~"http://xml.xxnamespace.com/Utility/Email/EmailService http://dev.xxnamespace.com/Utility/Email/EmailService/V10-TRX-EmailService.xsd~"><EmailServiceInformation><EmailServiceDetail __pbband=~"detail~"><ApplicationIdentifier> applicationidentifier </ApplicationIdentifier><AddresseeInformation><AddresseeDetail><Number> imcnumber </Number></AddresseeDetail></AddresseeInformation><EmailMessageInformation><Ema
---------- Done 2 Matches Found On "temp_xml":
---------- Finished Searching Target darwin for 'temp_xml'    (9:52:41 AM)

Maybe "extraneous" is an incorrect term. Apparantly, based upon the writeup within Wiki, the parser I am using does not interpret the """? How do I find which parser is being utilized and how to control it?
<<<
If the document is read by an XML parser that does not or cannot read external entities, then only the five built-in XML character entities (see above) can safely be used, although other entities may be used if they are declared in the internal DTD subset.
If the document is read by an XML parser that does read external entities, then the five built-in XML character entities can safely be used. The other 248 HTML character entities can be used as long as the XHTML DTD is accessible to the parser at the time the document is read. Other entities may also be used if they are declared in the internal DTD subset.
>>>

Is there a way to import large XML files into HANA efficiently are their any data services provided to do this?

1. Is there a way to import large XML files into HANA efficiently?
2. Will it process it node by node or the entire file at a time?
3. Are there any data services provided to do this?
This for a project use case i also have an requirement to process bulk XML files, suggest me to accomplish this task

Hi Patrick,
I am addressing the similar issue. "Getting data from huge XMLs into Hana."
Using Odata services can we handle huge data (i.e create schema/load into Hana) On-the-fly ?
In my scenario,
I get a folder of different complex XML files which are to be loaded into Hana database.
Then I gotta transform & cleanse the data.
Can I use oData services to transform and cleanse the data ?
If so, how can I create oData services dynamically ?
Any help is highly appreciated.
Thank you.
Regards,
Alekhya

SSL/TLS security certificate data match with XML Payload in SAP PI

Hi,
We are working on a solution where we would want to use SSL/TLS or WS Security with client server mutual authentication using client server certificates.
But, once the sender is authenticated using the certificates, can the XML payload be matched for the correctness with the certificate information? Is this available to PI integration engine at any time? Like Sender A autheticated as A using certificates, must be stopped if his XML payload is saying that he is sender B (which is most unlikely if we trust the senders but did not want to leave a loophole).
Any ideas here?
Thanks and Regards,
Vijay

Hi Wolfgang,
Cross-posting is discouraged and against the forum rules, because it is misused and makes a mess of the search due to distributed discussions and answers.
I will move it to the PI forum and add a watch on it as it is security forum related.
Unfortunately, the forum software does not have the option to "mirror" threads.
Cheers,
Julius
Edited by: Julius Bussche on Sep 14, 2009 9:50 PM

Reading XML payload from text file

Hello,
I have a text file with XML payload in it. The text file has some header information and then the XML payload. How do I read only the XML payload for processing in PI?
the structure of the text file is something like this..
Header Information
Payload
Sample:
#Datetime: 20140318-09:28:50.129
#MessageID: sample123
#Sender: TEST
#SenderParty: TEST
#ReceiverService:
#ReceiverParty: Test
#Interface: Test123
#InterfaceNamespace: http://test.com
<?xml version="1.0" encoding="UTF-8"?>
<Sample ><Header>......
Please advise.
Thank you.
Larry.

package com.learning.utils;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;
public class FetchPayload extends AbstractTransformation {
String strData = null;
@Override
public void transform(TransformationInput arg0, TransformationOutput arg1)
throws StreamTransformationException {
// TODO Auto-generated method stub
getTrace().addInfo("File Reading started ");
String strData = convertStreamToString(arg0.getInputPayload()
.getInputStream());
getTrace().addInfo("File Reading successfully completed ");
try {
getTrace().addInfo("Generating XML started");
strData.substring(strData.indexOf("<?xml"), strData.length());
arg1.getOutputPayload().getOutputStream().write(
strData.getBytes("UTF-8"));
getTrace().addInfo("Generating XML started");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
public String convertStreamToString(InputStream in) {
StringBuffer sb = new StringBuffer();
try {
InputStreamReader isr = new InputStreamReader(in);
Reader reader = new BufferedReader(isr);
int ch;
while ((ch = in.read()) > -1) {
sb.append((char) ch);
reader.close();
} catch (Exception exception) {
return sb.toString();
and u can get the xml as a resultant which u can use as input for next message mapping.
thanks and regards,
Praveen T

Xml payload related problem

Hi ALL
Currently I am doing a scenario in which i m encrypting the incoming xml payload(field by field) in java mapping and then decrypting it with in the adapter module.Now problem that i m facing is that after encryption if i get some special characters, the resulting xml payload becomes invalid and will not be parsed.

hi,
Try to put it into CDATA.
<script>
<b><![CDATA[</b>!@!#!@#!@#!@$@#$<b>]]></b>
</script>
regards,
Wojciech
Message was edited by: Wojciech Gasiorowski

Namespace in XML Payload for HTTP Sender

Hello,
I have just configured my snychronous HTTP <--> RFC Scenario. When I want to test it with some XML payload in the HTTP testing tool I get a mapping runtime exception.
After some investigation using the test tab in the message mapping I realized that missing namespace in the XML payload caused the error. I tried to add manually a namespace to the XML and then it worked
This is very strange I though that the namespace of the sender interface is automatically added to the XML payload?
Do you have any information on this?

Hi,
U need to copy the payload from Message mapping test message tab and use that payload.
or u can use the HTTP test client for testing the same.
sample HTML code to create HTTP Client:
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/66dadc6e-0a01-0010-9ea9-bb6d8ca48cc8
You can also use the Wfetch application by Microsift for the same..
download the same from below link
http://www.iisfaq.com/Default.aspx?tabid=2975
Thnx
Chirag

XML payload search in Xi messages

Hi All,
I need to get the XML pay load for a message which contains a specific value for a specific field. Also I need to get the XML message ID of this message. Is there any option available in Xi for this purpose?
Thanks in advance..

Hi,
sure it is but you need to use trex search engine for that
http://help.sap.com/saphelp_nw04/helpdata/en/43/6030fae92f5f87e10000000a1553f6/content.htm
once you configure trex you will
be able to search for XML payload from RWB
(runtime workbench of XI)
this is the first - fast way
there are also some non standard ways
but these are pretty slow and if you have many messages
they will not be very efficient in use:
/people/alessandro.guarneri/blog/2006/02/14/super-message-monitor-for-sap-xi
Regards,
michal
<a href="/people/michal.krawczyk2/blog/2005/06/28/xipi-faq-frequently-asked-questions"><b>XI / PI FAQ - Frequently Asked Questions</b></a>

XML Payload with attachment idoc to rnif

Hi Experts,
I am doing the scenario Idoc to RNIF. i have to pick up some1234 .pdf , fieldticket.pdf files and attached to the xml payload of out put and send it to the customers. How can we do this. please any one help me. If you have any documents please send me.

Anyone working/worked with RNIF adapter would like to share their experience.

Xml payload encoding from utf to iso

Hi Experts,
Could you please let me know how can I encode he xml payload from utf-8 to ISO-8859-1.
its bit urgent any help is appreciated.
Thanks & Regards,
Ranganath.

Hi Ranganath,
Here is the java mapping for PI 7.1 and above which will transform encoding type from utf-8 to ISO-8859-1.
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;
public class addAttributeToTag2 extends AbstractTransformation {
      * @param args
     public void execute(InputStream in, OutputStream out)
               throws StreamTransformationException {
          // TODO Auto-generated method stub
          try
               int c;
               int count=0;
               String s="";
               while(1>0)
                    c=in.read();
                    if(c<0)
                         break;
                    if(count<=2 && (char)c=='?')
                         count++;
                    if(count<=2)
                         s=s+(char)c;
                         if(count==2)
                              s=s.replaceAll("utf-8","ISO-8859-1");
                              s=s.replaceAll("UTF-8","ISO-8859-1");
                              count=3;
                              out.write(s.getBytes());
                         continue;
                    out.write(c);
                    //System.out.print((char)c);
               in.close();
               out.close();
          catch(Exception e)
     public void setParameter(Map arg0) {
          // TODO Auto-generated method stub
     public static void main(String[] args) {
          // TODO Auto-generated method stub
          try{
               addAttributeToTag2 genFormat=new addAttributeToTag2();
               FileInputStream in=new FileInputStream("C:\\Apps\\my folder\\sdn\\copy.xml");
               FileOutputStream out=new FileOutputStream("C:\\Apps\\my folder\\sdn\\copy1.xml");
               genFormat.execute(in,out);
               catch(Exception e)
               e.printStackTrace();
     public void transform(TransformationInput arg0, TransformationOutput arg1)
               throws StreamTransformationException {
          this.execute(arg0.getInputPayload().getInputStream(), arg1.getOutputPayload().getOutputStream());
if you are working in PI 7.0 the you need following code
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
import com.sap.aii.mapping.api.StreamTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
public class addAttributeToTag2 implements StreamTransformation {
      * @param args
     public void execute(InputStream in, OutputStream out)
               throws StreamTransformationException {
          // TODO Auto-generated method stub
          try
               int c;
               int count=0;
               String s="";
               while(1>0)
                    c=in.read();
                    if(c<0)
                         break;
                    if(count<=2 && (char)c=='?')
                         count++;
                    if(count<=2)
                         s=s+(char)c;
                         if(count==2)
                              s=s.replaceAll("utf-8","ISO-8859-1");
                              s=s.replaceAll("UTF-8","ISO-8859-1");
                              count=3;
                              out.write(s.getBytes());
                         continue;
                    out.write(c);
                    //System.out.print((char)c);
               in.close();
               out.close();
          catch(Exception e)
     public void setParameter(Map arg0) {
          // TODO Auto-generated method stub
     public static void main(String[] args) {
          // TODO Auto-generated method stub
          try{
               addAttributeToTag2 genFormat=new addAttributeToTag2();
               FileInputStream in=new FileInputStream("C:\\Apps\\my folder\\sdn\\copy.xml");
               FileOutputStream out=new FileOutputStream("C:\\Apps\\my folder\\sdn\\copy1.xml");
               genFormat.execute(in,out);
               catch(Exception e)
               e.printStackTrace();
However as Krish has pointedf out file adapter has option to set encoding type, you can try that option first.
regards
Anupam

XML Payload does not have namespace and prefix.

Hello
I have created a consumer business service which will be called from JDEdwards EOne, pulls data from database and send it to Fusion Middleware.
SO, I have created proxy using JAX-WS option. And suggested in oracle doc, I created proxy outside OMW and then copied it to my project. XML payload is getting generated without namespace and prefix. After some research, I modified package-info.java. Now, I am able to send the payload and if test it locally from Jdeveloper and take xml output using marshaller I can see it has namespace and prefix as well. BUt, when I run this from server it does not have namespace and prefix.
Please help.
Thanks
TK

Hi Naresh,
The "rejectedMessage" property is for 10G, I am not 100% sure about its implementation in 11G.
In 10G the faulted XML file moves to this location "Oracle_Home\bpel\domains\domain_name\jca\project_directory\rejectedMessages".
This property is used to move the files which are not valid XML or which are not schema compliant. For DB polling I don't think this property is used.
-Yatan

XML Payload extraction from XI Message

Hi to all,
I use the SXI_MONITOR transaction to see the XML Payload of XI Messages. In this transaction, each successfully message has a field "MainDocument" under "Response" section that contains the XML Payload of the message.
<B>There is a function module that I can use to extract the XML from MainDocument field?</B> I've tryed to look into SXMSCLUP and SXMSCLUR abap data tables but here I founded only the SOAP Header and SOAP Body data of the message. Here I can't see the "Payloads" data, so the MainDocument tag.
Can anyone help me?
Thanks!

Ciao Gabriele,
there are differente way to do it.
one time I got same issue and I got some answer on the community.
Unfortunately there was no more need for it so i did not implement but
easily speaking in all the mapping got the payload therefore u need just to create a full mapping adn to process the data u get.
In java for example is given as input stream that u can store some where and process.
I pasted the result below:
IN java
How to get the whole payload
in abap
/people/udo.martens/blog/2006/02/16/own-logging-of-xi-messages
please rewars point is useful thanks.
regards,
Giamma

Is 1mb a large xml payload?

Similar Messages

Maybe you are looking for