Processing of large messages

Hi,
is anyone experienced in boosting performance of large message processing in XI? I've got messages of about 3MB size. Their processing in XI takes about 5 minutes. The respective mappings are quite simple. I have seen that I can configure a special queue for large messages but the usage of this queue doesn't help.
Kind regards,
Heiko

Hi,
3MB essage shouldnt be taking 5 mins for getting processed. IS there a lot of load on your Inegration Server?? In SXMB_MONI do you see the message in Waiting stage for a lot of time and then is it getting processed??
if so hen You can try to priotirize this particular message to be processed first
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/59e837d3-0201-0010-c096-dc1869733413
cheers,
Naveen

Similar Messages

XI getting slow after process very large message.

Hi All,
I had load data more than 50 MB after that I cannot access any function from browser(RWB,message monitoring,..) but in ABAP side that still work fine. I guess that j2ee engine still process the message that is a reason. So how I can stop the process when I load the the large files. I already try delete queue,delete workitem from SWWL,cancel the message. But nothing happen XI still very slow.
Do I have to only start/stop J2EE engine?
Could you please to advice?
Thanks
Park

HI Park Saeiam ,
in adition to Marcos
If the "browser" functions are not working it is possible that the java part of the XI just crashed and you have to restart the J2EE engine. your message may show error HTTP STATUS CODE 401 authorization error
this error generally occurs when java part of the XI crashes and server is unable to recognise your browser and your credentials.
In that case you need to restart J2EE engine and also you need to manually restart all asynchronous messages which will come in error due to restarting of J2EE engine.
I advise you not to create such huge messages.
also you may try to send this kind of huge message through IDOC package
IDocs are normally sent in packages between systems. You configure this within the IDoc configura-tion partner profile by specifying processing mode Collect IDocs and transfer. IDocs are then proc-essed by report RSEOUT00 in the sender system. During inbound processing in the IDoc adapter of the Integration Server, this IDoc package is split up into single XI messages. Therefore, an IDoc pack-age cannot be processed as one unit within XI. This also has an impact on performance.
In SP11, a solution will be available for IDoc adapter outbound processing to collect IDoc-XML mes-sages and transfer them as an IDoc package, as in the traditional IDoc world.
The IDoc adapter is designed to be able to process IDoc-XML structures that represent IDoc pack-ages.
for more help you can refer SAP XI TUNING GUIDE
Thanks
sandeep sharma
PS: if helpful Reward points

Large Message picking Wrong Queue.....???

Hi,
     We are processing a large message of size around 50 MB and its going to queue XBTO9. But i want it to be processed through XBTM.
Can someone please reply of how can it be done.
Thanks
Sourav

Its in KB...
Parameter EO_MSG_SIZE_LIMIT
Meaning
   The parameter EO_MSG_SIZE_LIMIT enables you to process messages of a
   particular size in series. This applies for messages with the quality of
   service Exactly Once (EO). If the message is larger than the parameter
   value, the message is processed in an extra queue.
   The prefix for the outbound queue of the Integration Server is XBTM, for
   all others it is XBTL.
   This parameter is designed to improve performance since there is not
   usually enough memory to process multiple large messages simultaneously.
   Serial processing ensures that the main memory only has to process one
   large message at a time, while smaller messages and processes continue
   to be processed uninterrupted.
Possible Values
   0 - 2,097,151 (KB)
   Note: 0 means deactivated (no messages are processed in an extra queue);
   the maximum value 2,097,151 corresponds to the value of roughly 2 GB.
Nilesh

Response delay in Soap Sender while processing large messages

Hi Experts,
              In one of the scenario our sender adapter is SOAP. After receiving the message its sending back the Response as HTTP 200 OK for success messges it is working fine when the message is small. When ever i am getting a large message that time its taking little bit processing time so my sender system not getting the HTTP 200 OK with in the time. (Suppose with in 3 seconds). In this case source system is sendin again the same message thinking that message is not success.
My Sender system they dont want to increase the waiting time from 3 second to .. Seconds
I cont use AAE becasue we are at 7.0.
I have to use Adapter Engine. I will not using direct communicatin with Integration Server.
     Can you please let me know how can i resolve this issue..
Thanks & Regards,
Puru

>>In this case source system is sendin again the same message thinking that message is not success.My Sender system they dont want to increase the waiting time from 3 second to .. Seconds
Looks like you are doing asynchronous SOAP scenario.   You could adjust the message reprocessing time little bit later. I meant Pi does automatic reprocess messages after sometime if the mesg gets in to error state.
<IRRELEVANT LINK>
Edited by: Prateek Raj Srivastava on Mar 12, 2011 6:24 PM

Processing large message payload with weblogic JMS

Hi,
I have a requirement in my project to process 50MB-500MG-1GB file using the JMS queue.
We could able to process 50MB payload but it's talking almost 2-3min to post the message and 2-4min to consume the message. Are there any configurable parameters available to fine tune the JMS queue to handle large payload? or is there any limitation on the size of the queue to process ?
Please advise.
Thanks,
Sri

There is a way to defer large messages from the rest and process them using a job in a specific queue.
Please check this configuration :
<a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/14/80243b4a66ae0ce10000000a11402f/frameset.htm">http://help.sap.com/saphelp_nw2004s/helpdata/en/14/80243b4a66ae0ce10000000a11402f/frameset.htm</a><a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/14/8024">Message Selection Filter</a>
in this path:
Under Runtime - > Integration engin ->
Prioritized Message Processing
Queues for Prioritized Message
Good luck
Nimrod

Problem processing large message using dbadapter.

I have a process which is initiated by dbadapter fetch from table.
Its working fine when the records are less. But when the number of records
are more than 6000(more than 4MB) I am getting errors as below.
The process goes to off state after these errors.
Any body have any suggestions on how to process large messages ?
<2006-08-02 11:55:25,172> <ERROR> <default.collaxa.cube> <BaseCubeSessionBean::logError> Error while invoking bean "cube delivery": Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
<2006-08-02 11:55:36,473> <ERROR> <default.collaxa.cube> <BaseCubeSessionBean::logError> Error while invoking bean "delivery": Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
<2006-08-02 11:55:42,689> <ERROR> <default.collaxa.cube.activation> <AdapterFramework::Inbound> [OracleDB_ptt::receive(HccIauHdrCollection)] - JCA Activation Agent was unable to perform delivery of inbound message to BPEL Process 'bpel://localhost/default/IAUProcess~1.0/' due to: Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
<2006-08-02 11:56:22,573> <ERROR> <default.collaxa.cube.activation> <AdapterFramework::Inbound>
com.oracle.bpel.client.ServerException: Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
     at com.collaxa.cube.engine.delivery.DeliveryHandler.initialPostAnyType(DeliveryHandler.java:327)
     at com.collaxa.cube.engine.delivery.DeliveryHandler.initialPost(DeliveryHandler.java:218)
     at com.collaxa.cube.engine.delivery.DeliveryHandler.post(DeliveryHandler.java:82)
     at com.collaxa.cube.ejb.impl.DeliveryBean.post(DeliveryBean.java:181)
     at IDeliveryBean_StatelessSessionBeanWrapper22.post(IDeliveryBean_StatelessSessionBeanWrapper22.java:1052)
     at com.oracle.bpel.client.delivery.DeliveryService.post(DeliveryService.java:161)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase$DeliveryServiceMonitor.send(AdapterFrameworkListenerBase.java:2358)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase.executeDeliveryServiceSend(AdapterFrameworkListenerBase.java:487)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase.deliveryServiceSend(AdapterFrameworkListenerBase.java:545)
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.performSingleActivation(AdapterFrameworkListenerImpl.java:746)
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.onMessage(AdapterFrameworkListenerImpl.java:614)
     at oracle.tip.adapter.fw.jca.messageinflow.MessageEndpointImpl.onMessage(MessageEndpointImpl.java:121)
     at oracle.tip.adapter.db.InboundWork.onMessageImpl(InboundWork.java:370)
     at oracle.tip.adapter.db.InboundWork.onMessage(InboundWork.java:332)
     at oracle.tip.adapter.db.InboundWork.transactionalUnit(InboundWork.java:301)
     at oracle.tip.adapter.db.InboundWork.runOnce(InboundWork.java:255)
     at oracle.tip.adapter.db.InboundWork.run(InboundWork.java:189)
     at oracle.tip.adapter.fw.jca.work.WorkerJob.go(WorkerJob.java:51)
     at oracle.tip.adapter.fw.common.ThreadPool.run(ThreadPool.java:267)
     at java.lang.Thread.run(Thread.java:534)
<2006-08-02 11:57:52,341> <ERROR> <default.collaxa.cube.ws> <Database Adapter::Outbound> <oracle.tip.adapter.db.InboundWork runOnce> Non retriable exception during polling of the database ORABPEL-11624 DBActivationSpec Polling Exception.
Query name: [OracleDB], Descriptor name: [IAUProcess.HccIauHdr]. Polling the database for events failed on this iteration.
If the cause is something like a database being down successful polling will resume once conditions change. Caused by javax.resource.ResourceException: ORABPEL-12509 Unable to post inbound message to BPEL business process.
The JCA Activation Agent of the Adapter Framework was unsuccessful in delivering an inbound message from the endpoint [OracleDB_ptt::receive(HccIauHdrCollection)] - due to the following reason: com.oracle.bpel.client.ServerException: Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
Please examine the log file for any reasons. Make sure the inbound XML messages sent by the Resource Adapter comply to the XML schema definition of the corresponding inbound WSDL message element.
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.onMessage(AdapterFrameworkListenerImpl.java:684)
     at oracle.tip.adapter.fw.jca.messageinflow.MessageEndpointImpl.onMessage(MessageEndpointImpl.java:121)
     at oracle.tip.adapter.db.InboundWork.onMessageImpl(InboundWork.java:370)
     at oracle.tip.adapter.db.InboundWork.onMessage(InboundWork.java:332)
     at oracle.tip.adapter.db.InboundWork.transactionalUnit(InboundWork.java:301)
     at oracle.tip.adapter.db.InboundWork.runOnce(InboundWork.java:255)
     at oracle.tip.adapter.db.InboundWork.run(InboundWork.java:189)
     at oracle.tip.adapter.fw.jca.work.WorkerJob.go(WorkerJob.java:51)
     at oracle.tip.adapter.fw.common.ThreadPool.run(ThreadPool.java:267)
     at java.lang.Thread.run(Thread.java:534)
Caused by: ORABPEL-12509
Unable to post inbound message to BPEL business process.
The JCA Activation Agent of the Adapter Framework was unsuccessful in delivering an inbound message from the endpoint [OracleDB_ptt::receive(HccIauHdrCollection)] - due to the following reason: com.oracle.bpel.client.ServerException: Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
Please examine the log file for any reasons. Make sure the inbound XML messages sent by the Resource Adapter comply to the XML schema definition of the corresponding inbound WSDL message element.
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.onMessage(AdapterFrameworkListenerImpl.java:628)
     ... 9 more
Caused by: com.oracle.bpel.client.ServerException: Delivery callback message serialization failed.
An attempt to serialize the delivery callback messages for conversation "LocalGUID:75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6d", message "75f32d7727f922f9:1712b3a:10ccf9e4cf4:-7f6c" to binary format has failed. The exception reported is:
     at com.collaxa.cube.engine.delivery.DeliveryHandler.initialPostAnyType(DeliveryHandler.java:327)
     at com.collaxa.cube.engine.delivery.DeliveryHandler.initialPost(DeliveryHandler.java:218)
     at com.collaxa.cube.engine.delivery.DeliveryHandler.post(DeliveryHandler.java:82)
     at com.collaxa.cube.ejb.impl.DeliveryBean.post(DeliveryBean.java:181)
     at IDeliveryBean_StatelessSessionBeanWrapper22.post(IDeliveryBean_StatelessSessionBeanWrapper22.java:1052)
     at com.oracle.bpel.client.delivery.DeliveryService.post(DeliveryService.java:161)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase$DeliveryServiceMonitor.send(AdapterFrameworkListenerBase.java:2358)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase.executeDeliveryServiceSend(AdapterFrameworkListenerBase.java:487)
     at oracle.tip.adapter.fw.AdapterFrameworkListenerBase.deliveryServiceSend(AdapterFrameworkListenerBase.java:545)
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.performSingleActivation(AdapterFrameworkListenerImpl.java:746)
     at oracle.tip.adapter.fw.jca.AdapterFrameworkListenerImpl.onMessage(AdapterFrameworkListenerImpl.java:614)
     ... 9 more
.

To process 6000 messages in one shot is not the best practice in BPEL. For that you have to choose concepts like datawarehouse or so.
But you might want to process it in batch mode. So think of using batch option in DB adapter and try to define MaxRaiseSize and MaxTransactionSize for your DB adapter. Further explanation is here
http://download-west.oracle.com/docs/cd/B14099_19/integrate.1012/b25307/adptr_db.htm#CHDHAIHA

Parallel processing for one large message

I have some troubles from messaging performance perspective.
Sender:ABAP Proxy
Receiver:File Adapter
I'd like use parallel processing for one large message.
And the file for receiver is needed to be one file.
Could you let me know how to set them ?
Best regards,
Koji Nagai

Hi
Can you elaborate your requirement more?
How are you trying to achieve parallel processing in XI.
Since you mentioned that the source is Proxy, there should be some trigger mechanism say selection screen, you restrict the values here and use append strategy in File and can execute the same.
REgards
Krish

Issue while processing large message using schema exposed as WCFservice

I am using biztalk 2010 and am facing one issue.
I have a schema exposed as WCF service. Used the ecf pubulishing wizard for the same.
When we post a large message using this WCF service it throws http 400 error.
We have isolated the problem to webserver.
Is there a way to update the MEXbindings to set the request message size.
If anyone has faced similar problem before request your help

Hi,
Following settings in the web.config file of the WCF service may help.
<bindings>
    <basicHttpBinding>
        <binding
name="basicHttp" allowCookies="true"
maxReceivedMessageSize="10000000"
maxBufferSize="10000000"
maxBufferPoolSize="10000000">
            <readerQuotas
maxDepth="32"
maxArrayLength="100000000"
maxStringContentLength="100000000"/>
        </binding>
    </basicHttpBinding>
</bindings>
HTH
Sumit
Sumit Verma - MCTS BizTalk 2006/2010 - Please indicate "Mark as Answer" or "Mark as Helpful" if this post has answered the question

IBaseMessage.Context getting disposed for large message processing (There was a failure executing the receive pipeline: Reason: Cannot access a disposed object. Object name: 'MessageContext'.

Hi,
I am trying to parse a large message (2.7 MB file) in a custom Flat file Disassembler (inherited from the Microsoft.BizTalk.Component.FFDasmComp) this does exactly the same work for FFDasmComp but also promotes the count of disassembled messsages. This
works fine for small messages but for large messages I get this error
There was a failure executing the receive pipeline: "<pipeline name>, <Pipeline assembly>" Source: "Custom FlatFile Disassembler" Receive Port: "ReceivePort7" URI: "<the input file location>" Reason: Cannot access a disposed object.
Object name: 'MessageContext'.
In the event log
what I am doing inside the custom flat file assembly is, I am saving all the disassembled messages in a List<IBaseMessage> and then finally when GetNext() method returns null I am promoting the nRecordCount in the messageContext of all the disassembled
messages. This is where I get the error when trying to promote to the message context.(Cannot access a disposed object, Object name: 'MessageContext'). I think the messagecontext is getting disposed, but it should not as I still hold its reference in
the List<IBaseMEssage);
private long nRecordCount = 0;
private List<IBaseMessage> oDisassembledMessages = new List<IBaseMessage>();
Any idea why this happens ?
Thanks in advance, PLEASE HELP
Surya

Hi Surya,
Are you still facing this issue? and are you using Flat File Disassembler in your Receive pipeline?
If yes then set 'RecoverableInterchangeProcessing' to True.
Please have a look in below articles-
BizTalk
Pipeline - Flat File Disassembler Cannot Access a Disposed Object 'DataReader'
biztalk-esb-toolkitflat-file-debatching
BizTalk
Server: List of Errors and Warnings, Causes, and Solutions
This issue can also occur if you are using Large Size message, in this case you will have to optimize your code.
Thanks,
Prashant
Please mark this post accordingly if it answers your query or is helpful.

How To split large message using File Adapter

Hello everyone,
Here is my scenario FTP > XI > BI.
I got 2 questions:
(1) I am getting large message around 70 MB file.
    How we can split the message into multiple files 10 MB each before processing the file into XI?
(2) Is there is any way we can find out size of file which is on FTP without contacting FTP admin?
    through XI before processing the file?
Thanks
Vick

hi vick,
check the blog
Zip or Unzip your Payload with the new PayloadZipBean module of the XI Adapter Framework
Working with the PayloadZipBean module of the XI Adapter Framework
SAP XI acting as a (huge) file mover
The specified item was not found.
Managing bulky flat messages with SAP XI (tunneling once again) - UPDATED
The specified item was not found.
regards
kummari

Best practices for handling large messages in JCAPS 5.1.3?

Hi all,
We have ran into problems while processing larges messages in JCAPS 5.1.3. Or, they are not that large really. Only 10-20 MB.
Our setup looks like this:
We retrieve flat file messages with from an FTP server. They are put onto a JMS queue and are then converted to and from different XML formats in several steps using a couple of jcds with JMS queues between them.
It seems that we can handle one message at a time but as soon as we get two of these messages simultaneously the logicalhost freezes and crashes in one of the conversion steps without any error message reported in the logicalhost log. We can't relate the crashes to a specific jcd and it seems that the memory consumption increases A LOT for the logicalhost-process while handling the messages. After restart of the server the message that are in the queues are usually converted ok. Sometimes we have however seen that some message seems to disappear. Scary stuff!
I have heard of two possible solutions to handle large messages in JCAPS so far; Splitting them into smaller chunks or streaming them. These solutions are however not an option in our setup.
We have manipulated the JVM memory settings without any improvements and we have discussed the issue with Sun's support but they have not been able to help us yet.
My questions:
* Any ideas how to handle large messages most efficiently?
* Any ideas why the crashes occur without error messages in the logs or nothing?
* Any ideas why messages sometimes disappear?
* Any other suggestions?
Thanks
/Alex

* Any ideas how to handle large messages most efficiently? --
Strictly If you want to send entire file content in JMS message then i don't have answer for this question.
Generally we use following process
After reading the file from FTP location, we just archive in local directory and send a JMS message to queue
which contains file name and file location. Most of places we never send file content in JMS message.
* Any ideas why the crashes occur without error messages in the logs or nothing?
Whenever JMSIQ manager memory size is more lgocialhosts stop processing. I will not say it is down. They
stop processing or processing might take lot of time
* Any ideas why messages sometimes disappear?
Unless persistent is enabled i believe there are high chances of loosing a message when logicalhosts
goes down. This is not the case always but we have faced similar issue when IQ manager was flooded with lot
of messages.
* Any other suggestions
If file size is more then better to stream the file to local directory from FTP location and send only the file
location in JMS message.
Hope it would help.

Large Message Payload - Queue selection

Hi,
I have one message with payload size around 105MB when I am trying to send through XI this message gets failed in the queue (inbound) of integration engine saying some memory problem to process the message. I have seen the queue it is picking up some XBT09_000 but I want to use the queue XBTL* as this queue is for large messages. The whole thing is that when my message is around 100MB I want it route large message processing queue i.e XBTL*. Is there anyway to do this? Any help on this greatly appreciated.
Thanks.,
Daniel.LA

There is a way to defer large messages from the rest and process them using a job in a specific queue.
Please check this configuration :
<a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/14/80243b4a66ae0ce10000000a11402f/frameset.htm">http://help.sap.com/saphelp_nw2004s/helpdata/en/14/80243b4a66ae0ce10000000a11402f/frameset.htm</a><a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/14/8024">Message Selection Filter</a>
in this path:
Under Runtime - > Integration engin ->
Prioritized Message Processing
Queues for Prioritized Message
Good luck
Nimrod

Large Messages fail on Integration Engine

Hello Guys!
I have an integration scenario with <b>large</b> messages going from ECC to XI (RFC -> XI -> JDBC) and these messages frequently fail on Integration Engine (inbound queue SYSFAIL).
1) Can't XI process large messages?
OR
2) Is there an integration engine configuration parameter that limits message size? (<i>I think that's not the case, because sometimes I can manually restart a message that failed on a first attemp</i>)
OR
3) Is there an integration engine configuration parameter that limits processing time of a message?
Thanks
Julio

Hi Julio,
i think you can click on the sysfail to get more info. Check as well SM21 for the error. Assumedly it was too much for Java Runtime and you should find a temporary down destination AI_RUNTIME_JCOSERVER.
SAP note 723909 shows how to tune JVM and increase Java Heap Size.
You should control if more than one msg leads to the error. If that is the case, may be a simple change of QOS to EOIO could solve the task to avoid more than one msg processing parallel. Reducing parallel JCO server threads would be similar strategy.
A direct Java Maping could be another idea. Message Mappings will be generated to Java with a quite huge source code, a direct written mapping could be more performant / stabil.
Regards,
Udo

Optimization large message PI between MQ

Hello,
How quicken/accelerate receive large message from WebShere MQ to SAP PI
and send large message from SAP PI to WebShere MQ
I am send and receive 10 mb ten messages run the process 150-170 sec = 2,5-2,8 minutes.
thanks.

Hi
Have you gone through this
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/2016a0b1-1780-2b10-97bd-be3ac62214c7
Regards
Vishnu

One large message or multipels small ones?

Hi:
I'm working in an application using weblogic 8.1.
We receive in a JMS queue a large message containing thouthands of transactions to be processed.
What should be more eficient? processing the whole transactions in the message (with one MDB) or split the large message in smaller ones and process "in paralel" with multiples MDB and somehow join all the results to return as the whole process result (hiding the client from this split/join) ?
Keeping all the execution in the same transaction would be desireable, but we could create multiple independant transactions to process each small message, but the result must be consistent in what transactions were processed successfully and what weren't.
If splitting is the option: Is there any tip on doing this?
thanks in advance.
Guillermo.

Hi Guillermo,
I would recommend reading the book
Enterprise Integration Patterns by Gregor Hohpe
http://www.eaipatterns.com/
This book describes the split and join process very good.
If the messaging overhead is little compared to total job time it would be a good idea to split
(if the tasks also run well in paralel, get quick out of the DB if you run against one DB).
Are you doing transactions against a DB?
Are you using several computers to distribute the work or a multiprocessor computer?
You can also test performance and try to split to 1, 5 or 10 transactions/msg.
When you split, keep track of total number of transations per job (= N),
and when number of successful transactions + number of failed transactions = N you are finished.
You could send the result per msg to a success or failure channel.
Regards,
Magnus Strand

Processing of large messages

Similar Messages

Maybe you are looking for