OSB 11g - Maximum file size OSB can handle

Question

Hi,
What is the maximum size (of a file) which OSB can handle, when then file is polled using FTP, and then use XQuery transformation on it?
Thanks,
Sanjay

Answer 1

Processing large files in OSB will be limited by the constraints on both Heap Memory and CPU utilization.
To load larger objects in memory you will need enough Heap Size. If your heap size is not enough you will start getting Out of memory errors and weblogic server might crash. You can either add more memory to the server(which itself has a limit) or you can use content streaming. When you use content streaming the whole object will not be loaded into memory buffer and you will be able to consume large messages. But content streaming has certain limitations because the whole object is not loaded in memory. You can find these limitations at following link and decide if you can implement your use case with these limitations.
http://download.oracle.com/docs/cd/E13159_01/osb/docs10gr3/userguide/context.html#wp1110513
If not then the best possible option would be to break the file into smaller chunks and then using OSB to read and process those chunks.
Another limitation is the CPU utilization. If you are not transforming the large content CPU should not be an issue. But if you need to transform before passing the payload to backend system then it becomes complicated. Transformations (both XQuery and XSLT ) take a lot of CPU cycles and if you are transforming huge payloads it will consume 100% CPU resulting in overall deterioration of the performance. Like Anuj mentioned, it is not recommended to have complex transformations and transformations of large XMLs within OSB. Its best to move out such transformations to other systems. But, it is also not recommended that you move many transformations to various backend systems. If your architecture involved many flows which have transformations on large objects, you should think about spending on an XML Appliance meant for transformations. These products are purpose built to focus on doing complex XMLXML processing with little impact on the overall CPU utilization. They use hardware acceleration to achieve this. You can contact Oracle Sales team or Oracle support to get more information about this.
Regards
Abhi