Issue while Processing the Huge File in BPEL

Question

Hi,
We are facing an Issue while Processing a Hige file in BPEL Process (More than 1MB File). When i test the files with more than 1500 transactions (More than 1MB file) then the BPEL Process is automatically goes to OFF Mode or it goes to Perform Manually Recovery Queue.
Even we are facing this issue in Production also so we are using UNIX Script to Split the file before place the file in BPEL Input directory.Any Pointers to resolve this issue will be helpful.
Thanks,
Saravana

Answer 1

Hi,
Please find the answers.
1. Currently we are using SOA 10.1.2 Version and JDev10g
2. we are using File Adapeter
3. yes. We used debatching.
4. Yes. I am able to recover from Manual Recovery Queue
5. Please find the error message
<2009-05-21 04:32:38,461> <DEBUG> <ESIBT.collaxa.cube.engine.dispatch> <Dispatcher::adjustThreadPool> Allocating 1 thread(s); pending threads: 1, active threads: 0, total: 83
<2009-05-21 04:32:44,077> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> File : /harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B is ready to be processed.
<2009-05-21 04:32:44,077> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Processing file : /harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B
<2009-05-21 04:32:44,077> <DEBUG> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> onBatchBegin: Batch 'bpel://localhost/ESIBT/BPELProcess_810~1.0//Input5162009.B2B_1242894594000' (/harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B) starting...
<2009-05-21 04:32:44,077> <DEBUG> <ESIBT.collaxa.cube.translation> <TranslatorFactory::log> Inside TranslatorFactory
<2009-05-21 04:32:44,078> <DEBUG> <ESIBT.collaxa.cube.translation> <TranslatorFactory::log> using version attribute = NXSD
<2009-05-21 04:32:44,078> <DEBUG> <ESIBT.collaxa.cube.translation> <TranslatorFactory::log> loading xlator class...oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl
<2009-05-21 04:32:44,081> <DEBUG> <ESIBT.collaxa.cube.translation> <TranslatorFactory::log> class loaded
<2009-05-21 04:32:44,081> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Created translator : oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl@46908ae8
<2009-05-21 04:32:44,098> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Setting up Control dir for debatching error recovery
<2009-05-21 04:32:44,121> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Control dir for debatching error recovery : /opt01/app/ESIBT/oracle/esibt10gR2iAS/BPEL10gR2/iAS/j2ee/home/fileftp/controlFiles/localhost_ESIBT_BPELProcess_810~1.0_/inbound
<2009-05-21 04:32:44,121> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Invoking inbound translation for : Input5162009.B2B
<2009-05-21 04:32:44,121> <DEBUG> <ESIBT.collaxa.cube.translation> <NXSDTranslatorImpl::log> Starting translateFromNative
<2009-05-21 04:32:44,139> <DEBUG> <ESIBT.collaxa.cube.translation> <NXSDTranslatorImpl::log> Done with translateFromNative
<2009-05-21 04:32:44,139> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Completed inbound translation for : Input5162009.B2B
<2009-05-21 04:32:44,139> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> isTextFile : true
<2009-05-21 04:32:44,139> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Translated inbound batch index 1 of file {Input5162009.B2B} with corrupted message count = 1
<2009-05-21 04:32:44,139> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Error Reader created using charset :ASCII
<2009-05-21 04:32:44,139> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Sending message to Adapter Framework for rejection to user-configured rejection handlers : {
fileName=/harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B, startLine=1, startColumn=1, endLine=-1, endCol=-1, Exception=ORABPEL-11167
Error while reading native data.
[Line=1, Col=70] Expected "\t" at the specified position in the native data, while trying to read the data for "element with name HDR_STORE_NUM", using "style" as "terminated" and "terminatedBy" as "\t", but not found.
Ensure that "\t", exists at the specified position in the native data.
<2009-05-21 04:32:44,139> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Setting batchId in NativeRecord to bpel://localhost/ESIBT/BPELProcess_810~1.0//Input5162009.B2B_1242894594000
<2009-05-21 04:32:44,139> <WARN> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> [Read_ptt::Read(Object)] - onReject: The resource adapter 'File Adapter' requested handling of a malformed inbound message. However, the following bpel.xml activation property has not been defined: 'rejectedMessageHandlers'. Please define it and redeploy the business process. Will use the default Rejection Directory file:///opt01/app/ESIBT/oracle/esibt10gR2iAS/BPEL10gR2/iAS/integration/orabpel/domains/ESIBT/archive/jca/BPELProcess_810/rejectedMessages for now.
<2009-05-21 04:32:44,140> <WARN> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> [Read_ptt::Read(Object)] - onReject: Sending invalid inbound message to Exception Handler:
<2009-05-21 04:32:44,140> <INFO> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> Handing rejected message to DEFAULT rejection handler: file:///opt01/app/ESIBT/oracle/esibt10gR2iAS/BPEL10gR2/iAS/integration/orabpel/domains/ESIBT/archive/jca/BPELProcess_810/rejectedMessages since none of the configured rejection handlers [] succeeded.
<2009-05-21 04:32:44,140> <DEBUG> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> Finished persisting rejected message to file system under the name: /opt01/app/ESIBT/oracle/esibt10gR2iAS/BPEL10gR2/iAS/integration/orabpel/domains/ESIBT/archive/jca/BPELProcess_810/rejectedMessages/INVALID_MSG_BPELProcess_810_Read_20090521_043244_0140.dat
<2009-05-21 04:32:44,141> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Setting last error record to : -1
<2009-05-21 04:32:44,141> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Translator has failed to translate any message from batch number: 1
<2009-05-21 04:32:44,141> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Message not published as translation failed: {
File=/harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B, batchIndex=1, PublishSize=1
<2009-05-21 04:32:44,141> <ERROR> <ESIBT.collaxa.cube.activation> <AdapterFramework::Inbound> onBatchFailure: Batch 'bpel://localhost/ESIBT/BPELProcess_810~1.0//Input5162009.B2B_1242894594000' (/harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B) has failed due to: ORABPEL-11167
Error while reading native data.
[Line=1, Col=70] Expected "\t" at the specified position in the native data, while trying to read the data for "element with name HDR_STORE_NUM", using "style" as "terminated" and "terminatedBy" as "\t", but not found.
Ensure that "\t", exists at the specified position in the native data.
<2009-05-21 04:32:44,141> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Deleting file : /harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B after processing.
<2009-05-21 04:32:44,141> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Deleting file : Input5162009.B2B
<2009-05-21 04:32:44,141> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Deleted file : true
<2009-05-21 04:32:44,141> <DEBUG> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Removing file /harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B from files to be processed Map.
<2009-05-21 04:32:44,141> <INFO> <ESIBT.collaxa.cube.activation> <File Adapter::Inbound> Done processing File : /harbinger/prod/xlate/Outbound_Input_LDS_810/Input5162009.B2B
<2009-05-21 04:33:09,698> <DEBUG> <ESIBT.collaxa.cube.engine.data> <ConnectionFactory::getConnection> GOT CONNECTION 4 Autocommit = false
For this error message this shows due to some /t its not picking up the file. but even i am facing the same issue for all the files where load is huge.
Thanks,
Saravana