Unmarshal fails with OTD Tester

Question

Sorry to bother you again with another JCAPS-question, but I would like to know how to troubleshoot these kind of error messages. I tried to test my deployment dpData_Conversion_for_IP but it failed with an unmarshal error..now I have seen this error before and I know that other folks fixed it for me the last couple of times, but I'd like to learn on how to address an issue like this.
When I ran the otdtester I receive an error message like:
There was an error during unmarshal.
"...00100001\r\n"<POS>"L113218807..." failed input match, after RecordGroup[1]: left over 3086 bytes at end of data
When I enable Verbose Mode I get the following detailed output (see attachment) which basically tells me at te end that there are so many bytes of data left that could not be matched. Does it mean that my file I present to unmarshal contains more data than I told it to expect? How would you normally fix an issue like this? I looked at the source file and the otd and the field lengths do add up: the header ends at position 163 and the line items at position 125
Do the line item lines need to be adjusted (the number of times they repeat?)
My custom input otd used to work fine until last week then I made some changes to the EDI-mapping, but I didn't alter the field length, or the number of fields, so then it should not have any effect.
Below you will find the output in Verbose Mode:
[ parse #0 ]
[ parse 'root', pos=0, end=292, mandatory  ]
[ parse group, count=0, order=0, must ]
[ parse #1 ]
[ parse 'RecordGroup', pos=0, end=292, mandatory  ]
[ parse group, count=0, order=0, must ]
[ parse #2 ]
[ parse 'Header', pos=0, end=292 ]
[ parse group delim, Fog#25 ]
[ - push local delim list ]
[ - iteration #0 ]
[ parse 'RecType', pos=0, end=292 ]
[ parse fixed, Fog#3, count=0 ]
[ - match EXACT "H1" ]
[ - matched data 0 to 2: "H1" ]
[ - parse: result=1 ]
[ - parse: 'RecType' , done=false, pos=2, end=292, last=EOD ]
[ parse 'PackListNumber', pos=2, end=292 ]
[ parse fixed, Fog#4, count=0 ]
[ - matched data 2 to 22: "5160448             " ]
[ - parse: result=1 ]
[ - parse: 'PackListNumber' , done=false, pos=22, end=292, last=EOD ]
[ parse 'SequenceNumber', pos=22, end=292 ]
[ parse fixed, Fog#5, count=0 ]
[ - matched data 22 to 27: "00001" ]
[ - parse: result=1 ]
[ - parse: 'SequenceNumber' , done=false, pos=27, end=292, last=EOD ]
[ parse 'StockNoticeNumber', pos=27, end=292 ]
[ parse fixed, Fog#6, count=0 ]
[ - matched data 27 to 47: "51604480000100000000" ]
[ - parse: result=1 ]
[ - parse: 'StockNoticeNumber' , done=false, pos=47, end=292, last=EOD ]
[ parse 'CurrentDate', pos=47, end=292 ]
[ parse fixed, Fog#7, count=0 ]
[ - matched data 47 to 67: "                    " ]
[ - parse: result=1 ]
[ - parse: 'CurrentDate' , done=false, pos=67, end=292, last=EOD ]
[ parse 'SequenceNumber2', pos=67, end=292 ]
[ parse fixed, Fog#8, count=0 ]
[ - matched data 67 to 72: "00001" ]
[ - parse: result=1 ]
[ - parse: 'SequenceNumber2' , done=false, pos=72, end=292, last=EOD ]
[ parse 'RecvStoreNumber', pos=72, end=292 ]
[ parse fixed, Fog#9, count=0 ]
[ - matched data 72 to 89: "0000280942       " ]
[ - parse: result=1 ]
[ - parse: 'RecvStoreNumber' , done=false, pos=89, end=292, last=EOD ]
[ parse 'SchedSendDate', pos=89, end=292 ]
[ parse fixed, Fog#10, count=0 ]
[ - matched data 89 to 98: "080808   " ]
[ - parse: result=1 ]
[ - parse: 'SchedSendDate' , done=false, pos=98, end=292, last=EOD ]
[ parse 'CreationDate', pos=98, end=292 ]
[ parse fixed, Fog#11, count=0 ]
[ - matched data 98 to 107: "080808   " ]
[ - parse: result=1 ]
[ - parse: 'CreationDate' , done=false, pos=107, end=292, last=EOD ]
[ parse 'CreationTime', pos=107, end=292 ]
[ parse fixed, Fog#12, count=0 ]
[ - matched data 107 to 113: "031229" ]
[ - parse: result=1 ]
[ - parse: 'CreationTime' , done=false, pos=113, end=292, last=EOD ]
[ parse 'BillOfLadingNumber', pos=113, end=292 ]
[ parse fixed, Fog#13, count=0 ]
[ - matched data 113 to 133: "5160448             " ]
[ - parse: result=1 ]
[ - parse: 'BillOfLadingNumber' , done=false, pos=133, end=292, last=EOD ]
[ parse 'ASN', pos=133, end=292 ]
[ parse fixed, Fog#14, count=0 ]
[ - matched data 133 to 136: "ASN" ]
[ - parse: result=1 ]
[ - parse: 'ASN' , done=false, pos=136, end=292, last=EOD ]
[ parse 'TRANSACTION_ID', pos=136, end=292 ]
[ parse fixed, Fog#15, count=0 ]
[ - matched data 136 to 163: "ASN08080851604480..." ]
[ - parse: result=1 ]
[ - parse: 'TRANSACTION_ID' , done=false, pos=163, end=292, last=EOD ]
[ - retry parent delimiter ]
[ - retry failed: overshoot, del=164, pos=0 ]
[ - parse: result=0 ]
[ - for #2: count=-1 ]
[ - parse: result=0 ]
[ - parse: 'RecordGroup' , done=false, pos=0, end=292, last=EOD ]
[ - for #1: count=0 ]
[ - parse: result=1 ]
[ - parse: 'root' , done=false, pos=0, end=292, last=EOD ]
[ - for #0: count=1 ]

Answer 1

Hey, I got some help from the open-esb users forum. See below.
Mike,
After modifying the xsd (adding match, removing scavenger, moving delimiter
level to the top, removing 2 FILLER2 fields, etc.) and modifying the data a
bit (padding spaces to each detail record to a total 392 bytes excluding
CR&LF), I am able to parse the data. Please refer to the attached zip
including input, xsd, output XML, and parse log.
Any further questions, please let me know.
Thanks!
Lixin
Open ESB Community
http://open-esb.org
-----Original Message-----
From: [email protected] mailto:[email protected]
Sent: Friday, October 17, 2008 2:05 PM
To: [email protected]; 'Mike A Reynolds'
Subject: RE: Custom Encoders Questions
Mike,
There are a number of problems I have found so far as I start looking into
the provided xsd meta and input data:
(1) The header element is also delimited by CR&LF. I see you have put
delimiters for the detail and trailer elements, but the header element
misses to be delimited.
(2) CR&LF can be expressed as "\r\n" or "\x0D\x0A", but not "\n\r". I see
you have "\n\r" in your xsd file which is wrong.
(3) You said header record has 82 bytes, detail has 382 bytes and trailer
has 91 bytes. But I've found a few mismatches:
In your input data file, the header record has 80 bytes + CR+LF = 82
bytes, but your xsd meta definition gives header total byte count of 81
(without CR+LF), which mismatches the expected 82 bytes. You'll need to
correct this in order for the parsing to work. (For me, I have to remove the
"filler2" field in order to get the header parsed.)
In your input data file, each detail record has 350+CR+LF = 352 bytes,
which contradicts the expected 382 bytes. I haven't added up the total bytes
in your xsd meta file, but I suspect they have mismatches.
Similar mismatch may also exist for the detail record, but I haven't got
chance to sum the total.
(4) The header, detail and trailer records all have first byte ("H", "D" or
"T") as record type. You must specify the "RECORD_TYPE" field to match the
character "H", "D" or "T", respectively by putting the character in the
"Match" field and selecting "Alignment" to "exact", meaning exact match.
I see these characters were mistakenly put in the scavenger fields
instead. Please correct this.
After you modify your xsd meta encoding information based on above, you
should be able to parse the input data.
Let me know if there are any questions,
Regards
Lixin
Open ESB Community
http://open-esb.org
-----Original Message-----
From: [email protected]
mailto:[email protected] On
Behalf Of Mike A Reynolds
Sent: Thursday, October 16, 2008 3:11 PM
To: [email protected]
Subject: Custom Encoders Questions
I am trying to create a customer encoder for a batch file
consistes of a header record of 82 bytes, several detail records of 382
bytes and one trailer record of 91 bytes
HACCUMULATED BENEFIT TRANSACTION FILE 20080801 20080825 0
0
D ar3531 ar3531 ar3531 ar3531000
55887027260 30.000 30 5098765 20080804 000T114 120080806
82820P30000001024511 25.00 0.00 25.00 0.00 25.00
25.00 0.00 0.00 0.00 0.00 0.00T
TACCUMULATED BENEFIT TRANSACTION FILE 20080801 20080825
2894 0 0
(See attached file: accumLoad.xsd)
It seems to not like having multiple record formats and dies after reading
the header. How can I tell the encoder to filter on the RECORD_TYPE so that
it picks up the different formats.
Best Regards,
Michael Reynolds
Senior Solutions Engineer - Architecture and Engineering