Generic File Adapter Schema

Question

I am trying to pick up a .csv file that doesn't comform to a strict format, mainly because of human formatting errors in the file.
I've tried to create a fixed length schema and a comma delimited schema but both fail as soon as they get a file that doesn't exactly fit. How should I create the schema to cope with a file like this?
Regards
Jen

Answer 1

I too am trying to accomplish something similar.
My file is tab delimited and has 10 fields. It begins with 6 lines of header and ends with 1 line of footer. The Header is easy enough to skip over, but the footer is causing me no end of problems because the file Adapter is rejecting the entire file when it hits this malformed line. I tried to turn on uniqueMessageSeparator as ${eol} but that didn't seem to do anything different (I suspect it is because ${eol} is also a field terminator for the last field but I'm not sure).
Ideally I'd like to say any line that has 9 tabs is a Data line, and any line that doesn't is a NotData line but I haven't found any functions that will take a regular expression match as a condition for processing in the XSD.
I tried using the "choice" method of conditional processing but is seems that I must provice a string that is an exact match for nxsd:conditionValue and not a regex. I don't have a spcific string to look for.
I also tried to use the "Sequence" method of conditional processing that will treat any record that doesn't begin with a digit as a NotData Message of 1 string element terminated by ${eol} and any record that begins with 8 digits (which is a date in YYYYMMDD format) as a Data record using the "nxsd:startsWith" function but it seems I can only create multiple message types on files when there is some specific string to test. That's not going to work for me. There is no specific string on the records to identify them. The format of the line could be used to identify the records if a function supporting a regex could be found.
When I tried to deploy the "Sequence" method using a regex for the nxsd:startsWith value it seemed to put my BPEL Process into an infinite loop consuming 100% CPU. I had to kill the app server to stop it.
A detailed reference on how the File Adapter processes files and what functions are available for ignoring bad records and/or conditional processing would be appreciated. Also, any more ideas on how to get this file parsed would be appreciated.