Using a scanner to parse a text file

Question

Hi,
I'm running into some difficulties using a pattern for my scanner (java.util.Scanner). I need to scan a text file which consists of records. Each record has a keyword and a value (like property files):
Example:
FACILITY=TEST
LOT=1234567
DEVICE=abcdefgNormally, every record will be ended by a carriage return and line feed. So what I did was using a scanner with cr/lf as delimiter. That worked fine for most records but there are some records which span several lines (with cr/lf at the end of each line), e.g.:
MAP_XY.01.01=" Y-7 24/10 Y-6 24 22/17 15/7 Y-5 29/7 5 Y-4 32/22 20
19/15 13/4 Y-3 34/24 22/13 11/2 Y-2 36/30 28/24 22
21/12 10/4 2/0 Y-1 38/0 Y0 40/35 33/22 20/15 13/0 Y1 41
40/32 30/8 6/-1 Y2 42/26 24/-2 Y3 42/-5 Y4 47/11 9
8/3 1/-6 Y5 48/47 45/39 37/18 16/-7 Y6 49/8 6/-8 Y7 49
47/8 6/4 2/-8 Y8 50/9 7/3 1/-9 Y9 51/11 9/-10 Y10 51
50/33 31/-10 Y11 52/40 38/35 33/23 21/16 14/-11 Y12 52
51/34 32/15 13/4 2/-11 Y13 53/40 38/35 33/-12 Y14 53
52/35 33/27 25/18 16/13 11/9 7 5 2/0 -4/-12 Y15 53
52/-12 Y16 53/22 20/-5 -7/-13 Y17 54/47 45/-7 -9/-13 Y18 54
53/22 20 18/2 0/-13 Y19 55/16 14/13 11/-2 -4/-13 Y20 55
53/41 39/35 33/-14 Y21 55/52 50/48 46/16 14/10 8/-2 -4
-6/-14 Y22 54/33 31/7 5/4 2/0 -2/-9 -11/-13 Y23 55
54/53 51/26 24/-2 -4/-13 Y24 55/15 13/-14 Y25 55 53
52/-7 -9/-10 -12/-14 Y26 54/44 42/14 12/10 8/-4 -6
-7/-10 -12/-13 Y27 55/52 50/23 20/16 14/-13 Y28 54
53/12 10/-1 -3/-13 Y29 55/44 42/-13 Y30 55/15 13/10 8
7/-13 Y31 55/17 15/-13 Y32 54/44 42/-1 -3/-13 Y33 54
53/41 39/30 28/-13 Y34 54 52/37 35/19 17/12 9/8 5/3 1
0/-4 -6 -9/-11 Y35 54/-13 Y36 54/35 33/31 29/13 11
10/2 0/-2 -4/-13 Y37 54/-13 Y38 53/11 9/0 -2/-9 -11
-12 Y39 53/35 33/31 29/-12 Y40 53/22 20/15 13/-12 Y41 52
50/15 13/10 8/-11 Y42 52/11 9/5 3/-6 -8/-11 Y43 51
50/41 39/10 8/0 -2/-10 Y44 51/28 26 24/15 13/12 10
9/-10 Y45 50/43 41/32 30/4 2/-9 Y46 49/31 29/13 11
10/-8 Y47 49/47 45/30 28/24 22/-8 Y48 48/42 40 38/12 10
9/-7 Y49 47/42 40/38 36/-6 Y50 44/33 31/30 28/6 4/-5 Y51 44
43/28 26/17 15/13 11/-2 Y52 44/32 30/-2 Y53 43/32 30
29/13 11/-2 Y54 42/27 25/24 22/16 9 7/-1 Y55 41/21 19
18/0 Y56 39/15 13/6 4 2 Y57 38/20 18/3 Y58 36/5 Y59 34
33 31/30 28/7 Y60 31/28 26/10 Y61 27/14"
BIN_COUNT.01.09=12345Using the following delimiter does not work
fileScanner.useDelimiter("\\s*=\\s*");because the result would be something like:
keyword: FACILITY     value: TEST
LOTkeyword: 1234567
DEVICE     value: abcdefgWhat I need is a pattern which also handles carriage return/line feed until a new keyword (like LOT for example) occurs.
The correct result should be:
keyword: FACILITY     value: TEST
keyword: LOT            value: 1234567
keyword: DEVICE     value: abcdefg
keyword: MAP_XY.01.01 value: " Y-7 24/10 Y-6 24 22/17 15/7 Y-5 29/7 5 Y-4 32/22 20...
{code}
I hope, I did explain clearly enough...
Any help would be appreciated.
Thanks and best regards
- Stephan                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Answer 1

Any help would be appreciated.You have grouping and data in the groupings and you are attempting to parse it all at one go.
Won't work. (Might in a perfect world but not in this one.)
You need a parsers that do the following
1. Parser text file (stream) into records.
2. Parse records into data.
You can put both into one class or not.
Your record parser is going to be based roughly on the following scheme.
1. Find keyword,
2. Look up format defined for keyword
3. Attempt to read format found.
4. Error if format not found.
5. Else return the record found.
I say roughly because as noted above you need to know what the actual output looks like. It is possible to determine this if you have enough examples but it is a lot easier if you actually get the specification that tells you what it looks like.
I suspect, but don't know that the "format" in the above will consist of the following.
<word><text>
or
<word><text>
<space><text>
<space><text>
You could just create one format for the all but you will likely have record types that don't fit that and the earlier you detect errors the better.