Text document search

Question

I have 2 text files that I need to compare a child list and a parent list. The problem is leading zeros in one of the files. For example.
Child (600 mb)
123
134
4241
456Parent (1.5 gb)
0000123
00456
000789I have to find which numbers in the child list exist in the parent list. I would have just used a bash script and done a 'comm' on both the files, but the leading zeros is preventing me from this. So I decided to do this in java and I'd like to know if my logic seems correct (and if this would be the fastest way of doing it). Would it be faster to remove all the leading zeros from the both files, or would it be better to do something like this...
//looping through the parent list
//compare the lines and if they don't match continue
if ( childLine.elementAt(0) == 0 || parentLine.elementAt(0) == 0 ) {
   if ( childLine.length != parentLine.length ) {
      //trim leading zeros
      //compare again
}  else {
//continue through the loopLet me know if you need clarification. TIA.

Answer 1

prem1ers wrote:
I have 2 text files that I need to compare a child list and a parent list. The problem is leading zeros in one of the files. For example.
I have to find which numbers in the child list exist in the parent list. I would have just used a bash script and done a 'comm' on both the files, but the leading zeros is preventing me from this. So I decided to do this in java and I'd like to know if my logic seems correctAssuming that each line in both files is guaranteed to be a number, I would use Integer.parseInt() or Double.parseDouble() to convert the string to a numeric value and then compare those. However, unless you plan on reading one file in completely, or one file is known to be a subset of the other, the comparison logic is not straightforward.
I suggest you Google the 'diff' algorithm, which uses (as far as I remember) a "staircase" comparison. It may have you scratching your head for a while (as it did me :-)), but with files of that size it may be the best approach.
HIH
Winston
PS: 'comm' will only work if both files are sorted. If that is allowed, then of course the comparison is much more straightforward and would not require diff.
Edited by: YoungWinston on Nov 13, 2009 6:33 AM