Cross reference stream

Question

Hello!
I'm in real trouble, and I think that the answer is very close to me, but I can't get it by myself...
Here is my problem:
I'm currently improving the PDF parser that I've made, adding support for cross reference streams. After a careful reading of the ISO 32000 specification (with PDF 1.7 in background), I've wrote a piece of code that decode the stream and build the corresponding objects (I use Java).
But the decoded values are meaningless, and I don't understand why!
The data are like that:
filter predictor: 12
encoding : Flate
fields length : 1 2 1 (this is the W entry values)
column size : 4
I assume that there is one color per pixel formed by one byte.
First, I 'deflate' the data, then 'unfilter' them. Both operations look ok as I've made test for the Flate encoding (ok), and the for the filter I've use it with PDFBox source (and my work unfiltered well what PDFBox filtered...).
Note that I compared my outputs with those produced by PDFBox, and they are the same... Something may be wrong in the parameters I use, but I no idea now...
I think I missed out something as I read incorrect data as entry type 18...
I don't know where to go now, after a whole week of work!
If any have an idea, I'd be grateful to hear it!

Answer 1

This is ready algorithm to read PNG predictor styled Cross Reference Stream:
1. You need to read from PDF three variables: /Columns 5/Predictor 12 and /W[1 3 1] and of course stream
2. Deflate (encode) stream via zlib (without passing anything more than stream)
3. Divide encoded stream to rows, but length of each rows must be one bigger than columns variable (in this case is 6) because first value is type xref row
6. rows without first row go to UNprediction with this algorithm:
def self.do_png_post_prediction(data, bpp, bpr)
        result = ""
        uprow = thisrow = "\0" * bpr # just zero fill (byte zero - not char zero)
        ncols = data.size / bpr
        ncols.times do |col|    # just for each rows
          line = data[col * bpr, bpr]
          predictor = 10 + line[0]
          for i in (bpp..bpr-1) # just for each values of row without first value
            up = uprow[i]
            left = thisrow[i-bpp]
            upleft = uprow[i-bpp]
            case predictor
              when PNG_NONE
                thisrow = line
              when PNG_SUB
                thisrow[i] = ((line[i] + left) & 0xFF).chr
              when PNG_UP
                thisrow[i] = ((line[i] + up) & 0xFF).chr
              when PNG_AVERAGE
                thisrow[i] = ((line[i] + ((left + up) / 2)) & 0xFF).chr
              when PNG_PAETH
                p = left + up - upleft
                pa, pb, pc = (p - left).abs, (p - up).abs, (p - upleft).abs
                thisrow[i] = ((line[i] +
                  case [ pa, pb, pc ].min
                    when pa then left
                    when pb then up
                    when pc then upleft
                  end
                ) & 0xFF).chr    # AND 0xFF is very important, without this values growing extremely fast
            else
              puts "Unknown PNG predictor : #{predictor}"
              thisrow = line
            end
          end
          result << thisrow[bpp..-1]
          uprow = thisrow
        end
        result
      end
   7. Take bytes from UNpredicted rows, and in this case
   1st byte first byte is type
   2nd byte    
   3rd-5th - is this what You want - offset in decoded PDF to objects (Xref table): offset = 2^9*5th + 2^8*4th + 3rd