Maybe a Subpixel-Rendering problem ?

Question

Hallo there,
i’ve managed successfully to programm a Lens-Distortion Plugin, using the Skeleton Template as a base.
So far, every function from this Plugin is functioning like it should be. The single problem that I have, is that the Image is like “Pixelated“ on the edges maybe a Subpixel-rendering or antialiasing problem.
This can be seen at example foto down here. 1 is the original image and the 2 foto is the distorted image with my plugin. The subpixeling is really on each corner of the box.
I thought first, that i was rounding the variables, but i’ve checked my algorithmus and am using doubles for the parameters, so it can’t be round.
Do you have any Idea, what i am missing ?
Thanks in advanced,
Boban
The main function looks like this:
static PF_Err
MySimpleGainFunc16(
void    *refcon,
A_long    xL,
A_long    yL,
PF_Pixel16  *inP,
PF_Pixel16  *outP)
  PF_Err        err = PF_Err_NONE;
  DistInfo      *giP = reinterpret_cast<DistInfo*>(refcon);
  AEGP_SuiteHandler  suites(giP->in_data.pica_basicP);
  PF_FpLong      tempF = 0;
  PFBarrelCommon    *barrel = NULL;
  PF_Fixed      new_xFi = 0, new_yFi = 0;
  double        offset_x_ = 0, offset_y_ = 0;
  double        //GUI PARAMETERN
    c3_ = giP->low,    // Low Order par
    c5_ = giP->high,  // High Order par
    xp_ = (double)((giP->centerX)),  // Lens center X
    yp_ = (double)((giP->centerY)),  // Lens center Y
    in_width_ = giP->orig_width,      // Undistored x giP->footage_width;
    in_height_ = giP->orig_height,      // Undistored y giP->footage_height;
    orig_width_ = giP->orig_width + giP->footage_width,      // Undistored x giP->footage_width;
    orig_height_ = giP->orig_height + giP->footage_height,
    squeeze_ = giP->anaS,  // Ana... Squezze
    invSqueeze_ = 1 / squeeze_,
    f1 = orig_width_ / in_width_,
    f2 = orig_height_ / in_height_;
  Point2
    DistortPoint = Point2(1, 1),
    UnDistortPoint = Point2(1, 1),
    C3C5_ = Point2(c3_, c5_),
    outputAbsCent = barrel->calcAbsCent((float)xp_, (int)orig_width_, (float)yp_, (int)orig_height_),
    absCent_ = barrel->calcAbsCent((float)xp_, (int)in_width_, (float)yp_, (int)in_height_);
    offset_x_ = absCent_.x - outputAbsCent.x,
    offset_y_ = absCent_.y - outputAbsCent.y;
  Point2
    pt_ = Point2(((double)xL_*f1 + offset_x_), ((double)yL_*f2 + offset_y_)),
    norm_ = barrel->calcNorm((int)in_width_, (int)in_height_);
      switch (giP->distortion_mode)
        case DM_DISTORT:
          DistortPoint = barrel->PFBarrelCommon::solveDistort(C3C5_, absCent_, squeeze_, invSqueeze_, norm_, pt_);
          break;
        case DM_UNDISTORT:
          DistortPoint = barrel->PFBarrelCommon::solveUndistort(C3C5_, absCent_, squeeze_, invSqueeze_, norm_, pt_);
          break;
        default:
          break;
  new_xFi = (A_long)(DistortPoint.x) << 16;
  new_yFi = (A_long)(DistortPoint.y) << 16;
  ERR(suites.Sampling16Suite1()->subpixel_sample16(giP->in_data.effect_ref,
    new_xFi,
    new_yFi,
    &giP->samp_pb,
    outP));
  return err;

Answer 1

I would try connecting an external monitor, if the problem doesn't appear on the external monitor then you possibly have a failing internal LCD display panel or cable.
If the problem appears on the external monitor as well, then that would indicate to me its a graphics chip / logic board issue, this fault may qualify for Apple's free repair offer.
http://support.apple.com/kb/ts2377
What ever the outcome with the external monitor, it's worth taking it to an Apple store anyway, as they'll check it for free.