Camera Raw vs. Photo Ninja

Question

I've been hearing about a new product called Photo Ninja by the same people who make Noise Ninja, and some of the reviews by respected online personalities have actually touted it as producing superior images to Photoshop's Camera Raw.
Not being one to take others' opinions at face value without testing and looking at results with my own eyes, I requested an evaluation license to see for myself.
I'll say up front that going into the testing I figured Camera Raw would kick this contender to the curb.  It's been (and still is) my opinion that Camera Raw is the best raw converter available today bar none.
Upon first trying out the package, I immediately stumbled upon a number of usability issues that would make PhotoNinja a pain to use...  Things like unexpectedly slow control responses (I have one of the faster PC workstations available), some not quite intuitive controls, a UI that doesn't seem to conform to any known or typical computer standard (though I'll be the first to admit I have limited exposure to OSX)...  It's not surprising someone intimately familiar with Camera Raw might have some initial troubles with the UI of another package, but I stumbled through it.
Mostly I am interested in image quality.  Results - that which matters most to me.
I looked at a number of well-exposed Canon EOS-40D images I have captured, and some not so well-exposed.  I looked at images that were taken with wide angle zoom lenses and needed correction of various kinds of distortion.  In general, if I did something with either converter that made an image better, I attempted to equal or surpass the feat with the other converter.  When I couldn't get things any better I stopped and examined the results.
Before I end up writing an even bigger novel, here are my (brief) observations on image quality so far, followed by a few screen grabs showing comparisons.  Perhaps these can generate discussion...
OVERALL, Photo Ninja (PN) does a decent job - better than I expected, actually - but the devil is in the details.  Photoshop Camera Raw (ACR) just makes images that consistently look better.  I will say that I got to better looking results much more quickly with ACR, but I'm more familiar with it, so that may not be an important observation.
PN can't generate upsampled output (images with more pixels than the number of photosites in the camera), while ACR can.  That flavors the quality - in general it means that while I started out thinking PN might be making images with more detail, it turned out that the more I tweaked each converter the more detail I was ultimately able to bring out with ACR, while PN reached a limit primarily because it couldn't pack the output pixels any tighter.
ACR is generally capable of finer adjustments.  For example, PN can only rotate images in 1/2 degree increments.
Though PN does a quite decent job of de-mosaicing - visibly better than ACR's prior PV2010 process - the color and naturalness of appearance is better to my eye from Camera Raw with PV2012.  Tree branches against bright sky or white buildings just look better and more natural from Camera Raw.  That said, there were several cases where it appeared some lines were rendered more cleanly by PN (see the dark green trellis example below).
ACR's automatic chromatic aberration (CA) correction facilities seem more accurate overall, while PN left more colored fringes in most cases.  On the other hand, ACR in some cases left slightly more colored de-Bayering artifacts.
Both tools seem capable of recovering a LOT of information from nearly overexposed parts of images, though ACR gets the nod for making the transition into overexposure look more natural.
I don't know how much profiling / tweaking capability PN has, and I do know I've tweaked my ACR some, but colors just look cleaner and more healthy to me (with a few exceptions) from ACR.
Anyway, I've made the following screen grabs to illustrate some of the above, Photo Ninja conversions are on the left, Camera Raw conversions are on the right.  Note that the zoom levels are different because I opened images at upsampled resolutions in ACR (as I normally do) while PN cannot do this.
My conclusion:  Though Photo Ninja produces surprisingly good conversions, it's not really serious competition for professionals who want to get the most out of their raw image data.
-Noel

Answer 1

Well done, Noel !
Even though ACR won out in your tests, the makers of PN should be pleased with your report and your methodology.  Thanks for sharing.