Photoshop's Inherent Weakness - "out of gamut"

Question

Hi Folks, well I think it is about time to for me to say this.  After weeks and weeks of head scratching and emails and experiments on my 3800 I have to conclude that photoshop (in my case CS3) has a huge flaw - it does not truly represent colour that is either in gamut or out of gamut.  The truth is that ordinary photographs do indeed hide subtle colour inaccurracies that when seen as a single tint on a greyscale image really show up, and photoshop is incapable of dealing with what is in gamut and what is out of gamut with regards to  colour mixes on fill layers.  Untill I started to tone my images and combine colour fill layers on varying opacities and blend modes I NEVER had any problems with colour management (just tweaks here and there) but when I went into toning I discovered that Photoshop simply fails miserably to tell you what it is able to print within a colour space and what is outside that colour space.  Oh and by the way soft proofing?  Overated nonsense that is a joke and three quarters.
The only accuracy I got with regards to Photoshop and toning was when I placed these same images into the sRGB space, at that point I was shocked to see Not Perfection by any means, but certainly much more closer to the screen image that was, by the way, in the Adobe RGB1998 workspace.  So I tested many of my images that were in Adobe 1998 RGB on the screen and viewed them under the ASSIGN PROFILE = sRGB space and discovered a huge shift closer to the print out - not by any means accurate but very close indeed.  This, amongst many other experiments, proves that Adobe have failed here in a basic software operation, rendering out of gamut colour. As I said before - never any problems with ordinary colour photography, and this point is exceedingly important. 
Colour fill layers are decpetive and adobe needs to address this - I know of nobody that uses colour fill layers to achieve what I have done and all toning that I have ever read about has been a sort of a flick of a few switches OR using the duotones and tritones menu (which I find lacking in creativity).   The colour fill layer dialoh=gue box does not render what is truly out of gamut, and what is more it is not even as though my choices of colours were anywhere near the borders of out of gamut.
Conclusion?  Dissappointed in the overall rendering of accuracy of colour gamut with photoshop.

Answer 1

Hallo Rick, how are things?  Well to be honest I am so really fed up with this whole situation and have wasted three weeks trying to figure it out.  I am tired of repeating myself.  I have just, at the suggestion of another person, soft-proofed my adobe rgb 1998 toned image in the sRGB space with colour nubers set to "preserve bumbers" in the soft proof dialogue box.  I know see perfect matches between my print out and screen image.  NOW, this is important, I have NEVER had this problem with normal colour photographs.  So it is not all to do with colour management.  And as Tim on photo.net said:'
Chris, I'ld like to add some info I didn't mention in that thread that may explain a bit about Photoshop's Soft Proofing and out of gamut warning tools and their intended use. According to many discussions over at Adobe forums I've participated in on your issue, Soft Proofing is meant for photographs only.
A duotone or toned image as you call it is considered a graphic effect. In fact the old way of getting this look was performed by graphic artists and prepress technicians before digital made it a click and print affair. I used to be both a graphic artist and prepress technician and the process for getting a toned image printed on a commercial press before inkjets came around involved cutting light blocking overlay film knockouts for each Pantone ink that was to be mixed using darkroom exposure tricks performed by the prepress technician.
Color gamuts weren't an issue because the inks were custom mixed on press. Can't do that on an inkjet. This still can be done in Abobe's InDesign by picking spot colors and the separations are done through a RIP (Raster Image Processor) where the 1's and 0's get turned into those light blocking knockouts for burning plates used for printing to a commercial CMYK press.
Another aspect about why Gamut Warning and Soft Proofing is engineered for photographs is because photographs have by their nature a dithered pattern made up of millions of alternatiing pixel colors that act as a color slice and dice mincer that aids a printer that prints with a dithered pattern in smoothing out color TRANSITIONS it can't reproduce.
There's a lot of "faking it" going on utilizing this dither smoothing in an attempt to fool the eye into accepting the color the printer chooses in relationship with the rest of the surrounding colors in the image. This is why there are rendering intents named Relative, Saturation, Perceptual and Absolute in Soft Proofing dialog box and in Photoshop's printer dialog box.
You should try them out and see what happens with your toned images, but outcome can't be guaranteed because toned images don't possess enough alternating color dithering as in a photo to allow the printer to provide smooth color transitions. You'll notice their is a check box for "Dithering" in the Convert To dialog box. This isn't enough for toned images especially if choosing a color fill that doesn't allow the printer to "fake" this look. These printers including yours which uses a gamut limiting pigment inks for longevity over larger gamut dye based only have cyan, magenta, yellow and black to produce transitions in intense colors. That's not much to work with.
And remember saturation is tied to luminance levels. The printer must reproduce the luminance level as well as hue and saturation using just two inks, in your case magenta and yellow, to get that orange brown. The only way to get a darker and lighter version is to add black because adding cyan, a mid density ink, will make it a dull brown. This is just plain physics learned in any paint mixing class.
Pre-visualization tools such as Soft Proofing are primarily meant for allowing the remapping of intense colors like in flowers to allow editing for an alternative hue and saturation level that will allow the printer to "fake it" more precisely.
So basically photoshop might be the problem.  But, if you seriously think that I ma at fault then I certainly will give it one last try because I really am sick of this ridiculous situation.  I will follow your instructions to the letter and will go through it for the 10th time.
Kindest regards
Chris.