Colour Management - who does what - Some thoughts now the smoke is clearing

Question

First up, thanks very much to everyone who contributed their ideas and expertise to my recent query here, when I was seeking help for a problem with colour management issues when printing a magazine I edit. I have a ton of suggestions  to work through and study but the smoke is slowly clearing and it raises some interesting points which I think are worth recounting.
First of all, I have been editing short run magazines now for 25 years, at first part time and later on a professional contract basis.  I am not a trained graphic designer nor a trained printer. I did start out training as a graphic designer, many years ago but gave it up for a career in IT (as a networking specialist). That was full time until 10 years ago, although I did some freelance writing and editing in my spare time.
And yes, I did start originally with scissors and cut and paste, and moved on through black and white with spot colour and Pagemaker software  to full colour and InDesign today. One thing which may be different about my experience to most of yours is that I am a PC user and always have been. All my editing and graphics work has always been done on a PC - Pagemaker was our DTP package of choice for a long time and we supplemented this with Corel-Draw (which has a range of graphics handling options). All my software is legal and I always register it and keep it up to date. I have used the same graphic designer for quite a few years now and whenever we upgrade our software he goes and gets trained on the latest release.
Around 10 years ago I was offered the chance to edit a specialist short run magazine (not the current one). This was a chance I took and gave up the day job and became a full time freelance. Editing is not my main or only source of income. I am also  a freelance writer and photographer and heritage consultant and I have a specialist image library.   I sell my own sell my work - articles and pictures - to the national and local press. I also write books (non fiction) on commission. The magazine editing is really an extension of my interest in historic landscapes. I have never had any complaints, or problems, with the freelance work, photos and archived images I sell.  Clients include national newspapers here in the UK, national magazine groups and my books are available in national bookstore chains. I supply my work digitally, naturally, and it includes photos I have taken myself and items which I have scanned into my library of historical images and store on line. No reported colour management issues there.
I have always enjoyed a good relationship with my publishers and printers because I seek to be as professional as possible, which means delivering my stuff on time, to the required standard so that minimum intervention is required from them. This does assume that I have a clear brief from them on what they need from me.
Recently this approach has not been enough to avoid colour management issues with the short run magazine I currently edit. I have been wondering when  and where things went astray and date it back to the upgrade to InDesign two years ago. However it may have its roots in my earlier decision to use PCs not Macs for my work.
Until 4 years ago I had used the same printers for magazine editing for many years. They were a well respected firm specialising in short run magazines. They were not far from where I live and work and if there was a problem I would go over and discuss it with them. They were happy, and competent, to handle Pagemaker files generated on a PC and convert my rgb images to cmyk if there was any concern about the colour balance. On a few occasions I paid them to scan a photo for me. However 4 years ago the owner decided to retire and shut up shop. I needed to find a new printers and it had to be someone who specialised in short run magazines and could meet the budget of the charity I edit for. Also someone who could handle copy generated using Pagemaker running on a PC. I chose a printers I had used briefly in the past  where I knew some of the staff and was promised PC based Pagemaker would not be a problem. I even got this in writing. I started to send them proofs generated using Pagemaker v7 on my PC.
I soon found that although they had agreed they could handle Pagemaker on a PC in fact they had only a few PC based clients and were using a single ageing PC running Pagemaker to proof their work. In fact nearly all their jobs were Quark based. I was also told we had to supply CMYK images although not given any further requirement so I now did the conversions from rgb to CMYK using my PhotoPaint software. (There are quite a few settings in Corel for the conversion but there was no guidance  by the printer on which to use so to be honest it did not occur to me that it might be a problem).
Now of course I understand that the drive to get customers to supply CMYK images was a Quark driven requirement back in the late 1990s. I did not and do not use Quark so knew nothing for this.  I did have some early colour problems and font incompatibilities with the new printers and was pressured by their senior Graphic Designer (who designed for their own contract clients) to upgrade to InDesign and provide them with a .pdf, which I was assured would solve all my problems. The .pdf would be the same as the final printed magazine because "it would not require any further intervention by the printers".
I expect you are collectively throwing up your hands in horror at this point, but I think he was speaking genuinely. The creation of a .pdf  using InDesign, is widely promoted as the ultimate answer to all printing issues.   I have encountered it recently with a lot of printers' salesmen and my friend, who edits a learned journal, has just been told the same thing by her printers, to get her to upgrade to ID. Incidentally she also uses a PC.
So we upgraded our design process in house to InDesign and our graphic designer went on a course, two courses in fact. When we came to produce our first .pdf using ID, the printers'  Senior Graphic designer came on the phone and talked our designer through the ID Export function. I think he may at that time have told him to create a preset profile with MPC and the defaults, but to be honest I don't recall. We were never sent anything in writing about what settings we needed to match theirs. I continued to have intermittant colour management problems but put this down to my photos. Things came to head with the most recent issue where the colours were badly out on the cover, supplied by a press agency and taken by a professional photographer. The printers seemed to have little or no idea about possible causes.
Initially I thought that part of the underlying cause must lie in some mismatch between what I was sending the printers and what they expected to receive so I asked them to specify what I should send. All they said was use Profile preset as MPC setting and accept  the defaults which accompany it.
So I came on here looking for a solution. A lot of people were keen to offer their own experience which I really appreciate. However the messages could be conflicting. Some of you suggested it was the underlying cover photo which was at fault, some that it was my monitor which needed better calibration.
Many of you here said that part of the problem, if not the whole problem, was the way I was generating my CMYKs for the printer and I should use Photoshop to do this. You also mentioned a number of possible colour management settings which I should try.
At times the advice seemed to change tack. There were suggestions that the colour management issues I had  were nothing to do with the printers, that it was up to me not them. Quite a lot of you said I needed to be better informed about Colour Management issues. I agree, but I had never had any previously (maybe good luck, maybe good support from my previous printer) so I was not even aware that I needed to be better informed.  Some of you mildly chastised me for not finding out more and doing more to manage my own colour management with the switch to ID. To which I can only say if I had needed to train up, I would have done. I did not realise I needed to.  Nor was my designer aware of the issues as colour management was not really covered on his ID courses which were about typesetting and design.
Some of you even seemed to hint that unless I was prepared to use an expensive high end printer or effectively retrain as a print specialist or get my graphic designer to do so, then I probably shouldn't be in the magazine editing game at all. OK maybe that is a bit harsh but you get the drift.
The fact is that printing is much more accessible these days to all sorts of people and in particular to people with PCs. My brother lives in a large village in an isolated area and produces a village magazine which has been a great success. It is in black and white with spot colour but he would like to move to an all colour issue. He is a bit nervous of the colour management issues as he has no experience of graphic design and is his own designer using a low end entry level design package. He too uses a PC. The printers reps all tell him the same thing they tell me, that all he needs to supply is a .pdf using InDesign.
Somewhere I feel a black hole has developed, maybe back in the 1990s with Quark 4.11. A lot of printers standardised on that, and set up a work flow and prepress dependent on CMYK images as provided by the clients. They assumed the the clients would doing their own colour management. This approach also assumes everyone is using Quark on a Mac with the full range of Adobe software. When it became possible to generate .pdfs using InDesign, this was held out to users as the Holy Grail of magazine printing, even though their workflows and prepress were still based on Quark 4.11 principles. Any underlying colour management issues the clients now have to tackle themselves.
So now we have the situation in which I find myself, having to learn from scratch a good deal about colour management issues so that I can tell the printers what is needed for my magazine. Meanwhile all the printing salesmen, the ones I encounter anyway, are still busy pushing the InDesign to .pdf as the "be all and end all" solution. Some re-education is needed for all parties I think.

Answer 1

I am glad to see that the sun is peeping through the clouds.
I apologise for my Aussie-style straight talk earlier, but as I said before it was not directed personally at you but in the direction of others whom you epitomize, repeating a conversation I have had many times over the last 10 years or so where respectable, well-meaning photographers, designers and other contributors refuse to accept that colour management is being thrust upon them.
It is a simple fact of life, there is this 'new' thing that has butted into the very root of our trades and changed the most basic principles of printing and photography.  We expect that this kind of thing does not happen but the industry we now work in is not the same one we trained in twenty years ago.
Many printers are still struggling with the same conflict, so many tradespeople cannot accept this change.
This is exacerbated by the fact that colour management is so complicated to learn and implement and confounded by the fact that the default settings and a clumsy workflow often yield acceptable results with incorrect, generic settings, hence the old 'use InDesign and make a PDF and it will be ok' route.
When the chain of colour management includes the photographer, the photographer's client, the designer, the other designer maybe, the prepress person, and the platemaker, and a single incorrect click by any one of those can kill the CM it is not surprising that in the end when someone is looking back to see where it fell over they usually never find out.....   They will meet someone who says ' I never touched it, I simply opened the file and scaled it and closed it'.  And that person will be a reputable photographer or designer (and CLIENT) who has no idea they just broke it.  So what do we do?  We go with the generic setting that seems to yield adequate results therefore avoiding the confrontation. 
You need to understand the situation of the printer who took his business through the 'early' days of colour management, we had all kinds of very reputable sources supplying incorrect files, we did not have the expertise yet to be able to address the entire workflow, it would have meant training photographers and designers all through the best design houses and national institutions, because they blamed the printer.  Only in the last few years have I seen these people coming around to the fact that they bear responsibility for implementing their own cm and maintaining it through their own work.
Sadly, many high end sources are still not there, and I mean HIGH end!  Probably the ones that don't even visit this forum because they want to keep blaming the printer... They tend to live with the poor quality reproductions and just pull up the worst ones and fiddle with those and try to avoid the 'elephant in the room'.
I am sorry to say that it was not practical for a printer to reject mismanaged files for fear of losing clients who would happily accept less than perfect results in order to avoid the painful truth that was being told to them.  The best thing we could do was to gently make those clients aware that their workflow was imperfect and hope to show them how we could help...  Many print shops do not have someone knowledgeable enough or patient enough to do this, or the boss does not understand the issue either and tries to work around it to keep his jobs flowing in the expectation that all those experts in the chain will eventually tame the thing.
The many experts on this holy forum are waaaaayyyy ahead of the printing industry in general and photographers and designers in general in their understanding of colour management workflow.  I have seen first hand how reputable local industry people and trainers alike are spreading misinformation and bad techniques, when I discovered these forums back in about 2002 I found that they opened up a whole new galaxy of knowledge and facts that actually worked and made sense, unlike what I had been told locally....  This forum taught me what the Adobe text books did not, the Tech' teachers did not, local 'experts' did not! 
I tell all interested people to join these forums and learn to discriminate between the good and bad information.