How to install third-party SSD as boot disk in MacPro 5,1

Question

I had so much trouble with this that I thought I'd post a recipe that others could follow.
The first thing you should know is that it's not possible to get a non-Apple SSD recognized in a sled slot of a MacPro 5,1.  It just doesn't see them at all, even if (as some suggested) previously partitioned as a GUID partition (which works in a MacBook Pro, but not a MacPro 5,1).  This is true even with the most up-to-date EFI firmware.  You're going to have to use a PCIe adapter card.
The next thing you should know is that not all PCIe SATA3 SSD adapter cards are bootable on a MacPro 5,1.  For instance, even though the Sonnet Tempo SSD is advertised as working for a boot drive in a Mac Pro, it actually works in any Mac Pro *except* the 5,1.  In a 5,1 drives on the card do not work as boot drives, either with option boot or if you select it as the startup drive after booting from something else.  Also, EFI boot is extended by about a minute -- not the end of the world, but an annoyance.  The Tempo card does work well for add-on drives, its very fast.  I am pretty sure the Tempo Pro variant of the card also does not work, as the Sonnet support guys told me that they have to use use rEFIt for booting 5,1 MacPros in their lab.  I imagine if the Pro worked they would have pointed me at that instead.  (Note that rEFIt has been superseded by rEFInd, http://www.rodsbooks.com/refind/.  I didn't try using rEFIt or rEFInd because I found a better way, but if you want to dual-boot to Linux those utilities are apparently Da Bomb.)
An Apricorn Velocity x2 Duo card *will* boot, and does not have a long EFI boot delay.  That card is a little more expensive than the Tempo SSD (but not the Tempo Pro SSD) but it works properly.  It's worth the extra money, especially since it's the only card I could find that explicitly claimed to work as a boot drive on a 5,1 MacPro.  And it does.
One caveat: With the Tempo SSD the SSD drives were invisible until repartitioned in GUID format, which also happens if you try to use a third-party SSD inside a Macbook Pro.  I found that I could repartition one drive, a Crucial MX500 960GB, using a run-of-the-mill USB SATA2 adapter.  Unfortunately this solution did not work with a Samsung Evo 1TB drive, I had to use an external USB3 enclosure.  Moreover, the enclosure supported both FireWire and USB3 but FireWire did not work.  This FireWire issue might be due to the super-cheap enclosure I was using, so YMMV, but keep in mind that it's possible that you'll need either a USB3 expansion card in the MacPro, or a different Mac, to perform the initial repartition to GUID format.
It's possible that SSDs would be visible on the Velocity card even before repartitioning, I don't know as both of my drives had been repartitioned prior to trying the Velocity.  So, it might work ... but you might need to partition externally.
I don't know if pre-partitioning is necessary with the Velocity card, as I only ever tried pre-partitioned drives with it.  If you try the Velocity and the drive isn't seen, that's the problem and you'll have to use an external SATA adapter of some sort to repartition before installing on the Velocity card.
After partitioning the drive and installing on the Velocity card, it worked fine.  Yosemite installed properly and with reboot it blazed.
Next up, TRIM support.  For some reason known only to Apple, the Apple AHCI driver refuses to enable TRIM support for anything other than Apple drives.  It's possible that there are or were bugs in some of the early SSDs and Apple is just playing it safe, although I would have preferred to have the option to enable TRIM or not if that's the case.  Unfortunately there is no such option, you have to use a substitute AHCI driver.
Unfortunately if you're using Yosemite you cannot use a substitute driver without disabling kext extensions, because Apple will not sign anyone else's driver (at least not at the time of this writing).  That's unfortunate, I would prefer to have signed extensions if I could, but realistically speaking you're no worse off in terms of security than you were with Mavericks so it's not the end of the world.  With a little luck Apple will either start supporting TRIM on third-party drives or starts signing third-party drivers sometime in the future.
So: Before you can start TRIMming drives, you first need to run this command, or something like it:
sudo nvram boot-args=“kext-dev-mode=1”
I rather like seeing the boot process, rather than just the Apple logo, so I use this:
sudo nvram boot-args=“-v kext-dev-mode=1”
I tried both Trim Enabler (http://www.cindori.org/software/trimenabler/) and Chameleon SSD Optimizer (http://chameleon.alessandroboschini.com/).  As of this writing the Chameleon driver is more up-to-date, and comes with some additional utilities that are pretty nice.  Take your pick, though, both worked fine in my experiments.
With a TRIM-enabled AHCI driver, kext signing disabled, a Velocity card, and your favorite SSD you are in business.
The only weird thing after all of this is that non-boot SSDs on the Velocity card (or on the Tempo) are ejectable.  As far as I can determine there’s no way to mark them as non-ejectable.  That’s a minor annoyance compared to the performance you get out of the SSD, though.
Because, with this stuff in there, yowza is the MacPro 5,1 a fast machine.  You Will Like It.  Enjoy!

Answer 1

Then you should read these.
http://forums.macrumors.com/showthread.php?t=1804704&highlight=sonnet+tempo+pro
http://forums.macrumors.com/showthread.php?t=1721573&highlight=sonnet+tempo+pro
This was strange but has to do with how startup disk and nvram interact
http://forums.macrumors.com/showthread.php?t=1795765
I think your problems were solvable and not that the Sonnet card and 5,1 is not bootable, in fact the 5,1 is easier, there are more supported cards.
I have the 1,1 where the Sonnet Pro is recommended but for data only and are not bootable because of the 32-bit EFI firmware. NOT the 4,1 and 5,1. The 2008's are problem child and have a mix of PCIe 1.1 slots, first implementation of EFI64 that also has problems. Also the 4,1 has two PCIe 2.x slots 3 and 4 that share a controller but that simply limits bandwidth.
I would have left the TRIM support and NVRAM changes in Yosemite to a separate thread if you wanted to focus on the Sonnet Tempo Pro support in Mac Pro 2010-12 5,1s.
Also the make of SSD can vary and they also have their own firmware as do bootable controllers along with the EFI of the Mac, all of which have to work together.  Meaning it might be that Samsung 840 va 840 EVO vs 840 Pro will all be slightly different but my experience is Samsungs work well (as do others like Crucial with the proper firmware - and there was/is an issue with the firmware of the Samsung 840 but not with booting).
The PCIe card gets seen and treated as external and therefore 'eject' but of course you can't when it is the system boot drive.
One thing I found last week and meant to mention:  After installing or changing PCIe cards, RESET SMC made all the difference in the world with the long boot delay. Some PCIe cards will add a few seconds, system has to search the device tree which now shows another controller with multiple devices - and sometimes NVRAM and SMC are what I will call "dirty" and need to be rebuilt and reset.