Mac Pro Late 2013 USB problems / freezing

Question

I have a quad-core CPU Mac Pro late 2013 (Model Identifier: MacPro6,1).  MacOS X 10.9.5.
I have had all sorts of USB devices hooked up to it.  At any one time, I usually have all 4 ports filled.  I have a 3TB USB 3.0 disk that stores my large files, a USB mouse and keyboard (logitech with a usb mini dongle), a cable to charge my logitech USB cordless mouse, Lightning cable to my iPhone 5, and other things that I rotate in and out, like CF card reader, Audio Box USB audio interface from PreSonus, Sony Webcam, etc.
About 3 months into having the Mac Pro, I noticed that my keyboard went dead in the middle of using it.  The mouse was dead too.  I blamed the RF dongle that they both share, because the Apple Magic Trackpad (bluetooth) I have still functioned.  Try as I might, I couldn't get the keyboard or mouse to work again, so I used the Magic Trackpad to restart the machine, and then my keyboard and mouse worked again.
It wasn't until later that I realized that all the USB busses on the machine had frozen or "died" temporarily.  I realized it later because my USB hard drive complained about being "ejected improperly."
Now I have had the USB die on the Mac Pro at least 15 times over the last month and a half.  Usually once every two days or so.
I have tried (almost one by one) using some of the USB devices on the mac, and removing others to ascertain if it's a certain USB device that is causing this.  But the odd thing is that I never get a message from the OS like "xxx USB device is drawing too much power."
I'm going a little nuts here because I cannot see any rhyme or reason to the USB interface lock ups.  And each time it happens, all the USB devices go dead until I restart.  Sometimes, I'm able to SSH into the machine from my iPhone and issue a "shutdown -h now" and even though I see the Mac OS X UI shutdown, it never fully halts.  I often have to hold the power button to get the machine to turn off.
I really can't say if it's software related, hardware related or what.  I've tried to watch my workflow carefully to see if anything seems to make a pattern, but nothing yet.
Any suggestions? Is anyone else seeing behavior like this?  Do we think it's a USB device... or is my Mac Pro flakey?
Thanks!

Answer 1

I was using a hub, so I immediately removed that upon the first issues.  But the USB has still frozen after the removal of the hub. The HD is a self powered G-Tech drive.  I'll have to play around a bit more will all of your suggestions this weekend.
I woke up the computer just now to check the discussion, an am greeted with a notification from Mavericks that the "Disk Not Ejected Properly. (Eject disk before disconnecting or turning it off)"  which the disk is still connected, and the usb seems to still work fine.  In fact I was able to unplug the USB cable for the G-Tech HD, and then replug it in, everything reappeared fine.  The USB bus was not frozen (this time).
But it's this kind of flakiness that has me coming and going.  I haven't (yet) been able to identify whether the USB is the problem or the driver or a third thing I haven't figured out...