OS randomly sees my mouse click, doesn't do anything

Question

I've had a problem for the past few months that's been driving me crazy.  Often, but inconsistenly, I will click on something and nothing happens.  More often than not I will be in Mail.app and will click to reply to an email.  I hear my mouse click, the click target turns dark to acknowledge my click, and then... nothing.  If I click on another email, it will come up fine.  If I click back on the original email and click on Reply it will dutifully pop out the email for me to type my response.  The same thing will sometimes happen in Safari.  If I have multiple tabs open, sometimes I will click on the close tab button, but it does nothing.  If I click on another tab, then click back to my original tab and then click the close tab button the tab closes like it should have originally.  Yesterday it got just absurd.  With the Apple.com site up I could click on any link and nothing would happen.  Like with mail, the color of the button on the page would darken to indicate that I clicked, but it wouldn't go there.  If I right-clicked and chose Open In New Tab it worked fine.  Other webpages in other tabs worked fine.  If I closed that tab and reopened the site, it was still broken.  Only after a wipe and restore did the site start working again.
Then, as if almost just to mess with me, in Terminal it will frequently register a click as a double-click.  I know it's a single click, I hear only one click of the mouse.  I'm having to constantly close the inspector window that pops up when it sees my "double-click" that is a single click.  It will randomly happen in other places as well.
Being a problem without an error code or message or easy way to describe, I haven't had any luck in any searches for people reporting the same problem.
I'm using a Mac Mini with OS X 10.8.3.  The problem has been around after multiple OS updates didn't seem to appear after any particular change made to the computer.
I've been a 'computer guy' for a long time, used most mainstream OSes since Windows for Workgroups, so I know my way around.  I've been using a Mac as my main computer for 7-8 years now.  I've already done a complete wipe and restore of the computer (using Time Machine).  I have an investment of time in the apps and data on my computer (my work computer to boot) so I am loathe to do a nuke-and-pave solution of resintalling everything from scratch instead of from a backup.  My hope is that there is someone else out there who had this problem and figured it out that might save me that trouble (and save my sanity).  These kinds of intermittent, nonsense issues are the worst.
Thanks to anyone who reads this and helps.

Answer 1

Sorry it took so long macjack.  I ran down to a Best Buy and grabbed a cheap-o wired mouse.  Unfortunately I had gotten so used to the large size of my usual mouse that I found it's small size almost unusable.  Last night I got another, better mouse.  At first, everything seemed fine.  This morning I was replying to an email and I clicked on the Reply All button and nothing happened.  Same thing as before, the button darkens on click, like the OS is seeing my mouse click, but nothing happens.
So far though, since switching to the new mouse I have not had an instance of the double-clicks that should be single-clicks and vice-versa.  I'll give it a couple of more days before I declare that problem squared away.  Perhaps I was having an OS issue as well as a mouse issue.
I'm trying to decide if I should try another user account first, or safe-mode.  Being this is my work computer, both are going to be difficult to try without impacting productivity.