Ical invitations not working, PLEEEEASE help! (long, sorry)

Question

I have an iphone 4 running iOS 5.1.  My husband has a 3GS running 5.x (as current as he can get it, not sure if it's 5.1 or not).
I have my original apple ID from years ago, which I use for just about everything - it is NOT an apple email address.  Then I have the second mac.com apple ID I was forced to make.
Husband has a yahoo.com apple ID, and a me.com (which he never uses).
[Side note: WHY, OH WHY, won't Apple let us merge Apple IDs?? This is absurd!]
I have spent HOURS trying to troubleshoot my ical invitations, but can't figure it out.
When I invite him to an event from my phone-
     If I invite his me.com address, he gets an alert in ical.  When he accepts it, it appears in his calendar, and I get a notification (in iCal on my iMac, but NOT on my phone) that he accepted. ***THIS IS HOW IT SHOULD WORK***.
     If I invite his yahoo address (the one he uses exclusively) he gets an email that he's been invited.  If he accepts through email, I do not get a notification, but, if I look at the event details, it says he's accepted.  BUT!!!!! The event DOES NOT SHOW UP IN HIS CALENDAR.
But, I can deal with that.  As long as I invite his me.com address, it works as it should (except for me not getting the accept notification on my phone.)
Now, flip the tables.
He invites me:
He sends it to my me.com address- nothing happens.  I do not get an alert in ical (on my phone OR my mac), and I do not get an email.
He sends it to my main (non apple) email- still nothing happens.  Not via ical or email.
BUT- if he sends it to my WORK email (which is not in any way associated with any apple ID), i DO get an email.  But, as above, if I accept it- he can see that I accepted, but, it does not appear in my calendar.
I've read some threads here implicating the email addresses associated with apple IDs.  I went through all 4 of our apple IDs and made sure that all possible email addresses were "verified", so I'm not sure that's it. In fact the only email that worked at all for me was the one NOT associated.  But that's crazy- that it will only work if an email is NOT linked to an apple ID? What?
I don't know why it works for him (sort of), but not for me!  If anything I'd understand if the iphone 4 could accept invites but the 3GS couldn't, but it's the other way around!
I tried it about a million different ways this morning, and one thing I noticed:
when I invite his yahoo (and thus send an email) the from address was always crazy- NOT the default address in my email.  Like an old obscure me.com alias that I don't use and have not registered anywhere.  So, I would add that email to my apple ID.  Eventually got all weird aliases linked to an apple ID.  then it sent from my 'normal' mac.com address.
When he invites me (to my work address), the email says it's from his me.com, which is an account he doesn't even know he has, and doesn't use.  he only created it because he HAD to.
Why on earth is this so difficult and confusing?
We're not using google calendars or any third party stuff.  It's iphone to iphone. me.com to me.com.
Can anyone please help??

Answer 1

https://discussions.apple.com/message/17319571#17319571
instead of verifying email addresses associated with your appleID, try removing them.
I've read some of similar thread just now, and remove the associated email withing my apple ID
now, my friend can send invites to me ([email protected]) and i can reply within the ical and the event is created in the ical.
however, when she tried to send invites to my 2nd appleID (yahoo.com) i don't get anything.
I know this sounds very stupid. but at least i've got a temporary solutions.
hope this also works for you.
perhaps apple just us to use every service they've offer. there's too many ridiculous way to work in iCloud.
sometime i regret migrate to icloud. MobileMe was MUCH MUCH better in sync-ing