Screen unresponsive after holding phone to face during call

Question

Since buying my first iPhone 4 early in 2011, and on every one of the three subsequent replacements to the 4 and both the original and replacement 4s, I have had this problem:
1) Buy phone
2) Build from scratch (do not restore from older image...build fresh)
3) Put no apps on the phone...clean
4) Run 3g and wifi (though I've tested this with wifi off)
5) Take call (or place call)
6) Stay on call with phone to ear (this problem does not occur when on speaker or using earbuds or bluetooth unit...note...for tesing, mainly did not have bluetooth enabled)
7) One in 5 calls, after sufficient time with ear pressed to faceplate on call (typically 5 minutes), the screen becomes non-responsive.  Specifically:  1) pull phone from face; 2) screen lights up (suggesting this is not a sensor problem); 3) touch screen to either end call, add call, or if another call has come in (and this is the really annoying part when you are trying to conduct business on the phone) hold call and answer...any button...no responsiveness.  I notice that the buttons will sometimes show the depress animation, but that's it.  No application response...frozen, stuck on call. The only way to get out is for the caller to hang up or by depressing the top button on the phone...at which point the phone application says, "Call ending" in a flashing message.
As I said, this has happened on every single one of FIVE units (three replacements of the 4 and both the original and the genius bar replacement of the 4s). 
Note:  I was in the genius bar here in San Francisco, and actually demonstrated this problem to the genius.  He said, "Yeah, I see it...totally happening...never heard of this before.  Only thing I can do is replace your phone."  After five wasteful replacement activities, I'm starting to think that maybe this isn't a hardware problem and that replacing the phone isn't the problem. 
(Note- happened on both iOS5 and previous versions dating back to February.)
Frustrating.

Answer 1

I've had this happen twice today, and the first time it happened I was in an app, not on the phone at all!
However, I was listening to music with my headphones when friends came by. I pulled the headphones out of the jack to pause my music, talked to my friends, and when I went back to my iPhone, it was totally unresponsive to all buttons and screen touch. I did plug my headphones back in, and for some reason the music was turning on and off with the pause button on the headphones.
Later, I got a call from my mom. Oddly, I had pulled it out of the case for this call and was talking to her. She hung up, and my phone went totally unresponsive like earlier. After a minute or two, it did come back.
I've had the iPhone only since last night; I am not happy about this random unresponsive behavior! I do use the phone for work purposes, so not having access to it at all times is a real problem.