Motion Tracking gets slower with longer clips until it stops responding

Question

Whenever I do feature tracking to stabilize a shot, the first few frames analyse in a split second per frame but then it gets slower and slower until it takes tens of seconds per frame and finally stops responding to user input at all.
I see the same probem on a 32GB Ram, MacOS 10.8 iMac 4.8GHz Core i7 with a 2GB CUDA capable NVidia card
(The biggest and fastest Mac you can buy since MacPros are no longer sold until the next generation)
and MacOS 10.7.5 on a MacBook with 16GB or RAM, 2.4GHz Core i5.
On the MacBook I had clips starting to track with 15fps for the first 5% but ultimately taking 5 days to track a 12 minute clip.
I have stage scenes of 10-15 minutes an am tracking the edges of the stage to remove slight camera shakes. Then exporting them to ProRes to use in an FCPX timeline (because there is no integration between Motion5 and FCPX at all and the trackig in FCPX doesnt allow for manually defined feature markers.).
Tracking anything below 3-5 minutes is okay.
Even when opening Motion5 again with the tracking data loaded it hardly reacts to user input when moving in the timeline or attempting to play the clip.
There is usually no other effect applied.
It's not like it is any more complex or requires any more memory to track the motion of the same feature between frames 5000 and 5001 and between frame 5 and frame 6.

Answer 1

3 clips of 5 minutes takes mintes to track.
a clip of 15 minutes take hours.