Mixer Brush "Sample All Layers" Lag

Question

pretty sure this is a common problem, when using the mixer brush and turning on "sample all layers", sometimes the lag is unbearable- way too slow.
Other times, it works just as fast as the brush tool for me - absolutely no lag at all.
I've tried narrowing it down to certain conditions within Photoshop (such as the current zoom level, how many layers there are, are there any adjustment layers visible, document size in terms of inches and dpi, and as well what other programs are running in the background (Firefox, etc), how much RAM is available, etc.
But I can never pinpoint when the lag happens or not.
Is this a known issue and if so, does anyone know what are the ideal conditions to have it work normally without any lag?

Answer 1

I don't know of any solution, but I think all your guesses are right.  It will slow depending upon all sort of factors including brush type and size.  Most the digital painters I know will work from a copy merged visible layer to avoid this.