Tools for root cause analysis of stack corruption?

Question

I'm experiencing extremely rare stack corruption that results in SEGV core dumps for a large and complex C++ program. Having run all the standard tools such as IBM Rational Purify, Sun mdb/libumem and dbx/rtc (although unfortunately using dbx check -access takes inordinately long and eventually crashes before the main() function is executed), I am no closer to discovering the root cause of the stack corruption. I'm confident based on the tools run and on the stack trace from the core dump that the problem is not heap corruption, but stack corruption.
The environment is Sun Studio 12 (but not update 1) on Solaris 10 on SPARC. The program is compiled with minor optimisation (-xO2 -xbuiltin=%all).
Is anyone aware of other tools or approaches that could help pinpoint the problem? Your help would be much appreciated!
Thanks in advance,
Simon

Answer 1

If you are in fact having stack corruption issues, I don't think any of the tools you mentioned other than Purify would help you identify it.
You may also be simply running out of stack space, and not having corruption issues. Is your app multi-threaded? If so, you could increase the stack size your threads use to something larger than the default.
Another thing you can look for are syslog entries stating "no swap space to grow stack" for your process, you've run out of virtual memory. To avoid this, you can "pre-allocate" your stack memory with code similar to this:
void growStack( size_t bytes )
    char *mem = ( char * ) alloca( bytes );
    memset( mem, 0, bytes );
    return;
}That code, when called, will force the creation of stack memory virtual pages backed by swap, before your server gets into a situation where free memory might be in short supply.
I also seem to recall that Solaris under certain circumstances will allocate stack memory with the MAP_NORESERVE option, which means swap space won't be reserved for your stack. If your process gets swapped out, its stack(s) will be lost and you'll probably get a SIGSEGV or SIGBUS. See this bug:
[http://bugs.opensolaris.org/view_bug.do?bug_id=1221729|http://bugs.opensolaris.org/view_bug.do?bug_id=1221729]
I remember working a similar issue for a customer running large apps on Sun E15Ks, maybe about 5 years ago. To work around this behavoir, I think you'll need to explicitly allocate stack memory for any threads you may be creating. I think that's what we had to do.