Dbx rtc shared memory check

Question

Hi!
There are two programs (server + client ) which are interconnected via shared mem.
When Im trying to check client program with dbx( -C and check -access ), server is always dumping core in protocol classes of shmem and client is hanging on handshaking procedure. Is there is any way to disable DBX RTC(access checks ) to put traps or mark somehow the shared memory which is used by client program ?

Answer 1

Hi Leonard!
Threre is the problem with sources and stuff for server binary - it is "third side" software with no syms at all. so the core file is quite useless.
Ive found out the way to run correctly with server and mem access check - it is all about servers parameters about places(addr + page size rounding) of shmem and also Ive figured out another problem - there is an errors in server logs for semaphors and shmem lock calls. It looks like dbx is trapping whole mmap area from client.
Im still on investigation so I will post any usefull results here.
btw, skippatch is working quite strange actually...
Here is an example:
%dbx -C ./some_binary
(dbx)modules > ./loaded_modules
do some preparation stuff with output files so the file should looks like this( Ive ran with sed ):
rtc skippatch wait3.o
rtc skippatch wait4.o
rtc skippatch waitid.o
rtc skippatch waitpid.o
rtc skippatch walkstack.o
(dbx)check -access
(dbx) <copy/paste prepared file >
(dbx)run
theoretically, RTC should not work at all just because generated and pasted file contains all objects and librarys which is used by target binary.
but actually, RTC is patching and working as usual.
ps. 5.10 Generic_138888-01 sun4u sparc SUNW,Netra-T12 with dbx 7.7 ( from sunstudio 12.1 ).
pps. Ive read your short HOWTO about DBX RTC usage from the blog - very useful! Tommorow I will try to go with LD_AUDIT option.