Dbx: Unable to start the JVM

Question

I am trying to debug a Java class file using dbx. I would like to use this new feature that allows you switching between Java and C modes within the same debugger. However, when I try to execute command "debug myclass.class" in dbx, it complains that it is "Unable to start he Java virtual machine". The JVM that it attempts to start prints the message "(dbx) AgentThread: JVM_OnLoad(): debugger not attaching".
When I try to start the JVM by hand and attach to it with dbx, it attaches just fine, but then I cannot set any breakpoints in class files, because dbx does not seem to understand that they are Java classes: it attempts to treat them as C objects.
Thanks!

Answer 1

Dbx is not designed for debugging the jvm as such. While it is
possible to set breakpoints in jvm code (use "native stop ..." or
turn java mode off temporarily with "joff"), doing so could lead
to deadlocks. The way java debugging works in dbx is that
dbx makes function calls into the jvm to fetch java information
and to control the java threads. So a jvm breakpoint might be
hit when you do a (java) "threads" command in dbx, for example.
If you still wish to try it, you should be able to tell dbx to start
your java app using your debuggable jvm by setting the env
variable "jvm_invocation" to "/full/path/to/java -Xdebug" before
starting dbx. Then use "native step" or "joff" and "step" to step into
the jvm code.
Hope this helps!
Dave W.