System panic

Question

Hi,
First of all, sorry my poor english. I've a SUN Fire V890 server with 8 UltraSPARC IV+ 2,2 GHz CPU in a SUN Cluster environment. Once the system crashed. I used CS CTE Automated Crash Analysis tool. This program generated som files. In the summary I found the reason of the panic. Here is a part of the file:
Release: SunOS 5.10 Generic_137111-07
Architecture: sun4u
Pagesize: 8192
CPU 0x7 is RUNNING READY CONFIGURED ENABLED ### CPU=7 (7) ###
PANIC occurred on this CPU
cpu addr 0x180c000
running thread addr 0x2a100655ca0
pause thread addr 0x2a1007a5ca0
dispatched thread addr 0x2a100655ca0
interrupt stack is 0x2a10007ff50
interrupt thread list starts at 0x2a10004fca0
UltraSPARC-IV+ (portid 7 impl 0x19 ver 0x22 clock 1800 MHz)
in_prom 0
PANIC occurred on this thread
Kernel thread:
thread addr 0x2a100655ca0, proc addr 0x1837880, lwp addr 0x0
Thread bound to cpu id 0x7
t_state is 0x4 - TS_ONPROC
Scheduling info:
t_pri is 0x3c, t_epri is 0x0, t_cid is 0x0
scheduling class is: SYS
t_disp_time: is 0xebde72, 0t15457906
last switched: 3 mins 48 secs ago, 1 ticks before panic on cpu 0x7
pc is 0x10667e4, sp is 0x18450a0, t_stk 0x2a100655a90
Stack trace is:
unix: setjmp ()
unix: panicsys+0x48 (0x10837e0,0x2a100655158,0x1875c00,0x2a100654c51,0x2a100655450,0x4414001607)
unix: vpanic ()
unix: panic+0x1c (0x10837e0,0x31,0x2a1006553b0,0x0,0x0,0x18bf374,0x181b710)
unix: die+0x78 (0x31,0x2a1006553b0,0x0,0x0)
unix: trap+0x9e0 (0x2a1006553b0)
unix: priv_rtt
vxio: volkiomem_seek+0x84 (0x29fffd80440,0x0)
SUNW,UltraSPARC-IV+: bcopy ()
vxio: volkio_to_kio_copy+0xe4 (0x30008c38ec8,0x2000?,0x30010812040,0x0,0x2000,0x0)
vxio: volsio_stabilize+0xe4 (0x300130c3800,0x300130c3948,0x2a100655a30)
vxio: vol_mv_write_start+0xb8c (0x300130c3800,0x2a100655a30)
vxio: voliod_iohandle+0x44 (0x300130c3800,0x1,0x2a100655a30)
vxio: voliod_loop+0x720 (0x0,0x0)
unix: thread_start+0x4 (0x0,0x0,0x0,0x0,0x0,0x0)
idlethread is 0x2a10001fca0
In the /var/adm/messages file I found this:
Notifying cluster that this node is panicking
panic[cpu7]/thread=2a100655ca0:
BAD TRAP: type=31 rp=2a1006553b0 addr=0 mmu_fsr=0 occurred in module "SUNW,UltraSPARC-IV+" due to a NULL pointer dereference
I noticed that an error was in the SUNW,UltraSPARC-IV+ module.What do you think, is there a hardware error (CPU) in this machine?

Answer 1

I'm going to guess that this was a software issue.
It's only a guess but the snippets you provided suggest the event was rooted in a wayward software thread involving ...
vxio: volkiomem_seek
SUNW,UltraSPARC-IV: bcopy ()+
vxio: volkio_to_kio_copy
vxio: volsio_stabilize
vxio: vol_mv_write_start
vxio: voliod_iohandle
vxio: voliod_loop
Those all suggest a thread that just got misdirected during an I/O write.
CPU #7 was simply where it happened thus CPU #7 reported it to you.
Back in the day, in a past incarnation, when I was in the Sun Support Center,
the Hardware support staff never touched panic event questions.
It was always analyzed by the Kernel team, even if it was eventually found to be a failed hardware component. We let `Kernel` figure it out.
Your best option would be to use your service contract and open a support case with your local Sun Solution Center, but you might also consider posting this same inquiry to the Kernel forum.
[http://forums.sun.com/forum.jspa?forumID=865|http://forums.sun.com/forum.jspa?forumID=865]