IPMP failures on bge Interface

We've been testing IPMP on Solaris Sparc hosts that also have the Apani IPSec Agent installed. It works fine on older hosts that have 'qfe' and 'le' interfaces, but our v210's and T1000's with 'bge' interfaces have a problem. If we configure an IPMP group to use, say, bge0 and bge1 (with bge0 as the primary interface), it works fine. Disconnecting bge0 causes a failover to bge1, also fine. Disconnecting bge1 causes the following errors:
Nov 2 10:32:29 cs22 in.mpathd[146]: NIC failure detected on bge1 of group test
Nov 2 10:32:29 cs22 in.mpathd[146]: Successfully failed over from NIC bge1 to NIC bge0
Nov 2 10:32:37 cs2 in.mpathd[146]: All Interfaces in group test have failed
All interfaces fail, even though bge0 is still connected and was active before disconnecting bge1. The system recovers once bge0 is reconnected. The two interfaces are physically connected to the same switch, and the hostname.bgeX files are:
-------- hostname.bge0
cs22 netmask + broadcast + group test up \
addif cs21 deprecated -failover netmask + broadcast + up
-------- hostname.bge1
sp12 netmask + broadcast + group test up \
addif sp16 deprecated -failover netmask + broadcast + up
Any help would be appreciated, thanks in advance.

Hello again,
When gathering data for the previous reply, I also noticed that the default route had not been set. We usually do specify that, so I added that to the configuration. But, the host had found the correct router previously, it's 63.192.77.9. Specifying it did not change the problem symptoms, anyway. Here's the other requested info:
-> netstat -rn
Routing Table: IPv4
  Destination           Gateway           Flags  Ref   Use   Interface
63.192.77.0          63.192.77.12         U         1      5  bge1
63.192.77.0          63.192.77.22         U         1      1  bge0
63.192.77.0          63.192.77.22         U         1      0  bge0:1
63.192.77.0          63.192.77.12         U         1      0  bge1:1
224.0.0.0            63.192.77.22         U         1      0  bge0
default              63.192.77.9          UG        1      0 
127.0.0.1            127.0.0.1            UH        7     93  lo0
-> routeadm
              Configuration   Current              Current
                     Option   Configuration        System State
            IPv4 forwarding   disabled             disabled
               IPv4 routing   default (disabled)   disabled
            IPv6 forwarding   disabled             disabled
               IPv6 routing   disabled             disabled
        IPv4 routing daemon   "/usr/sbin/in.routed"
   IPv4 routing daemon args   ""
   IPv4 routing daemon stop   "kill -TERM `cat /var/tmp/in.routed.pid`"
        IPv6 routing daemon   "/usr/lib/inet/in.ripngd"
   IPv6 routing daemon args   "-s"
   IPv6 routing daemon stop   "kill -TERM `cat /var/tmp/in.ripngd.pid`"
r
-> arp -an
Net to Media Table: IPv4
Device   IP Address               Mask      Flags   Phys Addr
bge1   63.192.77.1          255.255.255.255       00:03:ba:c0:77:75
bge0   63.192.77.9          255.255.255.255       00:16:46:f1:b5:c2
bge1   63.192.77.9          255.255.255.255       00:16:46:f1:b5:c2
bge1   63.192.77.186        255.255.255.255       00:c0:4f:60:6a:ab
bge0   63.192.77.186        255.255.255.255       00:c0:4f:60:6a:ab
bge1   63.192.77.191        255.255.255.255       00:0c:f1:bf:1d:01
bge0   63.192.77.191        255.255.255.255       00:0c:f1:bf:1d:01
bge1   63.192.77.169        255.255.255.255       00:0c:f1:bf:1c:92
bge0   63.192.77.169        255.255.255.255       00:0c:f1:bf:1c:92
bge1   63.192.77.175        255.255.255.255       00:c0:4f:60:68:64
bge0   63.192.77.175        255.255.255.255       00:c0:4f:60:68:64
bge1   63.192.77.144        255.255.255.255       00:c0:4f:60:68:94
bge0   63.192.77.144        255.255.255.255       00:c0:4f:60:68:94
bge1   63.192.77.150        255.255.255.255       00:c0:4f:60:6a:70
bge0   63.192.77.150        255.255.255.255       00:c0:4f:60:6a:70
bge0   63.192.77.130        255.255.255.255       00:0c:f1:bf:1d:1f
bge1   63.192.77.130        255.255.255.255       00:0c:f1:bf:1d:1f
bge1   63.192.77.128        255.255.255.255       00:0c:f1:bf:1c:65
bge0   63.192.77.128        255.255.255.255       00:0c:f1:bf:1c:65
bge1   63.192.77.242        255.255.255.255       00:0d:56:0b:eb:2a
bge0   63.192.77.242        255.255.255.255       00:0d:56:0b:eb:2a
bge1   63.192.77.243        255.255.255.255       00:0f:1f:91:c1:9b
bge0   63.192.77.243        255.255.255.255       00:0f:1f:91:c1:9b
bge1   63.192.77.240        255.255.255.255       00:13:72:17:cb:13
bge0   63.192.77.240        255.255.255.255       00:13:72:17:cb:13
bge1   63.192.77.247        255.255.255.255       00:c0:4f:60:6a:e6
bge0   63.192.77.247        255.255.255.255       00:c0:4f:60:6a:e6
bge1   63.192.77.224        255.255.255.255       00:09:6b:2e:61:dd
bge0   63.192.77.224        255.255.255.255       00:09:6b:2e:61:dd
bge1   63.192.77.225        255.255.255.255       00:11:11:c4:9c:eb
bge0   63.192.77.225        255.255.255.255       00:11:11:c4:9c:eb
bge1   63.192.77.236        255.255.255.255       00:03:ba:eb:17:6d
bge0   63.192.77.236        255.255.255.255       00:03:ba:eb:17:6d
bge1   63.192.77.210        255.255.255.255       00:11:11:b1:2b:6e
bge0   63.192.77.210        255.255.255.255       00:11:11:b1:2b:6e
bge1   63.192.77.222        255.255.255.255       00:30:6e:08:ed:3a
bge0   63.192.77.222        255.255.255.255       00:30:6e:08:ed:3a
bge1   63.192.77.193        255.255.255.255       00:13:72:23:32:aa
bge0   63.192.77.193        255.255.255.255       00:13:72:23:32:aa
bge1   63.192.77.207        255.255.255.255       00:0c:f1:b6:26:aa
bge0   63.192.77.207        255.255.255.255       00:0c:f1:b6:26:aa
bge1   63.192.77.204        255.255.255.255       00:c0:4f:60:68:5b
bge0   63.192.77.204        255.255.255.255       00:c0:4f:60:68:5b
bge1   63.192.77.48         255.255.255.255       00:0a:95:99:e4:40
bge0   63.192.77.48         255.255.255.255       00:0a:95:99:e4:40
bge0   63.192.77.49         255.255.255.255       00:03:93:90:52:f6
bge1   63.192.77.61         255.255.255.255       00:c0:4f:60:6a:75
bge0   63.192.77.61         255.255.255.255       00:c0:4f:60:6a:75
bge1   63.192.77.35         255.255.255.255       00:30:6e:49:41:50
bge0   63.192.77.35         255.255.255.255       00:30:6e:49:41:50
bge1   63.192.77.36         255.255.255.255       00:16:35:3e:7d:0a
bge0   63.192.77.36         255.255.255.255       00:16:35:3e:7d:0a
bge0   63.192.77.42         255.255.255.255       00:11:11:c4:9d:05
bge1   63.192.77.42         255.255.255.255       00:11:11:c4:9d:05
bge1   63.192.77.40         255.255.255.255       00:0c:f1:bf:1f:8d
bge0   63.192.77.40         255.255.255.255       00:0c:f1:bf:1f:8d
bge1   63.192.77.41         255.255.255.255       00:0c:f1:bf:1d:10
bge0   63.192.77.41         255.255.255.255       00:0c:f1:bf:1d:10
bge0   63.192.77.19         255.255.255.255       08:00:20:f0:ea:e4
bge1   63.192.77.19         255.255.255.255       08:00:20:f0:ea:e4
bge1   63.192.77.16         255.255.255.255 SP    00:14:4f:2a:9b:83
bge0   63.192.77.22         255.255.255.255 SP    00:14:4f:2a:9b:82
bge0   63.192.77.23         255.255.255.255       00:09:6b:3e:2b:82
bge1   63.192.77.23         255.255.255.255       00:09:6b:3e:2b:82
bge0   63.192.77.21         255.255.255.255 SP    00:14:4f:2a:9b:82
bge1   63.192.77.29         255.255.255.255       00:09:6b:2e:46:51
bge0   63.192.77.29         255.255.255.255       00:09:6b:2e:46:51
bge0   63.192.77.1          255.255.255.255       00:03:ba:c0:77:75
bge1   63.192.77.12         255.255.255.255 SP    00:14:4f:2a:9b:83
bge0   63.192.77.115        255.255.255.255       00:0c:f1:bf:1c:e6
bge1   63.192.77.115        255.255.255.255       00:0c:f1:bf:1c:e6
bge1   63.192.77.122        255.255.255.255       00:10:83:f9:34:d4
bge0   63.192.77.122        255.255.255.255       00:10:83:f9:34:d4
bge1   63.192.77.125        255.255.255.255       00:0f:1f:91:bf:7d
bge0   63.192.77.125        255.255.255.255       00:0f:1f:91:bf:7d
bge1   63.192.77.99         255.255.255.255       00:0c:f1:bf:1a:52
bge0   63.192.77.99         255.255.255.255       00:0c:f1:bf:1a:52
bge1   63.192.77.100        255.255.255.255       00:0c:f1:b6:26:b4
bge0   63.192.77.100        255.255.255.255       00:0c:f1:b6:26:b4
bge1   63.192.77.101        255.255.255.255       00:0c:f1:bf:1c:fe
bge0   63.192.77.101        255.255.255.255       00:0c:f1:bf:1c:fe
bge1   63.192.77.107        255.255.255.255       00:0d:56:14:48:4d
bge0   63.192.77.107        255.255.255.255       00:0d:56:14:48:4d
bge1   63.192.77.110        255.255.255.255       00:c0:4f:60:6a:44
bge0   63.192.77.110        255.255.255.255       00:c0:4f:60:6a:44
bge1   63.192.77.108        255.255.255.255       00:14:bf:31:ec:e2
bge0   63.192.77.108        255.255.255.255       00:14:bf:31:ec:e2
bge0   63.192.77.80         255.255.255.255       00:16:cb:a6:5e:3d
bge1   63.192.77.80         255.255.255.255       00:16:cb:a6:5e:3d
bge1   63.192.77.92         255.255.255.255       00:40:63:d3:8c:46
bge0   63.192.77.92         255.255.255.255       00:40:63:d3:8c:46
bge1   63.192.77.68         255.255.255.255       00:0c:f1:b6:27:10
bge0   63.192.77.68         255.255.255.255       00:0c:f1:b6:27:10
bge1   63.192.77.69         255.255.255.255       00:13:72:17:ca:4a
bge0   63.192.77.69         255.255.255.255       00:13:72:17:ca:4a
bge1   63.192.77.73         255.255.255.255       00:03:93:d1:db:cc
bge0   63.192.77.73         255.255.255.255       00:03:93:d1:db:cc
bge1   63.192.77.77         255.255.255.255       00:30:65:a8:22:bc
bge0   63.192.77.77         255.255.255.255       00:30:65:a8:22:bc
bge1   224.0.0.0            240.0.0.0       SM    01:00:5e:00:00:00
bge0   224.0.0.0            240.0.0.0       SM    01:00:5e:00:00:00
-> ps -aef
     UID   PID  PPID   C    STIME TTY         TIME CMD
    root     0     0   0 15:11:12 ?           0:11 sched
    root     1     0   0 15:11:13 ?           0:00 /sbin/init
    root     2     0   0 15:11:13 ?           0:00 pageout
    root     3     0   0 15:11:13 ?           0:00 fsflush
  daemon   196     1   0 15:11:37 ?           0:00 /usr/sbin/rpcbind
    root     7     1   0 15:11:15 ?           0:10 /lib/svc/bin/svc.startd
    root     9     1   0 15:11:16 ?           0:16 /lib/svc/bin/svc.configd
    root   256     1   0 15:11:40 ?           0:00 /usr/sbin/cron
    root   335     1   0 15:11:49 ?           0:00 /usr/sbin/syslogd
    root   113     1   0 15:11:33 ?           0:00 /usr/sbin/nscd -S passwd,yes
    root   726   691   0 15:16:16 pts/1       0:00 ps -aef
  daemon   201     1   0 15:11:37 ?           0:00 /usr/lib/nfs/statd
    root   200     1   0 15:11:37 ?           0:00 /usr/sbin/keyserv
    root   192     1   0 15:11:36 ?           0:01 /opt/apani/uagent/nlagent
  daemon    86     1   0 15:11:26 ?           0:00 /usr/lib/crypto/kcfd
    root   152     1   0 15:11:35 ?           0:00 /usr/lib/inet/in.mpathd -a
    root   212     7   0 15:11:38 ?           0:00 /usr/lib/saf/sac -t 300
    root    89     1   0 15:11:26 ?           0:00 /usr/lib/picl/picld
  daemon   247     1   0 15:11:40 ?           0:00 /usr/lib/nfs/nfs4cbd
    root   102     1   0 15:11:28 ?           0:00 /usr/lib/power/powerd
    root    98     1   0 15:11:27 ?           0:00 /usr/lib/sysevent/syseventd
    root   215     1   0 15:11:38 ?           0:00 /usr/sbin/nis_cachemgr
  daemon   214     1   0 15:11:38 ?           0:00 /usr/lib/nfs/lockd
    root   213     1   0 15:11:38 ?           0:00 /usr/lib/utmpd
    root   217     7   0 15:11:38 console     0:00 -sh
    root   223   192   0 15:11:39 ?           0:00 inm -p9165
    root   222   212   0 15:11:39 ?           0:00 /usr/lib/saf/ttymon
  daemon   255     1   0 15:11:40 ?           0:00 /usr/lib/nfs/nfsmapid
    root   399   397   0 15:11:52 ?           0:00 /usr/sadm/lib/smc/bin/smcboot
    root   252     1   0 15:11:40 ?           0:04 /usr/lib/inet/inetd start
    root   398   397   0 15:11:52 ?           0:00 /usr/sadm/lib/smc/bin/smcboot
    root   317     1   0 15:11:48 ?           0:00 /usr/lib/autofs/automountd
    root   359     1   0 15:11:50 ?           0:00 /usr/lib/sendmail -bd -q15m
    root   448   447   0 15:11:53 ?           0:00 /usr/lib/locale/ja/wnn/jserver_m
    root   351     1   0 15:11:50 ?           0:02 /usr/lib/fm/fmd/fmd
    root   674   252   0 15:12:14 ?           0:00 /usr/sbin/in.telnetd
    root   347     1   0 15:11:50 ?           0:00 /usr/lib/ssh/sshd
   smmsp   360     1   0 15:11:50 ?           0:00 /usr/lib/sendmail -Ac -q15m
    root   461     1   0 15:11:53 ?           0:00 /usr/lib/locale/ja/atokserver/atokmngdaemon
    root   397     1   0 15:11:52 ?           0:00 /usr/sadm/lib/smc/bin/smcboot
    root   468   459   0 15:11:53 ?           0:00 htt_server -port 9010 -syslog -message_locale C
    root   441     1   0 15:11:53 ?           0:00 /usr/lib/locale/ja/wnn/dpkeyserv
    root   447     1   0 15:11:53 ?           0:00 /usr/lib/locale/ja/wnn/jserver
    root   459     1   0 15:11:53 ?           0:00 /usr/lib/im/htt -port 9010 -syslog -message_locale C
    root   512     1   0 15:11:55 ?           0:00 /usr/lib/snmp/snmpdx -y -c /etc/snmp/conf
    root   520     1   0 15:11:56 ?           0:00 /usr/lib/dmi/dmispd
    root   528     1   0 15:11:56 ?           0:00 /usr/sbin/vold
    root   521     1   0 15:11:56 ?           0:00 /usr/lib/dmi/snmpXdmid -s cstoc77022
    root   511     1   0 15:11:55 ?           0:00 /usr/dt/bin/dtlogin -daemon
    root   691   677   0 15:12:18 pts/1       0:00 bash
    root   677   674   0 15:12:14 pts/1       0:00 -sh
    root   585     1   0 15:11:57 ?           0:00 /usr/sfw/sbin/snmpd

Similar Messages

  • Solaris 10 u11 : Is it possible to mix IPMP with vertual network interface

    Hi
    I have a system with 4 Ethernet interfaces , I'd like to try to have virtual network interface on two of them . then configure IPMP between each two.
    - Is it possible to mix IPMP and VNIC ?
    - online , I found posts about having vnic on solaris by creating additional interface file (ex, /etc/hostname.bge0:1 ) and also the IPMP create such interface?
    - How can I know which IPMP version installed on the system ? and how to know I'm reading the right configuration guide?
    - on what basis should i choose the type of IPMP for my system (Link-based or Probe-based)
    Thanks in advance
    HEBA

    Hi.
    IPMP can coexist with VNIC.
    IPMP - part of Solaris. So you not need check version of IPMP. Just read Docs for Solaris.
    http://docs.oracle.com/cd/E23823_01/html/816-4554/ipmptm-1.html#scrolltoc
    It's not clear what Vnic you plan use. Is it VLAN or just additional IP on interface ?
    For test address of IPMP you should configure interface like bge0:1 with additional options.
    Regards.

  • Set up IPMP Solaris 10 -- two interfaces, one IP

    I have a tasking to set up failure-based IPMP on a T5120. I have been reading all the Sun documentation on setting up IPMP, but cannot find exactly what I am looking for. I have one IP address, and two connected NICs, and my task is to set up IPMP so that if e1000g0 fails, e1000g1 will take over. Is this possible, and if so, how?

    If you have two interfaces and only want link-based failure detection, just put the group $YOUR_GROUPNAME statement in /etc/hostname.$INTERFACE file.
    Say you have the two interfaces e1000g0 and e1000g1, your hostname is MyHostname and your group is MyGroup you would do the following:
    put
    MyHostname group MyGroup in /etc/hostname.e1000g0.
    Put group MyGroup in /etc/hostname.e1000g1.
    Either reboot the machine or manually configure ipmp:
    ifconfig e1000g0 group MyGroup
    ifconfig e1000g1 plumb group MyGroup up
    in /var/adm/messages you there should be an info that no test-adress was given and that ipmp will operate in link-failure detection mode only.
    Please note that officially you should create an ipmp instance first by issuing something like ifconfig $MyIPMP-Instance group MyGroup, but that step could be left out as ipmp instances are created implicitly.

  • IPMP Failure

    Hello buddies,
    We are facing a severe issues with the IPMP configuration that was working fine till the last patch ( Feb Patch)installation where we have virtual IP 192.168.1.1 and two physical IPs 192.168.1.2 & 192.168.1.3 that binded on ce0 and ce1 respectively. Let me come to the issue... All interfaces in the iPMP group are getting down ( three times in a day) and complaing that the default router is not pingable by the server through the two physical links. I am pretty sure that the switch where both links are connected are working fine without any outage. The follwing are the logs
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 594170 daemon.error] NIC failure detected on ce1 of group ipmp-pub
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 594170 daemon.error] NIC failure detected on ce1 of group ipmp-pub
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 594170 daemon.error] NIC failure detected on ce1 of group ipmp-pub
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 832587 daemon.error] Successfully failed over from NIC ce1 to NIC ce0
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 832587 daemon.error] Successfully failed over from NIC ce1 to NIC ce0
    Mar 15 19:54:13 serv1 in.mpathd[2095]: [ID 832587 daemon.error] Successfully failed over from NIC ce1 to NIC ce0
    Mar 15 19:54:39 serv1 in.mpathd[2095]: [ID 168056 daemon.error] All Interfaces in group ipmp-pub have failed
    Mar 15 19:54:39 serv1 in.mpathd[2095]: [ID 168056 daemon.error] All Interfaces in group ipmp-pub have failed
    Mar 15 19:54:39 serv1 in.mpathd[2095]: [ID 168056 daemon.error] All Interfaces in group ipmp-pub have failed
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 620804 daemon.error] Successfully failed back to NIC ce1
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 620804 daemon.error] Successfully failed back to NIC ce1
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 620804 daemon.error] Successfully failed back to NIC ce1
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 299542 daemon.error] NIC repair detected on ce1 of group ipmp-pub
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 299542 daemon.error] NIC repair detected on ce1 of group ipmp-pub
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 299542 daemon.error] NIC repair detected on ce1 of group ipmp-pub
    Mar 15 20:33:32 serv1 in.mpathd[2095]: [ID 237757 daemon.error] At least 1 interface (ce1) of group ipmp-pub has repaired
    Any help would be great appreciable.
    Thanks,
    Muhammed Afsal K.S

    lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1
         inet 127.0.0.1 netmask ff000000
    ce0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
         inet 192.168.217.6 netmask ffffff00 broadcast 192.168.217.255
         groupname ipmp1
         ether 0:3:ba:b0:5d:54
    ce3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
         inet 192.168.217.20 netmask ffffff00 broadcast 192.168.217.255
         groupname ipmp1
         ether 0:3:ba:95:5d:6e
    ce3:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4
         inet 192.168.217.4 netmask ffffff00 broadcast 192.168.217.255
    General speaking,
    When I switch float IP from ce0 to ce3, IPMP will say ce0 MAC is "trying to be our address ....", then ce0 test IP failed, FLOAT IP didn't failover.
    When I switch float IP from ce3 to ce0, IPMP will say ce3 MAC is "trying to be our address ....",
    then ce0 test IP failed, FLOAT IP didn't failover.
    In my viewpoint, float NIC MAC & address information may be cached in CICSO device's RAM, not released in time, I think.

  • Trunking setup on v240 with bge interfaces

    I'm trying to get trunking installed on a v240 with built in nic's. bge0 - 4 to be exact. I'm using trunking 1.3 and I've verified all the packages are installed. I've edited nettr.sh to reflect these cards. I get the following error when I try to run it.
    ./nettr: Apr 20 12:20:26 me1-backup trslm: NOTICE: trunk link (bge0): DL_NOTIFY_ACK: Aggregation capability not supported: Notifications = 0x11f
    ioctl failed: I/O error.
    Question: Does trunking 1.3 support bge? The docs seem to indicate it doesn't but I thought I'd ask.
    Main Question: Is there something the does support trunking on bge?
    Thanks in advance.

    My records say that bge is not supported with Sun Trunking, but since it is a gld3 driver, it should be supported by link aggregation on Solaris 10.
    Darren

  • SUN Cluster 3.2, Solaris 10, Corrupted IPMP group on one node.

    Hello folks,
    I recently made a network change on nodename2 to add some resilience to IPMP (adding a second interface but still using a single IP address).
    After a reboot, I cannot keep this host from rebooting. For the one minute that it stays up, I do get the following result from scstat that seems to suggest a problem with the IPMP configuration. I rolled back my IPMP change, but it still doesn't seem to register the IPMP group in scstat.
    nodename2|/#scstat
    -- Cluster Nodes --
    Node name Status
    Cluster node: nodename1 Online
    Cluster node: nodename2 Online
    -- Cluster Transport Paths --
    Endpoint Endpoint Status
    Transport path: nodename1:bge3 nodename2:bge3 Path online
    -- Quorum Summary from latest node reconfiguration --
    Quorum votes possible: 3
    Quorum votes needed: 2
    Quorum votes present: 3
    -- Quorum Votes by Node (current status) --
    Node Name Present Possible Status
    Node votes: nodename1 1 1 Online
    Node votes: nodename2 1 1 Online
    -- Quorum Votes by Device (current status) --
    Device Name Present Possible Status
    Device votes: /dev/did/rdsk/d3s2 0 1 Offline
    -- Device Group Servers --
    Device Group Primary Secondary
    Device group servers: jms-ds nodename1 nodename2
    -- Device Group Status --
    Device Group Status
    Device group status: jms-ds Online
    -- Multi-owner Device Groups --
    Device Group Online Status
    -- IPMP Groups --
    Node Name Group Status Adapter Status
    scstat:  unexpected error.
    I did manage to run scstat on nodename1 while nodename2 was still up between reboots, here is that result (it does not show any IPMP group(s) on nodename2)
    nodename1|/#scstat
    -- Cluster Nodes --
    Node name Status
    Cluster node: nodename1 Online
    Cluster node: nodename2 Online
    -- Cluster Transport Paths --
    Endpoint Endpoint Status
    Transport path: nodename1:bge3 nodename2:bge3 faulted
    -- Quorum Summary from latest node reconfiguration --
    Quorum votes possible: 3
    Quorum votes needed: 2
    Quorum votes present: 3
    -- Quorum Votes by Node (current status) --
    Node Name Present Possible Status
    Node votes: nodename1 1 1 Online
    Node votes: nodename2 1 1 Online
    -- Quorum Votes by Device (current status) --
    Device Name Present Possible Status
    Device votes: /dev/did/rdsk/d3s2 1 1 Online
    -- Device Group Servers --
    Device Group Primary Secondary
    Device group servers: jms-ds nodename1 -
    -- Device Group Status --
    Device Group Status
    Device group status: jms-ds Degraded
    -- Multi-owner Device Groups --
    Device Group Online Status
    -- IPMP Groups --
    Node Name Group Status Adapter Status
    IPMP Group: nodename1 sc_ipmp1 Online bge2 Online
    IPMP Group: nodename1 sc_ipmp0 Online bge0 Online
    -- IPMP Groups in Zones --
    Zone Name Group Status Adapter Status
    I believe that I should be able to delete the IPMP group for the second node from the cluster and re-add it, but I'm sure about how to go about doing this. I welcome your comments or thoughts on what I can try before rebuilding this node from scratch.
    -AG

    I was able to restart both sides of the cluster. Now both sides are online, but neither side can access the shared disk.
    Lots of warnings. I will keep poking....
    Rebooting with command: boot
    Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a File and args:
    SunOS Release 5.10 Version Generic_141444-09 64-bit
    Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
    Use is subject to license terms.
    Hardware watchdog enabled
    Hostname: nodename2
    Jul 21 10:00:16 in.mpathd[221]: No test address configured on interface ce3; disabling probe-based failure detection on it
    Jul 21 10:00:16 in.mpathd[221]: No test address configured on interface bge0; disabling probe-based failure detection on it
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],0:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],0:c,raw".
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],1:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],1:c,raw".
    Booting as part of a cluster
    NOTICE: CMM: Node nodename1 (nodeid = 1) with votecount = 1 added.
    NOTICE: CMM: Node nodename2 (nodeid = 2) with votecount = 1 added.
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    NOTICE: clcomm: Adapter bge3 constructed
    NOTICE: CMM: Node nodename2: attempting to join cluster.
    NOTICE: CMM: Node nodename1 (nodeid: 1, incarnation #: 1279727883) has become reachable.
    NOTICE: clcomm: Path nodename2:bge3 - nodename1:bge3 online
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    NOTICE: CMM: Cluster has reached quorum.
    NOTICE: CMM: Node nodename1 (nodeid = 1) is up; new incarnation number = 1279727883.
    NOTICE: CMM: Node nodename2 (nodeid = 2) is up; new incarnation number = 1279728026.
    NOTICE: CMM: Cluster members: nodename1 nodename2.
    NOTICE: CMM: node reconfiguration #3 completed.
    NOTICE: CMM: Node nodename2: joined cluster.
    NOTICE: CCR: Waiting for repository synchronization to finish.
    WARNING: CCR: Invalid CCR table : dcs_service_9 cluster global.
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    ip: joining multicasts failed (18) on clprivnet0 - will use link layer broadcasts for multicast
    ==> WARNING: DCS: Error looking up services table
    ==> WARNING: DCS: Error initializing service 9 from file
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],0:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],0:c,raw".
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],1:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],1:c,raw".
    /dev/md/rdsk/d22 is clean
    Reading ZFS config: done.
    NOTICE: iscsi session(6) iqn.1994-12.com.promise.iscsiarray2 online
    nodename2 console login: obtaining access to all attached disks
    starting NetWorker daemons:
    Rebooting with command: boot
    Boot device: /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0:a File and args:
    SunOS Release 5.10 Version Generic_141444-09 64-bit
    Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
    Use is subject to license terms.
    Hardware watchdog enabled
    Hostname: nodename1
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],0:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],0:c,raw".
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],1:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],1:c,raw".
    Booting as part of a cluster
    NOTICE: CMM: Node nodename1 (nodeid = 1) with votecount = 1 added.
    NOTICE: CMM: Node nodename2 (nodeid = 2) with votecount = 1 added.
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    NOTICE: clcomm: Adapter bge3 constructed
    NOTICE: CMM: Node nodename1: attempting to join cluster.
    NOTICE: bge3: link up 1000Mbps Full-Duplex
    NOTICE: clcomm: Path nodename1:bge3 - nodename2:bge3 errors during initiation
    WARNING: Path nodename1:bge3 - nodename2:bge3 initiation encountered errors, errno = 62. Remote node may be down or unreachable through this path.
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    NOTICE: CMM: Cluster doesn't have operational quorum yet; waiting for quorum.
    NOTICE: bge3: link down
    NOTICE: bge3: link up 1000Mbps Full-Duplex
    NOTICE: CMM: Node nodename2 (nodeid: 2, incarnation #: 1279728026) has become reachable.
    NOTICE: clcomm: Path nodename1:bge3 - nodename2:bge3 online
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    NOTICE: CMM: Cluster has reached quorum.
    NOTICE: CMM: Node nodename1 (nodeid = 1) is up; new incarnation number = 1279727883.
    NOTICE: CMM: Node nodename2 (nodeid = 2) is up; new incarnation number = 1279728026.
    NOTICE: CMM: Cluster members: nodename1 nodename2.
    NOTICE: CMM: node reconfiguration #3 completed.
    NOTICE: CMM: Node nodename1: joined cluster.
    WARNING: CMM: Open failed for quorum device /dev/did/rdsk/d3s2 with error 2.
    ip: joining multicasts failed (18) on clprivnet0 - will use link layer broadcasts for multicast
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],0:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],0:c,raw".
    /usr/cluster/bin/scdidadm: Could not stat "../../devices/iscsi/[email protected],1:c,raw" - No such file or directory.
    Warning: Path node loaded - "../../devices/iscsi/[email protected],1:c,raw".
    /dev/md/rdsk/d26 is clean
    Reading ZFS config: done.
    NOTICE: iscsi session(6) iqn.1994-12.com.promise.iscsiarray2 online
    nodename1 console login: obtaining access to all attached disks
    starting NetWorker daemons:
    nsrexecd
    mount: /dev/md/jms-ds/dsk/d100 is already mounted or /opt/esbshares is busy

  • Configure ce interface to accept all frames (regardless of 802.1q tag)?

    On Solaris 9,
    is it possible to configure a ce interface so it accepts
    all Ethernet frames it receives, regardless of the presence,
    absence, or value of an 802.1q tag?
    I'm trying to run packet sniffing software on a ce interface
    (I'm not using the interface for IP). The Ethernet switch
    I'm attached to forwards to me copies of traffic from many
    different VLANs, via a port mirroring function. Most of
    this traffic is tagged with various 802.1q VLAN tags.
    Some of this traffic may be untagged.
    My difficulty is that since the Sun Gigaswift 'ce' interface
    supports 802.1q, if I configure the interface so it
    doesn't "do" VLANs (e.g. 'ifconfig ce5 plumb'), the ce interface
    discards any incoming frames that have 802.1q tags.
    Alternatively, if I configure the ce interface so it "does"
    VLANs (e.g. 'ifconfig ce10005 plumb'), I end up having to
    configure dozens of logical ce interfaces (one per VLAN) each
    of which will only receive frames tagged for one particular VLAN;
    I don't end up with an interface that receives all VLAN frames.
    I want one logical interface that receives all the frames,
    so I can point my sniffing software at it.
    There's no problem doing this with a card that doesn't support
    802.1q, since that card will happily accept all frames and pass
    them up. So I can do this on qfe or bge interfaces. But I need
    to do this using interfaces available on a PCI card
    (my V880 has several port mirrors coming from various Ethernet switches),
    the interfaces need to be 1000 Mbps (which rules out the qfe card), and
    I'd like to stay with copper (which rules out the bge card).
    That's why I'm trying to do it with the quad ce card, but I'm
    stymied by it support for 802.1q. (If there were just some way
    I could turn off 802.1q support for specific ce instances.)
    Is there a way I can accomplish this with the
    copper Gigaswift interface (Cassini driver) in Solaris 9?
    If not, is there another 1000 Mbps copper interface (provided on PCI card)
    that can do this?
    Or does this become possible in Solaris 10?

    Oops, when I wrote "bge" above, I meant "ge", of course.
    (ge is fiber gigagbit ethernet interface and doesn't know about 802.1q.)

  • Trap on a link failure

    does someone know how to enable a snmp trap if an interface link goes down?
    There are 4 bge interfaces, the IP of the SNMP Manager configured. In the /var/adm/messages can be seen that a link goes down, but no snmp traps/notifications are being sent.
    Thanks in advance.

    Thanks John thanks Snehal.
    There a change in my work. Regarding on linking.
    My report must now have, a column with cells with 3 links inside it.
    Is there any code for having 3 link names in one cell.
    For example:
    The cell must contains:
    Link1 Link2 Link3
    which has a corresponding report to be linked. Is this possible.
    ="<a href=http://servername/businessobjects/enterprise115/desktoplaunch/opendoc/openDocument.jsp?sDocType=wid&sDocName=Report1'>Link1</a><a href=http://servername/businessobjects/enterprise115/desktoplaunch/opendoc/openDocument.jsp?sDocType=wid&sDocName=Report2>Link2</a>"
    It only displays Link1 but I need to display Link2 as well. Can anyone help me with this? Thanks.

  • Local zone using IPMP in global zone

    Hi all,
    I've installed a zone and when I'm booting it, I receive the following error :
    bge0:2: could not bring interface up: address in use by zone 'global': Cannot assign requested addres
    The bge2 is used in my global zone (member of IPMP group of 2 interfaces : bge0 - ce0).
    Is there a way to use the IPMP mecanism from my global zone to my local zone ?
    Something like :
    add net
    set address = xxx.xxx.xxx.xxx
    set physical = IPMP_group (instead of a physical interface)
    OR must I unplumb the bge0 in the global zone in order to use it in local zone ?
    Thanks for the help or the advices !
    Quentin.

    I think I won the 10 duke stars that I was giving for this topic ...
    I didn't see that the IP/interface I wanted to configure with my local-zone was already plumbed in global zone from a previous failed zone boot.
    I've unplumbed the logical interface in global zone and I've booted successfully the zone.

  • Zone with multiple logical interfaces

    Hi,
    How can multiple logical network interfaces be added to a running zone?
    I have configured and installed a whole root zone with one shared-ip network interface.
    Now, I need to add more logical interfaces to the same zone.
    On a physical server with a bge interface, I would create a /etc/hostname file for each logical interface such as
    $ ls /etc | grep host
    hostname.bge0
    hostname.bge0:1
    hostname.bge0:2
    hostname.bge0:3
    hostname.bge0:4
    hostname.bge1
    hostname.bge1:1
    hostname.bge1:2
    hostname.bge1:3
    hostname.bge3
    hosts
    $How can the above be done for a zone?

    Hi
    This requires 2 steps. Firstly update your zone configuration to add the logical interface and its associated IP address.
    Assuming from below you want the logical interface to be added to bge0 then do the following replacing the zonename and ip address for whatever is appropriate for you.
    # zonecfg -z itchyzone
    zonecfg:itchyzone1> add net
    zonecfg:itchyzone1:net> set address=192.168.1.21
    zonecfg:itchyzone1:net> set physical=bge0
    zonecfg:itchyzone1:net> end
    zonecfg:itchyzone1> exitYou can do the above as many times as you like to create multiple logical interfaces for the zone.
    After doing the above you will need to reboot the zone to get the new logical interface.
    However if you cant reboot the zone you can plumb a logical interface into the zone by running something similar to the following (change for your zonename, IP etc) from the global zone:
    # ifconfig bge0 addif 192.168.1.21 netmask + broadcast + zone itchyzone1 upyou will now have an extra interface in the zone (in this case called itchyzone1). To verify, login to the zone and run ifconfig -a and you will see your new interface.
    # zlogin itchyzone1
    [Connected to zone 'itchyzone1' pts/4]
    Last login: Mon Oct  5 22:24:15 on pts/4
    Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
    # ifconfig -a
    lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
            inet 127.0.0.1 netmask ff000000
    bge0:2: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
            inet 192.168.1.20 netmask ffffff00 broadcast 192.168.1.255
    bge0:3: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
            inet 192.168.1.21 netmask ffffff00 broadcast 192.168.1.255Hope this helps
    Martin

  • IPMP without Global Zone ip's

    I have a box with 4 Network adapters. Two are on a backend network segment for internal admin traffic only. Another two are on a Front end network for Client facing apps.
    In the Global zone I only want to have the backend interfaces configured and up. In the client faceing non-global zone I wish to bring up an interface that can take advatnage of IPMP between the two interface on the front end segment.
    Does anybody know how I can do this without haveing to bring the the Front end interfaces up in the Global Zone?

    I finally got around to testing this out and it looks like their is a bug.
    I plumbed the two interfaces in the Global zone without any IP's and added placed them both in the same group.
    I have two zones one with an IP bound to one interface and the other with an ip bound to the other interface.
    ce6000: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
    groupname prodfe
    ether 0:3:ba:5b:4:a1
    ce6000:1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    zone mishmash
    inet 10.121.6.30 netmask ffffff00 broadcast 10.121.6.255
    ce6004: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 8
    inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
    groupname prodfe
    ether 0:3:ba:78:25:ea
    ce6004:1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 8
    zone sorryServer
    inet 10.121.6.31 netmask ffffff00 broadcast 10.121.6.255
    The first time the interface fails, the IP moves over to the other interface without a problem.
    ce6000: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
    groupname prodfe
    ether 0:3:ba:5b:4:a1
    ce6000:1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    zone mishmash
    inet 10.121.6.30 netmask ffffff00 broadcast 10.121.6.255
    ce6000:2: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
    ce6000:3: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    zone sorryServer
    inet 10.121.6.31 netmask ffffff00 broadcast 10.121.6.255
    ce6004: flags=219000802<BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED,CoS> mtu 0 index 8
    inet 0.0.0.0 netmask 0
    groupname prodfe
    ether 0:3:ba:78:25:ea
    When it goes to fail the link back though the Zone ip becomes the Primary IP for the interface. Which then causes a bunch of other problems especilly when you try to shut the zone down. And if the link fails again the interface will on longer failover.
    ce6000: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
    groupname prodfe
    ether 0:3:ba:5b:4:a1
    ce6000:1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 7
    zone mishmash
    inet 10.121.6.30 netmask ffffff00 broadcast 10.121.6.255
    ce6004: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 8
    zone sorryServer
    inet 10.121.6.31 netmask ffffff00 broadcast 10.121.6.255
    groupname prodfe
    ether 0:3:ba:78:25:ea

  • IPMP getting [ID 390400 kern.notice] in messages file

    After implementing IPMP on two bge ports we are getting 4 of the following messages every 30 minutes :
    ip: [ID 390400 kern.notice] dst FFFFFFFF src 7F000001
    The IPMP setup works OK with failover to either port
    We are using Link status only (no test addresses) and I suspect the message relates to the fact the second port does not have an IP address associated with it
    ifconfig shows its address as 0.0.0.0
    Does anyone know if there is a way of stopping the messages.
    Regards
    Ian

    From the sync manpages:
    <snip>SYNOPSIS
    sync
    DESCRIPTION
    sync executes the sync system primitive. If the system is to
    be stopped, sync must be called to insure file system
    integrity. It will flush all previously unwritten system
    buffers out to disk, thus assuring that all file modifica-
    tions up to that point will be saved. See sync(2) for
    details.
    </snip>
    From the sync(2) man pages:
    <snip>
    NAME
    sync - update super block
    SYNOPSIS
    #include <unistd.h>
    void sync(void);
    DESCRIPTION
    The sync() function writes all information in memory that
    should be on disk, including modified super blocks, modified
    inodes, and delayed block I/O.
    Unlike fsync(3C), which completes the writing before it
    returns, sync() schedules but does not necessarily complete
    the writing before returning.
    USAGE
    The sync() function should be used by applications that
    examine a file system, such as fsck(1M), and df(1M), and is
    mandatory before rebooting.
    </snip>
    In case of a system crash, you would have to sync the buffers to disk to ensure data integrity (see above). The discrepancy (of unsync'ed disks) leads the system to start "fsck"ing the filesystem(s).

  • Cluster node reboots after network failure

    hi all,
    The suncluster 3.1 8/05 with 2 nodes (E2900) was working fine without any errors in the sccheck.
    yesterday one node rebooted saying a network failure,errors in the massage file are
    Jan 17 08:00:36 PRD in.mpathd[221]: [ID 594170 daemon.error] NIC failure detected on ce0 of group sc_ipmp0
    Jan 17 08:00:36 PRD Cluster.PNM: [ID 890413 daemon.notice] sc_ipmp0: state transition from OK to DOWN.
    Jan 17 08:00:47 PRD Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource PROD status on node PRD change to R_FM_DEGRADED
    Jan 17 08:00:47 PRD Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource PROD status msg on node PRD change to <IPMP Failure.>
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group CFS state on node PRD change to RG_PENDING_OFFLINE
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource PROD state on node PRD change to R_MON_STOPPING
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hafoip_monitor_stop> for resource <PROD>, resource group <CFS>, timeout <300> seconds
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hafoip_monitor_stop> completed successfully for resource <PROD>, resource group <CFS>, time used: 0% of timeout <300 seconds>
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource PROD state on node PRD change to R_ONLINE_UNMON
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource PROD state on node PRD change to R_STOPPING
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hafoip_stop> for resource <PROD>, resource group <CFS>, timeout <300> seconds
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource PROD status on node PRD change to R_FM_UNKNOWN
    Jan 17 08:00:50 PRD Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource PROD status msg on node PRD change to <Stopping>
    Jan 17 08:00:51 PRD ip: [ID 678092 kern.notice] TCP_IOC_ABORT_CONN: local = 172.016.005.025:0, remote = 000.000.000.000:0, start = -2, end = 6
    Jan 17 08:00:51 PRD ip: [ID 302654 kern.notice] TCP_IOC_ABORT_CONN: aborted 53 connections
    what can be the reason for reabooting?
    is there any way to avoid this, with only a failover?
    rgds
    Message was edited by:
    suj

    What is in that resource group? The cause is probably something with Failover_mode=HARD set. Check the manual reference section for this. The option would be to set the Failover_mode=SOFT.
    Tim
    ---

  • Cannot register

    I just got evil patch 119108-03 pushed on me.
    Now, in order to get patches (currently have base Soalris 10 x64 installed with just a few patches)
    I seem to need to register server. However, all I get
    is a dialog box stating "Cannot collect required system information" and nothing else.
    How is one supposed to get any patches now after
    July 25th ?????????
    I then thought there could be something wrong with
    119108-03 so I removed the patch and downloaded
    and installed the complete updateconnection package from the Sun Store. Same result.

    Found the probleme here.
    I had a moderately complex network configuration
    consisting of 2 bge interfaces, 2 VLAN interfaces on top of that, and running IPMP on all 4 four interfaces
    to create two subnetworks with dual IP paths.
    The updatemanager could not parse the output of
    ifconfig -a and thought I did not have any network
    interfaces at all. I temporarily re-configured with a
    single interface and then the Registration Wizard
    completed without any errors.

  • Problem with roamingin in VoIP SSID...

    Hi guys,
    My client has a WLC 5508 with a two dosens  of 1262s. I set SSID for the VoIP but when the client roams there is a  loss of packest. The client is using Cisco phones. Any help will be  appreciated.
    Pete
    (Cisco Controller) >show wlan 144
    WLAN Identifier.................................. 144
    Profile Name..................................... VoIP_Network
    Network Name (SSID).............................. Inside_144
    Status........................................... Enabled
    MAC Filtering.................................... Disabled
    Broadcast SSID................................... Enabled
    AAA Policy Override.............................. Disabled
    Network Admission Control
      Radius-NAC State............................... Disabled
      SNMP-NAC State................................. Disabled
      Quarantine VLAN................................ 0
    Maximum number of Associated Clients............. 0
    Maximum number of Clients per AP Radio........... 200
    Number of Active Clients......................... 10
    Exclusionlist Timeout............................ 60 seconds
    Session Timeout.................................. 1800 seconds
    CHD per WLAN..................................... Enabled
    Webauth DHCP exclusion........................... Disabled
    Interface........................................ 144_v
    Multicast Interface.............................. Not Configured
    WLAN IPv4 ACL.................................... unconfigured
    WLAN IPv6 ACL.................................... unconfigured
    DHCP Server...................................... Default
    DHCP Address Assignment Required................. Disabled
    Static IP client tunneling....................... Disabled
    Quality of Service............................... Platinum
    Scan Defer Priority.............................. 4,5,6
    Scan Defer Time.................................. 100 milliseconds
    WMM.............................................. Allowed
    WMM UAPSD Compliant Client Support............... Disabled
    Media Stream Multicast-direct.................... Disabled
    CCX - AironetIe Support.......................... Enabled
    CCX - Gratuitous ProbeResponse (GPR)............. Disabled
    CCX - Diagnostics Channel Capability............. Disabled
    Dot11-Phone Mode (7920).......................... Disabled
    Wired Protocol................................... None
    Passive Client Feature........................... Disabled
    Peer-to-Peer Blocking Action..................... Disabled
    Radio Policy..................................... 802.11b and 802.11g only
    DTIM period for 802.11a radio.................... 1
    DTIM period for 802.11b radio.................... 1
    Radius Servers
       Authentication................................ Global Servers
       Accounting.................................... Global Servers
          Interim Update............................. Disabled
       Dynamic Interface............................. Disabled
    Local EAP Authentication......................... Disabled
    Security
       802.11 Authentication:........................ Open System
       Static WEP Keys............................... Disabled
       802.1X........................................ Disabled
       Wi-Fi Protected Access (WPA/WPA2)............. Enabled
          WPA (SSN IE)............................... Enabled
             TKIP Cipher............................. Enabled
             AES Cipher.............................. Disabled
          WPA2 (RSN IE).............................. Disabled
                                                                    Auth Key Management
             802.1x.................................. Disabled
             PSK..................................... Enabled
             CCKM.................................... Disabled
             FT(802.11r)............................. Disabled
             FT-PSK(802.11r)......................... Disabled
          FT Reassociation Timeout................... 20
          FT Over-The-Air mode....................... Enabled
          FT Over-The-Ds mode........................ Enabled
          GTK Randomization.......................... Enabled
          SKC Cache Support.......................... Disabled
          CCKM TSF Tolerance......................... 1000
       Wi-Fi Direct policy configured................ Disabled
       EAP-Passthrough............................... Disabled
       CKIP ......................................... Disabled
       Web Based Authentication...................... Disabled
       Web-Passthrough............................... Disabled
       Conditional Web Redirect...................... Disabled
       Splash-Page Web Redirect...................... Disabled
       Auto Anchor................................... Disabled
       FlexConnect Local Switching................... Disabled
       FlexConnect Local Authentication.............. Disabled
       FlexConnect Learn IP Address.................. Enabled
       Client MFP.................................... Optional but inactive (WPA2 not configured)
       Tkip MIC Countermeasure Hold-down Timer....... 60
    Call Snooping.................................... Disabled
    Roamed Call Re-Anchor Policy..................... Disabled
    SIP CAC Fail Send-486-Busy Policy................ Enabled
    SIP CAC Fail Send Dis-Association Policy......... Disabled
    KTS based CAC Policy............................. Disabled
    Band Select...................................... Disabled
    Load Balancing................................... Disabled
    Multicast Buffer................................. Disabled
    Mobility Anchor List
    WLAN ID     IP Address            Status
    802.11u........................................ Disabled
      Access Network type............................ Not configured
      Network Authentication type.................... Not configured
      Internet service............................... Disabled
      HESSID......................................... 00:00:00:00:00:00
    Hotspot 2.0.................................... Disabled
      WAN Metrics configuration
        Link status.................................. 0
        Link symmetry................................ 0
        Downlink speed............................... 0
        Uplink speed................................. 0
    Mobility Services Advertisement Protocol....... Disabled
    (Cisco Controller) >
    (Cisco Controller) >
    (Cisco Controller) >debug client 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Nov 30 17:02:25.463: 2c:54:2d:ea:d4:0e Association received from mobile on AP 34:bd:c8:b2:b1:10
    *apfMsConnTask_2:  Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing  IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller  apf_policy.c:1697)
    *apfMsConnTask_2: Nov 30 17:02:25.464:  2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID  255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2:  Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e Applying site-specific Local  Bridging override for station 2c:54:2d:ea:d4:0e - vapId 144, site  'Floor_1', interface '144_v'
    *apfMsConnTask_2: Nov 30  17:02:25.464: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy  for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface  '144_v'
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e  Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId  144, site 'Floor_1', interface '144_v'
    *apfMsConnTask_2: Nov 30  17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL  'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller  apf_policy.c:1697)
    *apfMsConnTask_2: Nov 30 17:02:25.464:  2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID  255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e Processing WPA IE type 221, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e apfMsRunStateDec
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e apfMs1xStateDec
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Change state to START (0) last state RUN (20)
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e pemApfAddMobileStation2: APF_MS_PEM_WAIT_L2_AUTH_COMPLETE = 0.
    *apfMsConnTask_2: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Initializing policy
    *apfMsConnTask_2:  Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Change  state to AUTHCHECK (2) last state RUN (20)
    *apfMsConnTask_2:  Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 AUTHCHECK (2)  Change state to 8021X_REQD (3) last state RUN (20)
    *pemReceiveTask: Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 Removed NPU entry.
    *apfMsConnTask_2:  Nov 30 17:02:25.464: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3) DHCP  required on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2for this client
    *apfMsConnTask_2:  Nov 30 17:02:25.465: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3)  Plumbed mobile LWAPP rule on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2
    *apfMsConnTask_2:  Nov 30 17:02:25.465: 2c:54:2d:ea:d4:0e apfPemAddUser2  (apf_policy.c:268) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Associated to Associated
    *apfMsConnTask_2: Nov 30 17:02:25.465: 2c:54:2d:ea:d4:0e Scheduling deletion of Mobile Station:  (callerId: 49) in 1800 seconds
    *apfMsConnTask_2:  Nov 30 17:02:25.465: 2c:54:2d:ea:d4:0e Sending Assoc Response to  station on BSSID 34:bd:c8:b2:b1:10 (status 0) ApVapId 2 Slot 0
    *apfMsConnTask_2:  Nov 30 17:02:25.465: 2c:54:2d:ea:d4:0e apfProcessAssocReq  (apf_80211.c:6290) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Associated to Associated
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Creating a PKC PMKID Cache entry for station 2c:54:2d:ea:d4:0e (RSN 0)
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Setting active key cache index 0 ---> 8
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 0
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Initiating WPA PSK to mobile 2c:54:2d:ea:d4:0e
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Force Auth state
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Skipping EAP-Success to mobile 2c:54:2d:ea:d4:0e
    *dot1xMsgTask:  Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Starting key exchange to mobile  2c:54:2d:ea:d4:0e, data packets will be dropped
    *dot1xMsgTask: Nov 30 17:02:25.466: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                   state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:25.990: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:25.990: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTK_START  state (message 2) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:25.990: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:25.990: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                         state PTKINITNEGOTIATING (message 3), replay counter  00.00.00.00.00.00.00.01
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.015: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.015: 2c:54:2d:ea:d4:0e Received EAPOL-key in  PTKINITNEGOTIATING state (message 4) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e apfMs1xStateInc
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3)  Change state to L2AUTHCOMPLETE (4) last state RUN (20)
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  DHCP required on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2for this client
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  Plumbed mobile LWAPP rule on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e apfMsRunStateInc
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.016: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  Change state to RUN (20) last state RUN (20)
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Reached PLUMBFASTPATH: from line 5362
    *Dot1x: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Adding Fast Path rule
      type = Airespace AP Client
      on AP 34:bd:c8:b2:b1:10, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255, IPv6 ACL ID = 2
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Fast Path  rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging  Vlan = 144, Local Bridging intf id = 12
    *Dot1x_NW_MsgTask_6: Nov  30 17:02:26.017: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Successfully  plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e Key exchange done, data packets  from mobile 2c:54:2d:ea:d4:0e should be forwarded shortly
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                         state PTKINITDONE (message 5 - group), replay counter  00.00.00.00.00.00.00.02
    *spamApTask5: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e Sent EAPOL-Key M5 for mobile 2c:54:2d:ea:d4:0e
    *pemReceiveTask: Nov 30 17:02:26.017: 2c:54:2d:ea:d4:0e 10.123.201.4 Added NPU entry of type 1, dtlFlags 0x0
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.036: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:02:26.036: 2c:54:2d:ea:d4:0e Received EAPOL-key in  REKEYNEGOTIATING state (message 6) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:02:26.036: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e Association received from mobile on AP 34:bd:c8:b2:b1:10
    *apfMsConnTask_2:  Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing  IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller  apf_policy.c:1697)
    *apfMsConnTask_2: Nov 30 17:03:17.385:  2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID  255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2:  Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e Applying site-specific Local  Bridging override for station 2c:54:2d:ea:d4:0e - vapId 144, site  'Floor_1', interface '144_v'
    *apfMsConnTask_2: Nov 30  17:03:17.385: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy  for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface  '144_v'
    *apfMsConnTask_2: Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e  Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId  144, site 'Floor_1', interface '144_v'
    *apfMsConnTask_2: Nov 30  17:03:17.385: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL  'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller  apf_policy.c:1697)
    *apfMsConnTask_2: Nov 30 17:03:17.385:  2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID  255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2: Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_2: Nov 30 17:03:17.385: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e Processing WPA IE type 221, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e apfMsRunStateDec
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e apfMs1xStateDec
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Change state to START (0) last state RUN (20)
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e pemApfAddMobileStation2: APF_MS_PEM_WAIT_L2_AUTH_COMPLETE = 0.
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Initializing policy
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Change  state to AUTHCHECK (2) last state RUN (20)
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 AUTHCHECK (2)  Change state to 8021X_REQD (3) last state RUN (20)
    *pemReceiveTask: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 Removed NPU entry.
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3) DHCP  required on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2for this client
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3)  Plumbed mobile LWAPP rule on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e apfPemAddUser2  (apf_policy.c:268) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Associated to Associated
    *apfMsConnTask_2: Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e Scheduling deletion of Mobile Station:  (callerId: 49) in 1800 seconds
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e Sending Assoc Response to  station on BSSID 34:bd:c8:b2:b1:10 (status 0) ApVapId 2 Slot 0
    *apfMsConnTask_2:  Nov 30 17:03:17.386: 2c:54:2d:ea:d4:0e apfProcessAssocReq  (apf_80211.c:6290) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Associated to Associated
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Creating a PKC PMKID Cache entry for station 2c:54:2d:ea:d4:0e (RSN 0)
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Setting active key cache index 0 ---> 8
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 0
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Initiating WPA PSK to mobile 2c:54:2d:ea:d4:0e
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Force Auth state
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Skipping EAP-Success to mobile 2c:54:2d:ea:d4:0e
    *dot1xMsgTask:  Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Starting key exchange to mobile  2c:54:2d:ea:d4:0e, data packets will be dropped
    *dot1xMsgTask: Nov 30 17:03:17.389: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                   state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.422: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.422: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTK_START  state (message 2) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.422: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.423: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                         state PTKINITNEGOTIATING (message 3), replay counter  00.00.00.00.00.00.00.01
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e Received EAPOL-key in  PTKINITNEGOTIATING state (message 4) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e apfMs1xStateInc
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3)  Change state to L2AUTHCOMPLETE (4) last state RUN (20)
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  DHCP required on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2for this client
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  Plumbed mobile LWAPP rule on AP 34:bd:c8:b2:b1:10 vapId 144 apVapId 2
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e apfMsRunStateInc
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.433: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4)  Change state to RUN (20) last state RUN (20)
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Reached PLUMBFASTPATH: from line 5362
    *Dot1x: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Adding Fast Path rule
      type = Airespace AP Client
      on AP 34:bd:c8:b2:b1:10, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255, IPv6 ACL ID = 2
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Fast Path  rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging  Vlan = 144, Local Bridging intf id = 12
    *Dot1x_NW_MsgTask_6: Nov  30 17:03:17.435: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Successfully  plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6:  Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e Key exchange done, data packets  from mobile 2c:54:2d:ea:d4:0e should be forwarded shortly
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                         state PTKINITDONE (message 5 - group), replay counter  00.00.00.00.00.00.00.02
    *spamApTask5: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e Sent EAPOL-Key M5 for mobile 2c:54:2d:ea:d4:0e
    *pemReceiveTask: Nov 30 17:03:17.435: 2c:54:2d:ea:d4:0e 10.123.201.4 Added NPU entry of type 1, dtlFlags 0x0
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.447: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.447: 2c:54:2d:ea:d4:0e  Received EAPOL-key in REKEYNEGOTIATING state (message 6) from mobile  2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Nov 30 17:03:17.447: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *emWeb: Nov 30 17:03:46.162:  Configuring IPv6 ACL for WLAN:144, aclName passed is NULL
    *apfReceiveTask:  Nov 30 17:03:46.173: 2c:54:2d:ea:d4:0e apfSendDisAssocMsgDebug  (apf_80211.c:2162) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Associated to Disassociated
    *apfReceiveTask:  Nov 30 17:03:46.178: 2c:54:2d:ea:d4:0e Sent Disassociate to mobile on  AP 34:bd:c8:b2:b1:10-0 (reason 1, caller apf_ms.c:5558)
    *apfReceiveTask:  Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e Sent Deauthenticate to mobile on  BSSID 34:bd:c8:b2:b1:10 slot 0(caller apf_ms.c:5678)
    *apfReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e apfMsAssoStateDec
    *apfReceiveTask:  Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e apfMsExpireMobileStation  (apf_ms.c:5716) Changing state for mobile 2c:54:2d:ea:d4:0e on AP  34:bd:c8:b2:b1:10 from Disassociated to Idle
    *apfReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e pemApfDeleteMobileStation2: APF_MS_PEM_WAIT_L2_AUTH_COMPLETE = 0.
    *apfReceiveTask:  Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Deleted  mobile LWAPP rule on AP [34:bd:c8:b2:b1:10]
    *pemReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e 10.123.201.4 Removed NPU entry.
    *apfReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e apfMsRunStateDec
    *apfReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e apfMs1xStateDec
    *apfReceiveTask: Nov 30 17:03:46.183: 2c:54:2d:ea:d4:0e Deleting mobile on AP 34:bd:c8:b2:b1:10(0)

    Hi guys and Saravanan thank for the ideas....
    the qualituy is getting better, not satisfactory for the customer though...
    I have upgraded the firware as advised to 1.4.3 - I forgot to mention I have 7925g wifi phonee
    I set the 802.1x + cckm with eap-fast and WPA2 and definately the quality of the calls got a huge improvement but still not enough. What can be the reason for the confinuing problems during roaming?
    Guys, is it possible to set the CCKM without ACS (or WDS - i think that was the second option)
    here is some output:
    (Cisco Controller) show>wlan 3
    WLAN Identifier.................................. 3
    Profile Name..................................... test_wifi_144
    Network Name (SSID).............................. test144
    Status........................................... Enabled
    MAC Filtering.................................... Disabled
    Broadcast SSID................................... Enabled
    AAA Policy Override.............................. Disabled
    Network Admission Control
      Radius-NAC State............................... Disabled
      SNMP-NAC State................................. Disabled
      Quarantine VLAN................................ 0
    Maximum number of Associated Clients............. 0
    Maximum number of Clients per AP Radio........... 200
    Number of Active Clients......................... 1
    Exclusionlist.................................... Disabled
    Session Timeout.................................. 65535 seconds
    CHD per WLAN..................................... Enabled
    Webauth DHCP exclusion........................... Disabled
    Interface........................................ 144_v
    --More-- or (q)uit
    Multicast Interface.............................. Not Configured
    WLAN IPv4 ACL.................................... unconfigured
    WLAN IPv6 ACL.................................... unconfigured
    DHCP Server...................................... Default
    DHCP Address Assignment Required................. Disabled
    Static IP client tunneling....................... Disabled
    Quality of Service............................... Platinum
    Scan Defer Priority.............................. 4,5,6
    Scan Defer Time.................................. 100 milliseconds
    WMM.............................................. Required
    WMM UAPSD Compliant Client Support............... Disabled
    Media Stream Multicast-direct.................... Disabled
    CCX - AironetIe Support.......................... Enabled
    CCX - Gratuitous ProbeResponse (GPR)............. Disabled
    CCX - Diagnostics Channel Capability............. Disabled
    Dot11-Phone Mode (7920).......................... ap-cac-limit
    Wired Protocol................................... None
    Passive Client Feature........................... Disabled
    Peer-to-Peer Blocking Action..................... Disabled
    Radio Policy..................................... 802.11b and 802.11g only
    DTIM period for 802.11a radio.................... 2
    DTIM period for 802.11b radio.................... 2
    Radius Servers
    --More-- or (q)uit
       Authentication................................ 172.16.106.53 1645
       Accounting.................................... Disabled
       Dynamic Interface............................. Disabled
    Local EAP Authentication......................... Disabled
    Security
       802.11 Authentication:........................ Open System
       Static WEP Keys............................... Disabled
       802.1X........................................ Disabled
       Wi-Fi Protected Access (WPA/WPA2)............. Enabled
          WPA (SSN IE)............................... Disabled
          WPA2 (RSN IE).............................. Enabled
             TKIP Cipher............................. Disabled
             AES Cipher.............................. Enabled
                                                                   Auth Key Management
             802.1x.................................. Enabled
             PSK..................................... Disabled
             CCKM.................................... Enabled
             FT(802.11r)............................. Disabled
             FT-PSK(802.11r)......................... Disabled
          FT Reassociation Timeout................... 20
          FT Over-The-Air mode....................... Enabled
          FT Over-The-Ds mode........................ Enabled
    --More-- or (q)uit
          GTK Randomization.......................... Disabled
          SKC Cache Support.......................... Disabled
          CCKM TSF Tolerance......................... 1000
       Wi-Fi Direct policy configured................ Disabled
       EAP-Passthrough............................... Disabled
       CKIP ......................................... Disabled
       Web Based Authentication...................... Disabled
       Web-Passthrough............................... Disabled
       Conditional Web Redirect...................... Disabled
       Splash-Page Web Redirect...................... Disabled
       Auto Anchor................................... Disabled
       FlexConnect Local Switching................... Disabled
       FlexConnect Local Authentication.............. Disabled
       FlexConnect Learn IP Address.................. Enabled
       Client MFP.................................... Optional
       Tkip MIC Countermeasure Hold-down Timer....... 60
    Call Snooping.................................... Disabled
    Roamed Call Re-Anchor Policy..................... Disabled
    SIP CAC Fail Send-486-Busy Policy................ Enabled
    SIP CAC Fail Send Dis-Association Policy......... Disabled
    KTS based CAC Policy............................. Disabled
    Band Select...................................... Disabled
    Load Balancing................................... Disabled
    --More-- or (q)uit
    Multicast Buffer................................. Disabled
    Mobility Anchor List
    WLAN ID     IP Address            Status
    802.11u........................................ Disabled
      Access Network type............................ Not configured
      Network Authentication type.................... Not configured
      Internet service............................... Disabled
      HESSID......................................... 00:00:00:00:00:00
    Hotspot 2.0.................................... Disabled
      WAN Metrics configuration
        Link status.................................. 0
        Link symmetry................................ 0
        Downlink speed............................... 0
        Uplink speed................................. 0
    Mobility Services Advertisement Protocol....... Disabled
    (Cisco Controller) >debug client 2C542DEAD40E
    *apfMsConnTask_3: Dec 07 13:55:49.522: 2c:54:2d:ea:d4:0e Adding mobile on LWAPP AP 34:bd:c8:b3:d9:f0(0)
    *apfMsConnTask_3: Dec 07 13:55:49.522: 2c:54:2d:ea:d4:0e Association received from mobile on AP 34:bd:c8:b3:d9:f0
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Applying site-specific Local Bridging override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface '144_v'
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 0 0 0 0 0 0 0 0 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Processing RSN IE type 48, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e CCKM: Mobile is using CCKM
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Received RSN IE with 0 PMKIDs from mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Initializing policy
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 START (0) Change state to AUTHCHECK (2) last state AUTHCHECK (2)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 AUTHCHECK (2) Change state to 8021X_REQD (3) last state 8021X_REQD (3)
    *apfMsConnTask_3: Dec 07 13:55:49.523: 2c:54:2d:ea:d4:0e 0.0.0.0 8021X_REQD (3) DHCP Not required on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1for this client
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e 0.0.0.0 8021X_REQD (3) Plumbed mobile LWAPP rule on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e apfMsAssoStateInc
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e apfPemAddUser2 (apf_policy.c:268) Changing state for mobile 2c:54:2d:ea:d4:0e on AP 34:bd:c8:b3:d9:f0 from Idle to Associated
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e Stopping deletion of Mobile Station: (callerId: 48)
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e Sending Assoc Response to station on BSSID 34:bd:c8:b3:d9:f0 (status 0) ApVapId 1 Slot 0
    *apfMsConnTask_3: Dec 07 13:55:49.524: 2c:54:2d:ea:d4:0e apfProcessAssocReq (apf_80211.c:6290) Changing state for mobile 2c:54:2d:ea:d4:0e on AP 34:bd:c8:b3:d9:f0 from Associated to Associated
    *dot1xMsgTask: Dec 07 13:55:49.525: 2c:54:2d:ea:d4:0e Disable re-auth, use PMK lifetime.
    *dot1xMsgTask: Dec 07 13:55:49.525: 2c:54:2d:ea:d4:0e Station 2c:54:2d:ea:d4:0e setting dot1x reauth timeout = 65535
    *dot1xMsgTask: Dec 07 13:55:49.525: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Connecting state
    *dot1xMsgTask: Dec 07 13:55:49.525: 2c:54:2d:ea:d4:0e Sending EAP-Request/Identity to mobile 2c:54:2d:ea:d4:0e (EAP Id 1)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.574: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.574: 2c:54:2d:ea:d4:0e Received Identity Response (count=1) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.574: 2c:54:2d:ea:d4:0e EAP State update from Connecting to Authenticating for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.574: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Authenticating state
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.574: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.583: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.583: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=85) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.583: 2c:54:2d:ea:d4:0e WARNING: updated EAP-Identifier 1 ===> 85 for STA 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.583: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 85)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.591: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.591: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 85, EAP Type 3)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.591: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.602: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.602: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=86) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.602: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 86)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.621: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.621: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 86, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.621: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.625: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.625: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=87) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.625: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 87)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.653: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.653: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 87, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.653: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.655: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.655: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=89) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.655: 2c:54:2d:ea:d4:0e WARNING: updated EAP-Identifier 87 ===> 89 for STA 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.655: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 89)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.671: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.671: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 89, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.671: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.676: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.676: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=90) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.676: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 90)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.691: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.691: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 90, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.691: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.702: 2c:54:2d:ea:d4:0e Processing Access-Accept for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Resetting web IPv4 acl from 255 to 255
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Username entry (test960) created for mobile, length = 253
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Username entry (test960) created in mscb for mobile, length = 253
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Setting re-auth timeout to 65535 seconds, got from WLAN config.
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Station 2c:54:2d:ea:d4:0e setting dot1x reauth timeout = 65535
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Creating a PKC PMKID Cache entry for station 2c:54:2d:ea:d4:0e (RSN 2)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Resetting MSCB PMK Cache Entry 0 for station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 0
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Adding BSSID 34:bd:c8:b3:d9:f0 to PMKID cache at index 0 for station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: New PMKID: (16)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703:      [0000] ab 8f b5 75 ad c5 8e af 50 0d ce 4a f1 7b 16 9e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Disabling re-auth since PMK lifetime can take care of same.
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e CCKM: Create a global PMK cache entry
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Sending EAP-Success to mobile 2c:54:2d:ea:d4:0e (EAP Id 90)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: Including PMKID in M1  (16)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703:      [0000] ab 8f b5 75 ad c5 8e af 50 0d ce 4a f1 7b 16 9e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Starting key exchange to mobile 2c:54:2d:ea:d4:0e, data packets will be dropped
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                        state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Entering Backend Auth Success state (id=90) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.703: 2c:54:2d:ea:d4:0e Received Auth Success while in Authenticating state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.704: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Authenticated state
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTK_START state (message 2) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: 2c:54:2d:ea:d4:0e CCKM: Sending cache add
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: CCKM: Sending CCKM PMK (Version_1) information to mobility group
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: CCKM: Sending CCKM PMK (Version_2) information to mobility group
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.721: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                        state PTKINITNEGOTIATING (message 3), replay counter 00.00.00.00.00.00.00.01
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTKINITNEGOTIATING state (message 4) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e apfMs1xStateInc
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 8021X_REQD (3) Change state to L2AUTHCOMPLETE (4) last state L2AUTHCOMPLETE (4)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 L2AUTHCOMPLETE (4) DHCP Not required on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1for this client
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 L2AUTHCOMPLETE (4) Plumbed mobile LWAPP rule on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 L2AUTHCOMPLETE (4) Change state to DHCP_REQD (7) last state DHCP_REQD (7)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) pemAdvanceState2 5253, Adding TMP rule
    *Dot1x_NW_MsgTask_: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Adding Fast Path rule
      type = Airespace AP - Learn IP address
      on AP 34:bd:c8:b3:d9:f0, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255, IPv
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Fast Path rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging Vlan = 144, Local Bridging intf id = 12
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Successfully plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *Dot1x_NW_MsgTask_6: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *apfReceiveTask: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) State Update from Mobility-Incomplete to Mobility-Complete, mobility role=Local, client state=APF_MS_STATE_ASSOCIATED
    *apfReceiveTask: Dec 07 13:55:49.741: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) pemAdvanceState2 4891, Adding TMP rule
    *apfReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Replacing Fast Path rule
      type = Airespace AP - Learn IP address
      on AP 34:bd:c8:b3:d9:f0, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255,
    *apfReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Fast Path rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging Vlan = 144, Local Bridging intf id = 12
    *apfReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e 0.0.0.0 DHCP_REQD (7) Successfully plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *pemReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0
    *pemReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e Sent an XID frame
    *pemReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0
    *pemReceiveTask: Dec 07 13:55:49.742: 2c:54:2d:ea:d4:0e Sent an XID frame
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP received op BOOTREQUEST (1) (len 556,vlan 0, port 1, encap 0xec03)
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP selecting relay 1 - control block settings:
                            dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 0.0.0.0  VLAN: 0
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP selected relay 1 - 172.16.100.121 (local address 10.123.200.15, gateway 10.123.200.1, VLAN 144, port 1)
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP DISCOVER (1)
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 1
    *DHCP Socket Task: Dec 07 13:55:50.513: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 10.123.200.15
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   requested ip: 10.123.205.33
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP ARPing for 10.123.200.1 (SPA 10.123.200.15, vlanId 144)
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP selecting relay 2 - control block settings:
                            dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 10.123.200.15  VLAN: 144
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP selected relay 2 - 172.16.100.122 (local address 10.123.200.15, gateway 10.123.200.1, VLAN 144, port 1)
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP DISCOVER (1)
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 2
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 10.123.200.15
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP   requested ip: 10.123.205.33
    *DHCP Socket Task: Dec 07 13:55:50.514: 2c:54:2d:ea:d4:0e DHCP ARPing for 10.123.200.1 (SPA 10.123.200.15, vlanId 144)
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP received op BOOTREQUEST (1) (len 556,vlan 0, port 1, encap 0xec03)
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP selecting relay 1 - control block settings:
                            dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 10.123.200.15  VLAN: 144
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP selected relay 1 - 172.16.100.121 (local address 10.123.200.15, gateway 10.123.200.1, VLAN 144, port 1)
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP DISCOVER (1)
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 1
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 10.123.200.15
    *DHCP Socket Task: Dec 07 13:55:52.512: 2c:54:2d:ea:d4:0e DHCP   requested ip: 10.123.205.33
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP sending REQUEST to 10.123.200.1 (len 374, port 1, vlan 144)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP selecting relay 2 - control block settings:
                            dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 10.123.200.15  VLAN: 144
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP selected relay 2 - 172.16.100.122 (local address 10.123.200.15, gateway 10.123.200.1, VLAN 144, port 1)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP DISCOVER (1)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 2
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 10.123.200.15
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   requested ip: 10.123.205.33
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP sending REQUEST to 10.123.200.1 (len 374, port 1, vlan 144)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP received op BOOTREPLY (2) (len 322,vlan 144, port 1, encap 0xec00)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP setting server from OFFER (server 172.16.100.121, yiaddr 10.123.201.4)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP sending REPLY to STA (len 430, port 1, vlan 0)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP OFFER (2)
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 10.123.201.4
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:52.513: 2c:54:2d:ea:d4:0e DHCP   server id: 1.1.1.1  rcvd server id: 172.16.100.121
    *DHCP Socket Task: Dec 07 13:55:52.514: 2c:54:2d:ea:d4:0e DHCP received op BOOTREPLY (2) (len 322,vlan 144, port 1, encap 0xec00)
    *DHCP Socket Task: Dec 07 13:55:52.514: 2c:54:2d:ea:d4:0e DHCP dropping OFFER from 172.16.100.122 (yiaddr 10.123.205.33)
    *DHCP Socket Task: Dec 07 13:55:52.523: 2c:54:2d:ea:d4:0e DHCP received op BOOTREQUEST (1) (len 556,vlan 0, port 1, encap 0xec03)
    *DHCP Socket Task: Dec 07 13:55:52.523: 2c:54:2d:ea:d4:0e DHCP selecting relay 1 - control block settings:
                            dhcpServer: 172.16.100.121, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 10.123.200.15  VLAN: 144
    *DHCP Socket Task: Dec 07 13:55:52.523: 2c:54:2d:ea:d4:0e DHCP selected relay 1 - 172.16.100.121 (local address 10.123.200.15, gateway 10.123.200.1, VLAN 144, port 1)
    *DHCP Socket Task: Dec 07 13:55:52.523: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP REQUEST (3)
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 1
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 10.123.200.15
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   requested ip: 10.123.201.4
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP   server id: 172.16.100.121  rcvd server id: 1.1.1.1
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP sending REQUEST to 10.123.200.1 (len 382, port 1, vlan 144)
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP selecting relay 2 - control block settings:
                            dhcpServer: 172.16.100.121, dhcpNetmask: 0.0.0.0,
                            dhcpGateway: 0.0.0.0, dhcpRelay: 10.123.200.15  VLAN: 144
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP selected relay 2 - NONE
    *DHCP Socket Task: Dec 07 13:55:52.524: 2c:54:2d:ea:d4:0e DHCP received op BOOTREPLY (2) (len 322,vlan 144, port 1, encap 0xec00)
    *DHCP Socket Task: Dec 07 13:55:52.525: 2c:54:2d:ea:d4:0e Static IP client associated to interface 144_v which can support client subnet.
    *DHCP Socket Task: Dec 07 13:55:52.525: 2c:54:2d:ea:d4:0e apfMsRunStateInc
    *DHCP Socket Task: Dec 07 13:55:52.525: 2c:54:2d:ea:d4:0e 10.123.201.4 DHCP_REQD (7) Change state to RUN (20) last state RUN (20)
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Reached PLUMBFASTPATH: from line 5776
    *DHCP Soc: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Replacing Fast Path rule
      type = Airespace AP Client
      on AP 34:bd:c8:b3:d9:f0, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255, IPv6 ACL ID
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Fast Path rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging Vlan = 144, Local Bridging intf id = 12
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Successfully plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e Assigning Address 10.123.201.4 to mobile
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP success event for client. Clearing dhcp failure count for interface 144_v.
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP sending REPLY to STA (len 430, port 1, vlan 0)
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP transmitting DHCP ACK (5)
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP   op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP   xid: 0xf12d461 (252892257), secs: 0, flags: 0
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP   chaddr: 2c:54:2d:ea:d4:0e
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP   ciaddr: 0.0.0.0,  yiaddr: 10.123.201.4
    *DHCP Socket Task: Dec 07 13:55:52.526: 2c:54:2d:ea:d4:0e DHCP   siaddr: 0.0.0.0,  giaddr: 0.0.0.0
    *DHCP Socket Task: Dec 07 13:55:52.527: 2c:54:2d:ea:d4:0e DHCP   server id: 1.1.1.1  rcvd server id: 172.16.100.121
    *pemReceiveTask: Dec 07 13:55:52.527: 2c:54:2d:ea:d4:0e 10.123.201.4 Added NPU entry of type 1, dtlFlags 0x10
    *pemReceiveTask: Dec 07 13:55:52.527: 2c:54:2d:ea:d4:0e Sending a gratuitous ARP for 10.123.201.4, VLAN Id 144
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Association received from mobile on AP 34:bd:c8:b3:d9:f0
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Applying site-specific Local Bridging override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Processing RSN IE type 48, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e CCKM: Mobile is using CCKM
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Received RSN IE with 0 PMKIDs from mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Removing BSSID 34:bd:c8:b3:d9:f0 from PMKID cache of station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Resetting MSCB PMK Cache Entry 0 for station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:01.509: 2c:54:2d:ea:d4:0e Setting active key cache index 0 ---> 8
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e apfMsRunStateDec
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e apfMs1xStateDec
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Change state to START (0) last state RUN (20)
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e pemApfAddMobileStation2: APF_MS_PEM_WAIT_L2_AUTH_COMPLETE = 0.
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Initializing policy
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 START (0) Change state to AUTHCHECK (2) last state RUN (20)
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 AUTHCHECK (2) Change state to 8021X_REQD (3) last state RUN (20)
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3) DHCP required on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1for this client
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3) Plumbed mobile LWAPP rule on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e apfPemAddUser2 (apf_policy.c:268) Changing state for mobile 2c:54:2d:ea:d4:0e on AP 34:bd:c8:b3:d9:f0 from Associated to Associated
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e Stopping deletion of Mobile Station: (callerId: 48)
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e Sending Assoc Response to station on BSSID 34:bd:c8:b3:d9:f0 (status 0) ApVapId 1 Slot 0
    *apfMsConnTask_3: Dec 07 13:57:01.510: 2c:54:2d:ea:d4:0e apfProcessAssocReq (apf_80211.c:6290) Changing state for mobile 2c:54:2d:ea:d4:0e on AP 34:bd:c8:b3:d9:f0 from Associated to Associated
    *dot1xMsgTask: Dec 07 13:57:01.512: 2c:54:2d:ea:d4:0e Disable re-auth, use PMK lifetime.
    *dot1xMsgTask: Dec 07 13:57:01.512: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Connecting state
    *dot1xMsgTask: Dec 07 13:57:01.512: 2c:54:2d:ea:d4:0e Sending EAP-Request/Identity to mobile 2c:54:2d:ea:d4:0e (EAP Id 1)
    *pemReceiveTask: Dec 07 13:57:01.513: 2c:54:2d:ea:d4:0e 10.123.201.4 Removed NPU entry.
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.654: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.654: 2c:54:2d:ea:d4:0e Received Identity Response (count=1) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.654: 2c:54:2d:ea:d4:0e EAP State update from Connecting to Authenticating for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.654: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Authenticating state
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.654: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.684: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.684: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=86) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.684: 2c:54:2d:ea:d4:0e WARNING: updated EAP-Identifier 1 ===> 86 for STA 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.684: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 86)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.695: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.695: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 86, EAP Type 3)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.695: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.699: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.699: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=87) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.699: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 87)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.806: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.806: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 87, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.806: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.809: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.809: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=88) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.809: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 88)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.874: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.874: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 88, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.874: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.880: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.880: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=90) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.880: 2c:54:2d:ea:d4:0e WARNING: updated EAP-Identifier 88 ===> 90 for STA 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.880: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 90)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.903: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.903: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 90, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.903: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.909: 2c:54:2d:ea:d4:0e Processing Access-Challenge for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.909: 2c:54:2d:ea:d4:0e Entering Backend Auth Req state (id=91) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:01.909: 2c:54:2d:ea:d4:0e Sending EAP Request from AAA to mobile 2c:54:2d:ea:d4:0e (EAP Id 91)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.061: 2c:54:2d:ea:d4:0e Received EAPOL EAPPKT from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.061: 2c:54:2d:ea:d4:0e Received EAP Response from mobile 2c:54:2d:ea:d4:0e (EAP Id 91, EAP Type 43)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.061: 2c:54:2d:ea:d4:0e Entering Backend Auth Response state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.076: 2c:54:2d:ea:d4:0e Processing Access-Accept for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.076: 2c:54:2d:ea:d4:0e Resetting web IPv4 acl from 255 to 255
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.076: 2c:54:2d:ea:d4:0e Setting re-auth timeout to 65535 seconds, got from WLAN config.
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Station 2c:54:2d:ea:d4:0e setting dot1x reauth timeout = 65535
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Creating a PKC PMKID Cache entry for station 2c:54:2d:ea:d4:0e (RSN 2)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Resetting MSCB PMK Cache Entry 0 for station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 0
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Adding BSSID 34:bd:c8:b3:d9:f0 to PMKID cache at index 0 for station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: New PMKID: (16)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077:      [0000] 16 bf c0 3e 07 00 79 b1 51 ca d3 47 44 69 1b a1
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Disabling re-auth since PMK lifetime can take care of same.
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e CCKM: Create a global PMK cache entry
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Sending EAP-Success to mobile 2c:54:2d:ea:d4:0e (EAP Id 91)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: Including PMKID in M1  (16)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077:      [0000] 16 bf c0 3e 07 00 79 b1 51 ca d3 47 44 69 1b a1
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Starting key exchange to mobile 2c:54:2d:ea:d4:0e, data packets will be dropped
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                        state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Entering Backend Auth Success state (id=91) for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e Received Auth Success while in Authenticating state for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.077: 2c:54:2d:ea:d4:0e dot1x - moving mobile 2c:54:2d:ea:d4:0e into Authenticated state
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTK_START state (message 2) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: 2c:54:2d:ea:d4:0e CCKM: Sending cache add
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: CCKM: Sending CCKM PMK (Version_1) information to mobility group
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: CCKM: Sending CCKM PMK (Version_2) information to mobility group
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.113: 2c:54:2d:ea:d4:0e Sending EAPOL-Key Message to mobile 2c:54:2d:ea:d4:0e
                                                                                                                        state PTKINITNEGOTIATING (message 3), replay counter 00.00.00.00.00.00.00.01
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e Received EAPOL-Key from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e Received EAPOL-key in PTKINITNEGOTIATING state (message 4) from mobile 2c:54:2d:ea:d4:0e
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e apfMs1xStateInc
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e 10.123.201.4 8021X_REQD (3) Change state to L2AUTHCOMPLETE (4) last state RUN (20)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4) DHCP required on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1for this client
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4) Plumbed mobile LWAPP rule on AP 34:bd:c8:b3:d9:f0 vapId 3 apVapId 1
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e apfMsRunStateInc
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.164: 2c:54:2d:ea:d4:0e 10.123.201.4 L2AUTHCOMPLETE (4) Change state to RUN (20) last state RUN (20)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Reached PLUMBFASTPATH: from line 5362
    *Dot1x: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Adding Fast Path rule
      type = Airespace AP Client
      on AP 34:bd:c8:b3:d9:f0, slot 0, interface = 1, QOS = 2
      IPv4 ACL ID = 255, IPv6 ACL ID = 2
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Fast Path rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 7006  Local Bridging Vlan = 144, Local Bridging intf id = 12
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Successfully plumbed mobile rule (IPv4 ACL ID 255, IPv6 ACL ID 255)
    *Dot1x_NW_MsgTask_6: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e Stopping retransmission timer for mobile 2c:54:2d:ea:d4:0e
    *pemReceiveTask: Dec 07 13:57:02.166: 2c:54:2d:ea:d4:0e 10.123.201.4 Added NPU entry of type 1, dtlFlags 0x0
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e CCKM: Received REASSOC REQ IE
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e Reassociation received from mobile on AP 34:bd:c8:b2:b1:10
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e Applying site-specific Local Bridging override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface '144_v'
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_2: Dec 07 13:57:03.265: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Processing RSN IE type 48, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e CCKM: Mobile is using CCKM
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Received RSN IE with 0 PMKIDs from mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Found an cache entry for BSSID 34:bd:c8:b3:d9:f0 in PMKID cache at index 0 of station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Removing BSSID 34:bd:c8:b3:d9:f0 from PMKID cache of station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Resetting MSCB PMK Cache Entry 0 for station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e Setting active key cache index 0 ---> 8
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e CCKM: Processing REASSOC REQ IE
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e CCKM: using HMAC SHA1 to compute MIC
    *apfMsConnTask_2: Dec 07 13:57:03.266: 2c:54:2d:ea:d4:0e CCKM: Received a valid REASSOC REQ IE
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e CCKM: Initializing PMK cache entry with a new PTK
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Resetting MSCB PMK Cache Entry 0 for station 2c:54:2d:ea:d4:0e
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 0
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Creating a PKC PMKID Cache entry for station 2c:54:2d:ea:d4:0e (RSN 2) on BSSID 34:bd:c8:b3:d9:f0
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Setting active key cache index 0 ---> 8
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e CCKM: using HMAC SHA1 to compute MIC
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Including CCKM Response IE (length 54) in Assoc Resp to mobile
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Sending Assoc Response to station on BSSID 34:bd:c8:b2:b1:10 (status 202) ApVapId 1 Slot 0
    *apfMsConnTask_2: Dec 07 13:57:03.267: 2c:54:2d:ea:d4:0e Scheduling deletion of Mobile Station:  (callerId: 22) in 3 seconds
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Association received from mobile on AP 34:bd:c8:b3:d9:f0
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Applying site-specific Local Bridging override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Applying Local Bridging Interface Policy for station 2c:54:2d:ea:d4:0e - vlan 144, interface id 12, interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Applying site-specific override for station 2c:54:2d:ea:d4:0e - vapId 3, site 'Floor_1', interface '144_v'
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv4 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1697)
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Changing IPv6 ACL 'none' (ACL ID 255) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1864)
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e processSsidIE  statusCode is 0 and status is 0
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e processSsidIE  ssid_done_flag is 0 finish_flag is 0
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e STA - rates (4): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e suppRates  statusCode is 0 and gotSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e extSuppRates  statusCode is 0 and gotExtSuppRatesElement is 1
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Processing RSN IE type 48, length 22 for mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e CCKM: Mobile is using CCKM
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Received RSN IE with 0 PMKIDs from mobile 2c:54:2d:ea:d4:0e
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e Setting active key cache index 8 ---> 8
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e unsetting PmkIdValidatedByAp
    *apfMsConnTask_3: Dec 07 13:57:04.925: 2c:54:2d:ea:d4:0e apfMsRunStateDec
    *apfMsConnTask_3: Dec 07 13:57:04.926: 2c:54:2d:ea:d4:0e apfMs1xStateDec
    *apfMsConnTask_3: Dec 07 13:57:04.926: 2c:54:2d:ea:d4:0e 10.123.201.4 RUN (20) Change state to START (0) last state RUN (20)

Maybe you are looking for