Duplicated IP error on some clients

Question

Hello,  Dear All.
We have typical deployment with WLC550x (7.0.116.0) and 16 APs (AIR-LAP1242G-E-K9) placed on same site. WLC connected to 3560 (with LAG , and dhcp relay)  , and all wireless clients( Motorolla MC3100 handled PC ) work with same WPA2-PSK SSID. All APs configured as HREAP group and SSID has local switching and auth settings. DHCP server for clients work on Windows 2008r2 failover cluster, APs give addresses from WLC builtin server.
Almost all works great.  But sometimes some clients go insane . After wake up,  they show duplicate IP error and wont connect to nework.  On DHCP server  this IP shown as leased to client mac(without any errors and so).
Client reboot wont resolve this issue.
After reboot client try another dhcp address (after marking dchp decline message) but also without luck, with same error and another IP.
All this looks like client side problem . But when i try debug arp on root switch 3560  i get following situation.
After client wake-up
*Apr 10 18:44:32.773: IP ARP: rcvd req src 10.116.51.59 0023.68cb.a8fc, dst 10.116.51.59 Vlan51
*Apr 10 18:44:32.782: IP ARP: rcvd req src 10.116.51.59 0023.68cb.a812, dst 10.116.51.59 Vlan51
After reboot
*Apr 10 19:16:40.123: IP ARP: rcvd req src 10.116.51.24 0023.68cb.a8fc, dst 10.116.51.24 Vlan51
*Apr 10 19:16:40.131: IP ARP: rcvd req src 10.116.51.24 0023.68c9.a29b, dst 10.116.51.24 Vlan51
*Apr 10 19:16:40.459: IP ARP: rcvd req src 10.116.51.27 0023.68cb.a8fc, dst 10.116.51.27 Vlan51
*Apr 10 19:16:40.467: IP ARP: rcvd req src 10.116.51.27 0023.68cb.a9b6, dst 10.116.51.27 Vlan51
Where  0023.68cb.a8fc problem clent mac and   0023.68cb.a812,
0023.68cb.a9b6, 0023.68c9.a29b    - another full working clients  MACs(with another ip address).
Looks like another client (or ??  ap or controller) send ARP reqest with same IP right after problem client. How this possible ?
I'll be
appreciate for any opinions
and comments!

Answer 1

Why you think so ?
Reboot of controller resolve problem for some time.
Clients work flawlessly when controller disconnected from network and APs in H-REAP mode. Also same clients (handles PC models) works  in online mode with wlc 4400 without problems.
There are two difference in this deployment:
1. wlc 5500 and 7.0 major release
2. APs in  H-REAP mode and WLAN configured as localy switched
From my point of view problem related with controller  incorrect  behavior(or I think so).
Process of obtaining  ip addresses interrupted by  controller   for some unclear  reasons.
1.Client obtain IP address from DHCP without problem(via dhcp relay on 3750)
2. Client check that no one use it by sending  ARP request "ARP Who has address (leased from DHCP address here)"
3. No response mean that address free.
4. Controller interrupt process. By sending ARP resposne  "ARP  (leased from DHCP address here) used by (MAC of one of  client what used it in past)   "
5. After this client show duplicated ip error.
6. Controller log message
%APF-4-REGISTER_IPADD_ON_MSCB_FAILED:apf_foreignap.c:1504 Could not Register IP Add on MSCB. Identity theft alert for IP address. Address:00:23:68:cb:a9:87
I think this mean controller failed to update some internal table  with new ip-mac pair and prevent client work.
But how disable such controller behavior I dont know.
I disabled controller DHCP proxy , I disabled client ip learning in WLAN settings , I disabled any exclusion policies.