CSS11501 commit vip redundancy script failure

Question

Hi all,
recently when i run the commit vip redundancy script, i encountered the following error. This script has never failed in the past. Upon checking the backup CSS, i did notice that my most recent changes were actually synced. The following is the debug i have captured while running the script. Can someone please help to look at it? Thanks!
active-lb# script play commit_vip_redundancy "local 167.168.165.10 remote 167.168.165.9 -a -d"
active-lb#
Checking available disk space on systems ...
Checking the disk space locally before continuing with the script.
Verifying that another local session is not running the script.
Creating script/vipr_config_sync_lock file.
Verifying app and redundancy configs ...
Verifying that app session is up with backup switch.
Making sure app session is up.
Seconds to wait before calling it quits:    60
Checking the disk space remotely before continuing with the script.
Checking local and remote switch versions ...
Storing the running code versions of the local and remote switch.
Storing the local switch's version.
Retrieving the remote switch's version.
Checking remote version for 4.0
Checking if switch is BACKUP for any virtual routers and if
the state is 'No Service'.
Checking vip redundancy state....
Checking if backup switch is Master for any VRIDs.
If it is, either a local interface that once held redundant vips
has been removed or the Backup is a Master for another vip-redundant
relationship.
Checking compatibility of systems......
LOCAL switch is a  css11501,7
REMOTE switch is a  css11501,7
Mode: INTERFACE - Checking to see if interface Ethernet-Mgmt  exists on remote system.
Mode: INTERFACE - Checking to see if interface e9  exists on remote system.
Mode: INTERFACE - Checking to see if interface e8  exists on remote system.
Mode: INTERFACE - Checking to see if interface e7  exists on remote system.
Mode: INTERFACE - Checking to see if interface e6  exists on remote system.
Mode: INTERFACE - Checking to see if interface e5  exists on remote system.
Mode: INTERFACE - Checking to see if interface e4  exists on remote system.
Mode: INTERFACE - Checking to see if interface e3  exists on remote system.
Mode: INTERFACE - Checking to see if interface e2  exists on remote system.
Mode: INTERFACE - Checking to see if interface e1  exists on remote system.
Working
Saving Master running-config to startup-config and archiving startup-config.
Copying running-config to startup-config.
Archiving startup-config.
Swapping Master and Backup ip addresses in tmp.cfg for app
and redundancy interface.
Checking for multiple APP sessions between redundant peers.
App Session IP: 167.168.165.9, Local IP address: 167.168.165.10
Checking IP Address size differences : <167.168.165.9> <167.168.165.10>
Adding APP session IP length difference to LOCALCONFIG byte size  : 1
Adding APP session IP length difference to REMOTECONFIG byte size : 0
Removing CIRCUIT and INTERFACE modes from tmp.cfg.
Checking for ips with matching subnets for circuit sync.
Checking IP Address size differences : <167.168.165.10> <167.168.165.9>
Replacing local system VRID priorities and preempt settings with remote settings.
Local IP: 167.168.165.10 VRID: 40 PRIORITY: 254 PREEMPT: True
Remote IP: 167.168.165.9 VRID: 40 PRIORITY: 1 PREEMPT: False
Checking Number length differences : <254> <1>
Checking IP Address size differences : <167.168.165.132> <167.168.165.131>
Replacing local system VRID priorities and preempt settings with remote settings.
Local IP: 167.168.165.132 VRID: 41 PRIORITY: 254 PREEMPT: True
Remote IP: 167.168.165.131 VRID: 41 PRIORITY: 1 PREEMPT: False
Checking Number length differences : <254> <1>
Checking IP Address size differences : <167.168.166.3> <167.168.166.2>
Replacing local system VRID priorities and preempt settings with remote settings.
Local IP: 167.168.166.3 VRID: 42 PRIORITY: 254 PREEMPT: True
Remote IP: 167.168.166.2 VRID: 42 PRIORITY: 1 PREEMPT: False
Checking Number length differences : <254> <1>
IP addr bytes to add to LOCAL BYTE COUNT : 1
IP addr bytes to add to REMOTE BYTE COUNT : 1
Checking for SSL configuration ...
Working
Using rcmd to copy tmp.cfg to a file on Backup switch.
Archiving copy to Backup's startup-config.
Archiving Backup's current startup-config.
Restoring startup-config (new copy) to startup-config.
Clearing running-config.
Script playing the copy script of the Master's running-config.
Checking to make sure backup App goes down
Making sure app session is down.
Seconds to wait before calling it quits:    9168
Copy success being verified by comparing byte sizes of archived running-
     configs of the Master switch and the Backup switch.
Making sure app session is up.
Seconds to wait before calling it quits:    9108
Waiting for completion signal from remote switch ...
Verifying running-config copy success ...
Comparing the byte count now.
Accounting for preempt configurations.
Adding 0 bytes to LOCALCONFIG byte count
Adding 24 bytes to REMOTECONFIG byte count
Accounting for priority value length differences.
Adding 0 bytes to LOCALCONFIG byte count
Adding 6 bytes to REMOTECONFIG byte count
Accounting for IP address size differences.
Adding 1 bytes to LOCALCONFIG byte count
Adding 1 bytes to REMOTECONFIG byte count
File copy Vipr Config Sync Failed. Commit unsuccessful!
localconfig: 218346 bytes
remoteconfig : 218320    bytes

Answer 1

Hi Daniel,
According to this output, the synchronization process goes well, but it fails when it has to verify what it synchronized. If you look at the last two lines, you will see that the size of the original local configuration is slightly bigger than the remote one. That's causing the validation to fail.
The first thing we need to confirm is whether the configuration really got properly replicated (you will have to check line by line to make sure it's the same). If so, we would just need to figure out why the file sizes are different.
There were some cases in the past in which the hard drive on the secondary device was failing, and as a result, the file was getting corrupted, leading to the size mismatch, so that's a possibility we need to take into account. Anyway, it would probably be better to open a TAC case to have it investigated in more detail.
Regards
Daniel