Problem with local mirror syncing

Question

Hi mens! And sorry, my English is bad.
I rewrite script from here for syncing mirror for my local network.
That's what happened http://ix.io/1aK or http://paste.pocoo.org/raw/263591/ or permanent http://ix.io/user/atommixz
It support http (maybe ftp?) mirroring other repos. Using lftp for it.
#!/bin/bash
# The script to sync a local mirror of the Arch Linux repositories and ISOs
# Copyright (C) 2007 Woody Gilk <[email protected]>
# Modifications by Dale Blount <[email protected]>
# and Roman Kyrylych <[email protected]>
# and Vadim Gamov <[email protected]>
# and Aleksey Frolov <[email protected]>
# Licensed under the GNU GPL (version 2)
USECOLOR=yes
. /etc/rc.d/functions
# Filesystem locations for the sync operations
SYNC_HOME="/home/mirror"
SYNC_LOGS="$SYNC_HOME/logs"
SYNC_FILES="$SYNC_HOME/files"
SYNC_LOCK="$SYNC_HOME/mirrorsync.lck"
SYNC_REPO=(core extra community multilib iso archlinuxfr catalyst)
#SYNC_REPO=(archlinuxfr catalyst)
typeset -A REPO_URL
REPO_URL=(
[archlinuxfr]='http://repo.archlinux.fr/x86_64'
[catalyst]='http://catalyst.apocalypsus.net/repo/catalyst/x86_64'
#SYNC_SERVER=distro.ibiblio.org::distros/archlinux
SYNC_SERVER=mirror.yandex.ru::archlinux
RATE_LIMIT=50 # in kb/s
TIME_OUT=5 # in sec
# Set the format of the log file name
# This example will output something like this: sync_20070201-8.log
#LOG_FILE="pkgsync_$(date +%Y%m%d-%H).log"
LOG_FILE="pkgsync_$(date +%Y%m%d).log"
#Watchdog part (time in seconds of uninterruptable work of script)
# Needed for low-speed and/or unstable links to prevent
# rsync hunging up.
# New instance of script checks for timeout, if it occurs
# it'll kill previous instance, in elsecase it'll exit without
# any work.
WD_TIMEOUT=10800
# Do not edit the following lines, they protect the sync from running more than
# one instance at a time
if [ ! -d $SYNC_HOME ]; then
printhl "$SYNC_HOME does not exist, please create it, then run this script again."
exit 1
fi
if [ -f $SYNC_LOCK ];then
OPID=`head -n1 $SYNC_LOCK`;
TIMEOUT=`head -n2 $SYNC_LOCK|tail -n1`;
NOW=`date +%s`;
if [ "$NOW" -ge "$TIMEOUT" ];then
kill -9 $OPID;
fi
MYNAME=`basename $0`;
TESTPID=`ps -p $OPID|grep $OPID|grep $MYNAME`;
if [ "$TESTPID" != "" ];then
printhl "exit";
exit 1;
else
rm $SYNC_LOCK;
fi
fi
echo $$ > "$SYNC_LOCK"
echo `expr \`date +%s\` + $WD_TIMEOUT` >> "$SYNC_LOCK"
# End of non-editable lines
# Create the log file and insert a timestamp
touch "$SYNC_LOGS/$LOG_FILE"
printhl "Starting sync on $(date --rfc-3339=seconds)" | tee -a "$SYNC_LOGS/$LOG_FILE"
for repo in ${SYNC_REPO[@]}; do
repo=$(echo $repo | tr [:upper:] [:lower:])
printhl "Syncing $repo" | tee -a "$SYNC_LOGS/$LOG_FILE"
NEXT=false
for i in ${!REPO_URL[*]}; do
if [ $i = $repo ]; then
mkdir -p "$SYNC_FILES/$repo"
cd "$SYNC_FILES/$repo"
lftp -c "\
set xfer:log no; \
set net:limit-rate $[RATE_LIMIT * 1000]; \
mirror \
--delete \
--only-newer \
--verbose=3 \
${REPO_URL[$repo]}" | tee -a "$SYNC_LOGS/$LOG_FILE"
date --rfc-3339=seconds > "$SYNC_FILES/$repo.lastsync"
NEXT=true
#sleep $TIME_OUT
fi
done
if $NEXT; then continue; fi
rsync -rtvHh \
--bwlimit=$RATE_LIMIT \
--no-motd \
--delete-after \
--delete-excluded \
--prune-empty-dirs \
--delay-updates \
--copy-links \
--perms \
--include="*/" \
--include="latest/*x86_64.iso" \
--include="latest/*sum*.txt" \
--include="archboot/latest/*.iso" \
--include="os/x86_64/*" \
--exclude="*" \
$SYNC_SERVER/$repo "$SYNC_FILES" | tee -a "$SYNC_LOGS/$LOG_FILE"
# Create $repo.lastsync file with timestamp like "2007-05-02 03:41:08+03:00"
# which may be useful for users to know when the repository was last updated
date --rfc-3339=seconds > "$SYNC_FILES/$repo.lastsync"
# Sleep 5 seconds after each repository to avoid too many concurrent connections
# to rsync server if the TCP connection does not close in a timely manner
sleep $TIME_OUT
done
# Insert another timestamp and close the log file
printhl "Finished sync on $(date --rfc-3339=seconds)" | tee -a "$SYNC_LOGS/$LOG_FILE"
printsep >> "$SYNC_LOGS/$LOG_FILE"
# Remove the lock file and exit
rm -f "$SYNC_LOCK"
unset REPO_URL
exit 0
But I'm have problem. If I'm run
sudo pacman -Syu
on my server, it's fine work.
[atommixz@fileserver ~]$ date; sudo pacman -Syu; echo "------"; date; sudo pacman -Syu
Сбт Сен 18 19:55:47 MSD 2010
:: Синхронизируются базы данных пакетов...
core 35,7K 14,2M/s 00:00:00 [#############################################################] 100%
extra 465,9K 189,3M/s 00:00:00 [#############################################################] 100%
community 383,1K 198,6M/s 00:00:00 [#############################################################] 100%
archlinuxfr не устарел
:: Запускается полное обновление системы...
нечего выполнять
[atommixz@fileserver ~]$ date; sudo pacman -Syu
Сбт Сен 18 19:55:48 MSD 2010
:: Синхронизируются базы данных пакетов...
core не устарел
extra не устарел
community не устарел
archlinuxfr не устарел
:: Запускается полное обновление системы...
нечего выполнять
But if I'm try it on my desktop, it work wrong. Always reget base. But it is updated properly.
[atommixz@relentless ~]$ date; sudo pacman -Syu; echo "------"; date; sudo pacman -Syu
Сбт Сен 18 19:58:42 MSD 2010
:: Синхронизируются базы данных пакетов...
core 35,7K 34,0M/s 00:00:00 [#############################################################] 100%
extra 465,9K 58,7M/s 00:00:00 [#############################################################] 100%
community 383,1K 57,9M/s 00:00:00 [#############################################################] 100%
multilib 19,3K 34,7M/s 00:00:00 [#############################################################] 100%
archlinuxfr 18,6K 42,6M/s 00:00:00 [#############################################################] 100%
catalyst 2,2K 67,7M/s 00:00:00 [#############################################################] 100%
:: Запускается полное обновление системы...
нечего выполнять
Сбт Сен 18 19:58:43 MSD 2010
:: Синхронизируются базы данных пакетов...
core 35,7K 34,0M/s 00:00:00 [#############################################################] 100%
extra 465,9K 58,7M/s 00:00:00 [#############################################################] 100%
community 383,1K 64,8M/s 00:00:00 [#############################################################] 100%
multilib 19,3K 48,9M/s 00:00:00 [#############################################################] 100%
archlinuxfr 18,6K 38,9M/s 00:00:00 [#############################################################] 100%
catalyst 2,2K 55,5M/s 00:00:00 [#############################################################] 100%
:: Запускается полное обновление системы...
нечего выполнять
What am I doing wrong?

Answer 1

I'm not sure, but your script may be too old.  That is, it may no longer work.
I believe that creating a local mirror is discouraged due to the high bandwidth needed to do that.
Perhaps you could try this instead.