Pacman cache

Hi,
here is a ruby script to generate a custom.db.tar.gz from the archive directory (/var/cache/pacman/pkg) and not use the abs source.
this way custom.db.tar.gz have only what are in the archive directory and not the full list, and pacman know what are in that repository.
i use it to have a cache directory to populate other computers in the local network, and pacman download from a local store if found it or from the net if not.
this script is in testing yet, not have help and command line options, if you need to change somethings, in the first lines are the paths that the script use.
if you don't trust this script, changing the 'destdir' variable to some user access directory, a non root user can execute this script an generate the custom.db.tar.gz
i hope this be helpfull for someone...  blames, patch, sugestion are welcome....
require 'pathname'
require 'fileutils'
class GenCache
public #----------------------------------------------------------
attr_reader :cache_dir, :tmp_dir, :infoFile, :destdir
attr_writer :cache_dir, :tmp_dir, :infoFile, :destdir
def initialize()
@cache_dir = '/var/cache/pacman/pkg/'
@tmp_dir = '/tmp/gencache/'
@infofile = '.PKGINFO'
@destdir = '/var/cache/pacman/pkg/'
@cache_name = 'custom.db.tar.gz'
end
def generateCache()
if protec( "createTmp", @tmp_dir, nil ) { |par1,par2| Dir.mkdir( par1 ) }
begin
# move to temp dir
Dir.chdir(@tmp_dir)
# generate filesystem structure
loopdir()
# tar filesystem structure
protec( "tar", @cache_name, "*" ) { |par1,par2| system( "tar czf "+ par1 + " " + par2 ) == 0 }
# move to repository
protec( "move", @cache_name, @destdir+@cache_name ) { |par1,par2| FileUtils.mv( par1, par2 ) }
rescue => err
# eat the error
ensure
# clean filesystem structure
protec( "cleanTmp", @tmp_dir, nil ) { |par1,par2| FileUtils.rm_r( par1 ) }
end
end
end
private #----------------------------------------------------------
def doMd5( hashfile )
hash = ""
open("|md5sum "+hashfile ) { |i| hash = i.gets.gsub(/[ t].*/,'') }
hash = "n%MD5SUM%n" + hash
return hash
end
def protec( method, param1, param2 )
rc = false
begin
if yield param1, param2
rc = true
end
rescue => err
puts "nError in GenCache::protec("+ method +") [" + err.message + "]"
raise
end
return rc
end
def writedata( datafile, package, fullname )
fst_groups = fst_replace = fst_depend = fst_conflict = fst_provides = true
have_md5 = false
desc = ""
depends = ""
datafile.each do |rc|
if rc[0,1] != "#"
if rc.strip.index("pkgname") == 0
desc += "%NAME%n"
desc += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("pkgver") == 0
desc += "n%VERSION%n"
desc += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("pkgdesc") == 0
desc += "n%DESC%n"
desc += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("md5sum") == 0
desc += "n%MD5SUM%n"
desc += rc.gsub(/^.*=[ t]/,'')
have_md5 = true
elsif rc.strip.index("group") == 0
if not have_md5
desc += doMd5( fullname )
have_md5 = true
end
if fst_groups
desc += "n%GROUPS%n"
fst_groups = false
end
desc += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("replaces") == 0
if fst_replace
desc += "n%REPLACES%n"
fst_replace = false
end
desc += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("depend") == 0
if fst_depend
depends += "%DEPENDS%n"
fst_depend = false
end
depends += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("conflicts") == 0
if fst_conflict
depends += "n%CONFLICTS%n"
fst_conflict = false
end
depends += rc.gsub(/^.*=[ t]/,'')
elsif rc.strip.index("provides") == 0
if fst_provides
depends += "n%PROVIDES%n"
fst_provides = false
end
depends += rc.gsub(/^.*=[ t]/,'')
end
end
end
if not have_md5
desc += doMd5( fullname )
have_md5 = true
end
begin
f = File.new( @tmp_dir+package+"desc", File::CREAT|File::TRUNC|File::RDWR, 0644)
f.write( desc )
f.close()
f = File.new( @tmp_dir+package+"depends", File::CREAT|File::TRUNC|File::RDWR, 0644)
f.write( depends )
f.close()
rescue => err
print "nerror creating generating control files ["+err.message+"]n"
raise
end
end
def loopdir()
loop=1
pth = Pathname.new( @cache_dir )
pth.dir_foreach do |i|
unless i.directory?()
if ( i.to_s.index( "db.tar.gz" ) == nil )
if protec( "untar", @cache_dir+i, @infofile ) { |par1,par2| system( "tar xzf "+ par1 + " " + par2 ) == 0 }
puts "Error untaring " + @cache_dir+i + " : " + $?.to_s
exit 1
end
package = i.to_s.gsub(/.pkg.tar.gz/,'') + "/"
Dir.mkdir( @tmp_dir + package )
writedata( IO.readlines( @infofile ), package, @cache_dir + i )
loop = loop + 1
print "."
end
end
end
puts "n" + loop.to_s + " files - "
end
end
cache = GenCache.new()
cache.generateCache()

Stebalien wrote:You could try: http://xyne.archlinux.ca/projects/pacserve/
It doesn't use NFS but should be faster.
I thought about using it but i didn't wanted to depend on a extra package. Before using it, i want to know if my problem is easily solvable.

Similar Messages

  • Another simple bash script to clean pacman cache

    here is a simple script that I have written which cleans pacman cache folder in a way that only the packages which are now "updated" in the repositories, will get removed. This is different from what "pacman -Sc" does. pacman -Sc also removes any package which is not installed, while this script keeps all the packages which are updated whether they are installed or not.
    The functionality is some how like what "apt-get autoclean" does in debian.
    to use this script you need to run it with "list" or "clean" arguments.  you can also use the "help" argument for more help.
    I hope it helps
    #! /bin/bash
    # clcache - This script cleans pacman cache folder in a way that only the packages
    #+ which are now updated in the repositories, will get removed. This is
    #+ different from what "pacman -Sc" does. pacman -Sc also removes any package
    #+ which is not installed, while this script keeps all the packages which are
    #+ updated whether they are installed or not.
    # I have tweaked this script to be as fast as possible, it might still need a
    #+ couple of minutes to compelete based on the size of your cache folder.
    # to use this script you need to run it with "list" or "clean" arguments.
    # you can also use the "help" argument for more help.
    # This script is written by "Ali Mousavi". Please report bugs to [email protected]
    DIR="/var/cache/pacman/pkg" #the default directory of pacman cache.
    ROOT_UID=0 #Only users with $UID 0 have root privilages.
    TMPFILE="/tmp/cache.tmp"
    # Run as root
    if [ "$UID" -ne "$ROOT_UID" ]
    then
    echo "Must be root to run this script"
    exit 1
    fi
    # Check for the arguments
    if [ -z "$1" ]
    then
    echo -e 'What should I do?\nValid Argument are "list", "clean" or "help"'
    exit 1
    elif [ "$1" = "list" ]
    then
    ACTION="ls"
    MESSAGE="Are you sure you want to continue?"
    elif [ "$1" = "clean" ]
    then
    ACTION="rm -vf"
    MESSAGE="Are you sure you want to remove outdated packages? This process can not be undone!"
    elif [ "$1" = "help" -o "$1" = "-h" -o "$1" = "--help" ]
    then
    echo -e "This script checks the packages in your pacman cache directory and removes the packages that are outdated. It doesn't matter if the package is installed or not.\n\n3 arguments can be passed to the script:\n\nlist:\n\tchecks for package that are outdated and prints the names.\n\nclean:\n\tremoves outdated packages.\n\nhelp,-h,--help:\n\tprints this help text.\n\nThis script is written by \"Ali Mousavi\". Please report bugs to [email protected]"
    exit 0
    else
    echo 'Valid Argument are "list", "clean" or "help"'
    exit 1
    fi
    # Check if the user is sure!
    echo "This might take a while based on the amount of cached packages."
    echo -n "$MESSAGE(y/n) "
    read ANS
    if [ $ANS = "y" -o $ANS = "Y" -o $ANS = "yes" ]
    then
    echo "Processing packages..."
    elif [ $ANS = "n" -o $ANS = "N" -o $ANS = "No" ]
    then
    echo "Exiting on user request"
    exit 0
    else
    echo "Invalid answer"
    exit 1
    fi
    # Process the packages
    cd $DIR #change to cache directory.
    pacman -Sl | awk '{ print $2" "$3; }' > $TMPFILE
    for f in $(ls $DIR)
    do
    pname=$(file $f | cut -d: -f1) #Produces filename, like: fetchmail-6.3.19-1-i686.pkg.tar.xz"
    spname=$(echo $pname | sed 's/-[0-9][0-9]*.*//g') #removes package version: fetchmail
    pver=$(echo $pname | sed 's/.*-\([0-9\-\.][0-9\-\.]*-[0-9\-\.][0-9\-\.]*\).*/\1/g') #using pacman -Qi for all files takes a lot of time.
    if [ $(echo $pver | grep '[^0-9\-\.\-\-]' | wc -l) != 0 ] #checks if package version is alright
    then
    pver=$(pacman -Qpi $f | grep Version | awk '{print $3}')
    fi
    newpver=$(grep -e "^$spname " $TMPFILE | awk '{ print $2 }')
    if [[ $newpver != $pver ]]
    then
    $ACTION $f
    fi
    done
    rm -f $TMPFILE
    echo "Outdated packages processed successfully!"
    exit 0
    Last edited by tuxitop (2011-09-13 09:24:26)

    tuxitop wrote:# Check for the arguments
    if [ -z "$1" ]
    then
    echo -e 'What should I do?\nValid Argument are "list", "clean" or "help"'
    exit 1
    elif [ "$1" = "list" ]
    then
    ACTION="ls"
    MESSAGE="Are you sure you want to continue?"
    elif [ "$1" = "clean" ]
    then
    ACTION="rm -vf"
    MESSAGE="Are you sure you want to remove outdated packages? This process can not be undone!"
    elif [ "$1" = "help" -o "$1" = "-h" -o "$1" = "--help" ]
    then
    echo -e "This script checks the packages in your pacman cache directory and removes the packages that are outdated. It doesn't matter if the package is installed or not.\n\n3 arguments can be passed to the script:\n\nlist:\n\tchecks for package that are outdated and prints the names.\n\nclean:\n\tremoves outdated packages.\n\nhelp,-h,--help:\n\tprints this help text.\n\nThis script is written by \"Ali Mousavi\". Please report bugs to [email protected]"
    exit 0
    else
    echo 'Valid Argument are "list", "clean" or "help"'
    exit 1
    fi
    1. `echo -e 'foo\nbar\nbaz'` gets long and unreadable quickly. Instead, use here documents:
    cat <<EOF
    What should I do?
    Valid Argument are "list", "clean" or "help"
    EOF
    2. Use a case command, looks cleaner:
    case "$1" in
    list) ... ;;
    clean) ... ;;
    help|-h|--hep) ... ;;
    # Check if the user is sure!
    echo "This might take a while based on the amount of cached packages."
    echo -n "$MESSAGE(y/n) "
    read ANS
    if [ $ANS = "y" -o $ANS = "Y" -o $ANS = "yes" ]
    then
    echo "Processing packages..."
    elif [ $ANS = "n" -o $ANS = "N" -o $ANS = "No" ]
    then
    echo "Exiting on user request"
    exit 0
    else
    echo "Invalid answer"
    exit 1
    fi
    Try:
    read -p "hello: " -r
    echo $REPLY
    And again, `case` should be cleaner in this case.
    # Process the packages
    cd $DIR #change to cache directory.
    pacman -Sl | awk '{ print $2" "$3; }' > $TMPFILE
    While you quoted a lot, you left these two out. "$DIR" and "$TMPFILE" should be quoted, otherwise whitespaces will break the code.
    for f in $(ls $DIR)
    Apart from the same missing quotes, calling `ls` is a waste here. The following is sufficient and (maybe surprisingly) more accurate:
    for f in *
    How is it more accurate? Run this test script:
    #!/bin/bash
    DIR=/tmp/foo
    mkdir -p "$DIR"
    cd "$DIR"
    touch a\ b c$'\n'd
    for i in *; do
    printf '+%s+\n' "$i"
    done
    printf '%s\n' ---
    for i in $(ls $DIR); do
    printf '+%s+\n' "$i"
    done
    Let's not go too far here. Just get the idea.
    do
    pname=$(file $f | cut -d: -f1) #Produces filename, like: fetchmail-6.3.19-1-i686.pkg.tar.xz"
    Calling `file` here is, again, unnecessary. Also, filename of a package can contain ":", e.g., vi-1:050325-1-i686.pkg.tar.xz, which breaks your code.
    Don't complicate things:
    pname=$f
    spname=$(echo $pname | sed 's/-[0-9][0-9]*.*//g') #removes package version: fetchmail
    Broken for ntfs-3g-2011.4.12-1-i686.pkg.tar.xz, nvidia-173xx-utils-173.14.31-1-i686.pkg.tar.xz, etc...   Something less lousy:
    sed 's/\(-[^-]\+\)\{3\}$//' <<< "$pname"
    pver=$(echo $pname | sed 's/.*-\([0-9\-\.][0-9\-\.]*-[0-9\-\.][0-9\-\.]*\).*/\1/g') #using pacman -Qi for all files takes a lot of time.
    Although this might work for now, this would break if we had an architecture that starts with a digit, e.g. 686.  Something less lousy:
    sed 's/\(.*\)-\([^-]\+-[^-]\+\)-[^-]\+$/\2/' <<< "$pname"
    if [ $(echo $pver | grep '[^0-9\-\.\-\-]' | wc -l) != 0 ] #checks if package version is alright
    then
    pver=$(pacman -Qpi $f | grep Version | awk '{print $3}')
    fi
    Again, calling `wc` here is a waste. Anyway, why is this check necessary at all?
    newpver=$(grep -e "^$spname " $TMPFILE | awk '{ print $2 }')
    if [[ $newpver != $pver ]]
    then
    $ACTION $f
    fi
    done
    rm -f $TMPFILE
    echo "Outdated packages processed successfully!"
    exit 0
    The post is getting too long, so so much from me. If there's anything you don't understand yet, read bash(1).
    If I sound harsh or anything, don't be discouraged. Just keep reading, keep improving.
    Last edited by lolilolicon (2011-09-13 12:53:04)

  • [Solved] What is the best way to share pacman cache ?

    I'm wondering what is the best way to share the pacman cache .
    I went with the obvious choice 'sshfs' . But the cache on the server is owned by root so I can't use it dynamically unless I enable root logins , right ?
    Is it safe to enable root logins and mount with the root user ?
    What is the best and safest way to share pacman cache dynamically ?
    Last edited by Nezmer (2008-12-07 01:13:20)

    Interesting. I use sshfs and yes use root login to do business. But I see it this way you want more security created a stronger password. You want more security than that disable passwords and use keys. But once you start NFS you are surely introducing some security issue just as if you enable root ssh login, so why is NFS better?
    Just asking out of interest, I claim to know nothing about security.

  • Run a self-updating network shared pacman cache

    About a month ago I began working out how to share a pacman cache without the problems that the solutions proposed until now had.
    I wanted my cache to be:
    easily configurable
    in control of the packages in the cache
    capable of handling multiple clients simultaneously
    self-updating
    Especially the last point comes in handy if you frequently want to update your system without having to wait for a package download first.
    All you need for this to work is a device that is able to share files per SAMBA/NFS and that is online most of the time (preferably 24/7).
    If you're interested, head over to github and check it out.
    Last edited by n0stradamus (2012-05-28 13:36:28)

    Yep, well capable of doing that which is why I'm asking - I can't do that without having to have difficult conversations with the security team here. And, no, I don't need to be told about the lack of security implications in doing this
    As far as keys, I've just set the SigLevel to "Never" as I trust the packages on my workstation, but this all feels a little hinky.

  • Install Arch with nfs shared pacman cache

    Hello,
    I would like to make a post of some information that worked for me. I think it could find its way into the wiki, but I prefer not to be the one to do that. Two reasons for that:
    1. Just becuase it worked for me doesn't mean it is proper. If someone can confirm it is an acceptable method...
    2. I am a wiki reader, not a writer
    My situation is that I have a 3G internet conection. I like to only download the packages once... I run nfs4 exports from my desktop computer to share user data, pacman cache and sync folders See the Pacman tips I just did fresh installs of my systems. After the main system was running, I wanted to use the existing pacman cache to install the second system. I followed the Beginners guide and also used the Install from ssh article. I created my partitions with gparted on a live stick. Boot into the Arch install media, and follow the instructions for mounting the partitions. And here is where I make a brief detour to utilize my existing pacman cache.
    For reference, my nfs exports are on the desktop computer (hostname is nas) and the pkg cache is exported as:
    nas:/pkg
    Databases are exported as:
    nas:/sync
    mkdir -p /mnt/var/cache/pacman/pkg
    mkdir -p /mnt/var/lib/pacman/sync
    mount the nfs exports:
    mount -t nfs4 nas:/pkg /mnt/var/cache/pacman/pkg
    mount -t nfs4 nas:/sync /mnt/var/lib/pacman/sync
    Now I am back on track with the Beginners guide...
    pacstrap -i /mnt base base-devel
    While in the chroot, be sure to install nfs-utils (and openssh if you want to continue ssh after boot)
    When running the genfstab, it will create tons of options for the nfs4 mounts. I edited those in nano to look like other references on the site:
    nas:/pkg /var/cache/pacman/pkg nfs4 defaults,x-systemd.automount 0 0
    nas:/sync /var/lib/pacman/sync nfs4 defaults,x-systemd.automount 0 0
    And of course unmounting at the end of the Beginners guide requires unmounting these two directories too.

    Yep, well capable of doing that which is why I'm asking - I can't do that without having to have difficult conversations with the security team here. And, no, I don't need to be told about the lack of security implications in doing this
    As far as keys, I've just set the SigLevel to "Never" as I trust the packages on my workstation, but this all feels a little hinky.

  • [SOLVED] Make Pacman cache packages installed with -U

    When installing a package with -U, pacman doesn't cache packages as it does when installing with -S. Is there a way to change this behavior?
    I read somewhere that specifying the package's full path and prepending "file://", as in "pacman -U file://path/to/package.pkg.tar.xz", but this often complains of an "invalid or corrupted package (PGP signature)".
    How can I make pacman always copy installed packages to the cache?
    MOD EDIT: change 'closed' to 'solved' in title to avoid confusion
    Last edited by fukawi2 (2013-07-06 13:09:21)

    You can't.  This has been discussed on the bugtracker several times.
    https://bugs.archlinux.org/task/35699
    https://bugs.archlinux.org/task/31243
    https://bugs.archlinux.org/task/15143
    https://bugs.archlinux.org/task/18012
    If you're building the packages yourself with makepkg, you can set PKGDEST to your cache in makepkg.conf.  Otherwise you could write a small pacman wrapper that copies the package after installing it.

  • Partial clearing of pacman cache

    Is there a way of partially clearing the cache of old packages. I'll explain, every now and then I have to downgrade a package, due to some sort of issue, (usually wine or ati packages). Down grading to the previous version is usually good enough, but sometimes I need to go back further. So that I can do this I don't clear the cache to frequently.
    Now I know that doing a "pacman -Sc" will remove all the old packages upto the currently installed one, and a "pacman -Scc" removes everything. But it would be nice (for me anyway) if you could add a parameter to the "c" to leave that many versions of a package in the cache
    example "pacman -Sc=3" this would leave the 3 latest versions of packages alone, but clear out anything older. If there wasn't 3 versions of a package, in the cache, then it would be left alone.
    People could keep the number of copies that they feel comfortable with, or not specify a number and things would work as they currently do.
    Does anyone know of a way to do a partial clearing alongs the lines discribed?
    Thanks

    test1000 wrote:
    you could always go into /var/cache/pacman/pkg/ and delete the packages you don't need.
    posting your suggestion to the bugzilla might yield better chances of it being implemented as a feature.
    Thanks for the input, I was just asking to see if anyone new of a way to achieve this currently, and also to see if it other thought it would be useful.

  • Sharing pacman cache in same machine between different Arch installs

    How may I ask?

    This is how I do that!
    Download pacman packages into a cachedir located in a flash drive.  Package the individual packages with depends in the flash drive such that they will be installed fully.
    The cache is then accessed with pacman to install the selected package.
    The parameters are:
      pacman --cachedir /media/disk/upgrade -Sw (package name(s))
    This places the .pkg.tar.gz into /media/disk/upgrade.  At this point, gather the elements of the depends and package into a folder with an ID such as open-office1.  Difficult to explain for all packages because some are not includiing depends!
       Then to install at your convenience,;
         pacman -Ud /media/disk/upgrade/open-office1/*.pkg.tar.gz.........
    This will install the packages contained in open-office1 at 15-20MB/sec.
    This can be portable via the flash device.
    Hope it fills your needs.................
    EDIT: Normal pacman downloads can still be handled via ...pacman -S package name because the cachedir is not imposed.
    EDIT:  You could copy the entire /var/cache/pacman/pkg into the cachedir if desired altho the depends may not be available therein , so the method I described may be best.
    Last edited by lilsirecho (2008-02-11 22:05:47)

  • [SOLVED] Network Shared Pacman Cache problems

    Hi all,
    I've got a couple of Arch servers and an Arch workstation that I'm working on; only the workstation has internet access and I need to keep the servers up to date and be able to install packages.
    At the moment, the servers have the workstation's pacman package and cache directories mounted via SSHFS, but I've got problems with keys (pacman not being able to look up keys) and system upgrades. Nothing that I can't work around, but I'm thinking there must be a more "professional" way to get this done.
    TL;DR - What's the best way to set up internetless Arch servers with a network pacman repo/cache?
    Last edited by syngin (2012-04-03 07:00:21)

    Yep, well capable of doing that which is why I'm asking - I can't do that without having to have difficult conversations with the security team here. And, no, I don't need to be told about the lack of security implications in doing this
    As far as keys, I've just set the SigLevel to "Never" as I trust the packages on my workstation, but this all feels a little hinky.

  • Pacman cache share over LAN is a pain!

    Hi fellows. As Arch Wiki - Pacman Tips say, i setup a nfs server in my home server so all the laptops can upgrade packages more quickly and without increasing load in the servers.
    It all sounds great, but turns out there is one big problem. When a client is using the server to upgrade, the integrity check of packages takes a very long time. And believe me, downloading them is even faster than the that.
    Is there something I can tweak to make this faster?
    This is my fstab line in each client:
    myserver:/var/cache/pacman/pkg /var/cache/pacman/pkg nfs4 defaults 0 0
    And this is the line of /etc/exports in the server:
    /var/cache/pacman/pkg *(rw,async,subtree_check,no_root_squash)
    Thanks for your help in advance!

    Stebalien wrote:You could try: http://xyne.archlinux.ca/projects/pacserve/
    It doesn't use NFS but should be faster.
    I thought about using it but i didn't wanted to depend on a extra package. Before using it, i want to know if my problem is easily solvable.

  • Clearcache - another script to selectively clear your pacman cache.

    https://aur.archlinux.org/packages.php?ID=50498
    I learned there are a number of scripts designed to do the same thing after I
    wrote the basic framework, but I went ahead and finished it anyway.
    clearcache has a number of features I haven't seen elsewhere,
    usage: clearcache [options] [targets]
    Options:
      -h, --help          display this help menu
      -d, --dry           dry run, simply list targets instead of removing
      -q, --quiet         only prints filenames for -d/--dry or -l/--list-targets
      -f, --force         force removal of package files
      -i, --ignore <pkg>  specify package(s) to ignore. this can be a comma
                          separated list, or '-' reads a list from stdin, one
                          per line. can be used more than once.
      -k, --keep <num>    specify number of old package files to keep. by default,
                          clearcache will keep the two most recent versions.
      -l, --list-targets  list targets to remove, using PAGER/less, before
                          removing. may not be used with -d/--dry
      -t, --threads <num> use <num> threads. the default value is 5. using a
                          large number will not gain you much, and is not
                          recommended, but there is no hard limit.
      -u, --uninstalled   remove all package files for packages that are not
                          currently installed, while still keeping the
                          specified number of old files for installed packages
      -v, --verbose       output more
      there MUST be a space between the option and its argument.
    Targets:
      targets are package names, as you would specify them to pacman.
      if targets are specified, only package files belonging to them will be
      removed. if no targets are specified, all packages except for thos
      specified to be ignored will be removed.
    It now uses the .PKGINFO file within the package itself, in order to know which
    package it belongs to (Thanks again brisbin).  This eliminates any issues with
    parsing the name from the version info, in the filename.
    Obviously using alpm or such would be faster, but this was a quick project.  I
    wrote a short snippet for myself, and decided to extend it.
    Anyway, hope someone else finds this useful, let me know if you find any bugs,
    or have any feature reqs.
    Potential issues:
      As seems to be the most common issue with scripts like this, it assumes the
      most recent package in the cache is the one installed, or the one you would
      want to keep.  I'm considering parsing pacman output to compare the versions,
      at least as an option.
    EDIT:
      Now with multithreading!  It has taken about 10% off the execution time on a single
      core netbook, and gets faster on better processors.  If nothing else, it was a good
      experiment, but I think it turned out rather well.
    Last edited by freak (2011-07-07 01:56:45)

    https://aur.archlinux.org/packages.php?ID=50498
    I learned there are a number of scripts designed to do the same thing after I
    wrote the basic framework, but I went ahead and finished it anyway.
    clearcache has a number of features I haven't seen elsewhere,
    usage: clearcache [options] [targets]
    Options:
      -h, --help          display this help menu
      -d, --dry           dry run, simply list targets instead of removing
      -q, --quiet         only prints filenames for -d/--dry or -l/--list-targets
      -f, --force         force removal of package files
      -i, --ignore <pkg>  specify package(s) to ignore. this can be a comma
                          separated list, or '-' reads a list from stdin, one
                          per line. can be used more than once.
      -k, --keep <num>    specify number of old package files to keep. by default,
                          clearcache will keep the two most recent versions.
      -l, --list-targets  list targets to remove, using PAGER/less, before
                          removing. may not be used with -d/--dry
      -t, --threads <num> use <num> threads. the default value is 5. using a
                          large number will not gain you much, and is not
                          recommended, but there is no hard limit.
      -u, --uninstalled   remove all package files for packages that are not
                          currently installed, while still keeping the
                          specified number of old files for installed packages
      -v, --verbose       output more
      there MUST be a space between the option and its argument.
    Targets:
      targets are package names, as you would specify them to pacman.
      if targets are specified, only package files belonging to them will be
      removed. if no targets are specified, all packages except for thos
      specified to be ignored will be removed.
    It now uses the .PKGINFO file within the package itself, in order to know which
    package it belongs to (Thanks again brisbin).  This eliminates any issues with
    parsing the name from the version info, in the filename.
    Obviously using alpm or such would be faster, but this was a quick project.  I
    wrote a short snippet for myself, and decided to extend it.
    Anyway, hope someone else finds this useful, let me know if you find any bugs,
    or have any feature reqs.
    Potential issues:
      As seems to be the most common issue with scripts like this, it assumes the
      most recent package in the cache is the one installed, or the one you would
      want to keep.  I'm considering parsing pacman output to compare the versions,
      at least as an option.
    EDIT:
      Now with multithreading!  It has taken about 10% off the execution time on a single
      core netbook, and gets faster on better processors.  If nothing else, it was a good
      experiment, but I think it turned out rather well.
    Last edited by freak (2011-07-07 01:56:45)

  • Pacman transparent HTTP cache

    Hey all,
    I'm a fairly new Arch user, and to be honest, a fairly new Linux user in general. Over the years I tried so many times to make the transition from Windows to Linux and after finding Arch, I felt it was finally time, but I digress...
    I now run 2 Arch machines here and one Arch virtual machine, but I found myself with a little problem. Having 3 machines meant that it required either 3 times the bandwidth to keep them updated, or a lot of hassle with copying packages around the network. I even experimented with using the Pacman cache directory on an NFS share, but none of these were acceptable to me.
    So this afternoon I sat down and coded a solution I was happy with.
    I run a small Linux server here, Debian based, which serves as a small file server to the local network and also hosts a few services for myself and a friend. This server also runs Lighttpd which I use for developing with PHP and Perl.
    The idea was that this machine would be a mirror for Arch to update from, but I didn't want to mirror everything, just those packages which I used. After much searching through Google I discovered someone who had done something similar for Debian based distributions, apt-cache. It's essentially a small web-server which, when queried for a package, first checks it's local cache, and if it doesn't find the file, it downloads it from an official mirror, both storing it locally and sending it to the client.
    I've never coded in Java personally and I didn't want to have 2 web-servers running when one would suffice, so I set about coding something similar in PHP.
    The end result is 130 lines of code, and a url.rewrite rule, which achieves exactly what I was after. It works like this:
    1) Pacman requests a file from the local server
    2) Local server checks to see if it has the file
    3a) Local server cannot find the file so it requests it from an Arch mirror
    4a) The file is simultaneously downloaded, written to disk and sent to Pacman.
    3b) Local server has the file
    4b) The file is sent to Pacman
    The end result is a transparent cache which will only have to download the file once. An example of the speed increase is as follows:
    # pacman -S --downloadonly kernel26
    kernel26 21.7M 806.4K/s 00:00:28
    # rm /var/cache/pacman/pkg/kernel26-2.6.21.5-1.pkg.tar.gz
    # pacman -S --downloadonly kernel26
    kernel26 21.7M 8.5M/s 00:00:03
    As you can see, after the package was cached on the server, it didn't need re-downloading and as such it transferred to the local machine at LAN speeds.
    My mirror entries for the repositories looks like this:
    Server = http://192.168.0.1/pacman-cache/pkg/current/os/i686/
    Server = http://192.168.0.1/pacman-cache/pkg/extra/os/i686/
    etc....
    So, my question is this; would anyone out there be interested in the code? Right now it still needs a lot of work before it could be made public as there's very little error checking; I need to handle unexpected conditions like a broken download, and I also have to add handling to deal with the db.tar.gz files being updated, but as and when I feel it's ready would anyone use it?
    I'd appreciate any input anyone felt like sharing, even feature requests =^.^=
    PS: I hope this is in the right sub-forum... I didn't think it belonged in the actual Pacman forum, but if it did, apologies!

    Just a little update
    I solved the timeout problem. It wasn't a misconfiguration but rather a bug in the code that was causing it to stall randomly when retrieving a remote package, it'll now happily retrieve multiple packages without a problem.
    The following new features have been added since I last posted:
    Logging support
    It's a little primitive, but it works. The location of the log file is customizable so it should be possible for logrotate to work with it, hence I've not added any rotation system of my own.
    Setup support
    The cache script now functions correctly when initially installing Arch via the /arch/setup script. I've run it through once or twice, but it could do with further testing.
    Testing & Unstable repository support
    Self explanatory really
    Currently the script only supports the i686 architecture (due to some hard-coded paths), I'd need to do some recoding to support x86_64 as well, it's something I'm considering, assuming I can get VMWare to run a 64bit Arch install. Resuming is still on the "maybe" pile as I'm still trying to come up with a way of coding it cleanly, it's a case of trying to balance effort vs reward on this one.
    I'm also currently working on a quick-and-simple administration interface for the cache. It should let you see what files are cached, remove selected ones or an entire repository worth of cache. Perhaps even have the ability to verify the local files against the md5 summary files I build. It's in the early stages right now, but it'll hopefully be complete in the near future.
    Overall it works very well and depending on how many bugs I run into when giving it a real test, I should be able to release it here in the not-too-distant future, then anyone who's interested can play with it and perhaps improve it beyond my original design.

  • By default a diferent folder to cache of pacman

    I try understand if it's possible.
    I've installed arch with a 10Gb partition, and  now I'm having troubles with free space. There's some way that could I use change default place of pacman's cache?
    It'll be in /home/user folder. I've created a folder there....but, I forget use the "default dir option" on pacman.
    Another issue.... does yaourt will use the same folder to?

    Awebb wrote:
    Allan wrote:No idea what yaourt will do, but given it is a wrapper to pacman, it should work too.
    It will not store the .xz files created my makepkg in the pacman cache. It puts them into /tmp, where they are safe…
    It will store them where you tell it to store ...

  • [SOLVED] error pacman: /var/cache/pacman/pkg exists in filesystem

    Hi guys, yesterday night I tried to update my system and I got the following error which seems weird:
    resolving dependencies...
    looking for conflicting packages...
    Package (1) Old Version New Version Net Change
    core/pacman 4.2.0-5 4.2.0-6 0.00 MiB
    Total Installed Size: 4.22 MiB
    Net Upgrade Size: 0.00 MiB
    :: Proceed with installation? [Y/n] y
    (1/1) checking keys in keyring [##########################################################] 100%
    (1/1) checking package integrity [##########################################################] 100%
    (1/1) loading package files [##########################################################] 100%
    (1/1) checking for file conflicts [##########################################################] 100%
    error: failed to commit transaction (conflicting files)
    pacman: /var/cache/pacman/pkg exists in filesystem
    Errors occurred, no packages were upgraded.
    It seems weird to me because it conflicts to the whole folder and not to any specific package. Any clue, what is going on and how to solve it?
    Last edited by theodore (2015-01-26 19:51:37)

    clfarron4 wrote:
    theodore wrote:
    Allan wrote:Is your /var/cache/pacman/pkg a directory or a symlink?
    yes I have it in a different partition and then I have symilinked it
    I'd have thought it would have been better to set a different directory for the pacman cache, mounting the partition to that through /etc/fstab and using the CacheDir variable in /etc/pacman.conf, but I'll leave that to your judgement.
    If you don't want to do that, then I'll leave someone else to put up a solution (my brain's screaming bind mounts for some unknown reason).
    So, you are saying that I should enable the CacheDir in /etc/pacman.conf and point it to the partition where I have now the pacman cache.

  • Getting pacman to cache local packages (from AUR)

    I want pacman to cache the packages that I build from the AUR. I know that you can trick pacman into doing this by using file://path/to/pkg.
    What I want to know is how to make this behavior default for makepkg when it installs built packages using the -i switch. Is that possible?

    Yeah, the AUR script.
    Thanks.. I missed paccache's ability to look in custom directories.
    For now it looks like the best option is what you suggested, then.. To have makepkg put packages into an ~/aurcache of sorts.
    progandy wrote:Maybe there should be an option to add user defined directories.
    Either that, or an option for makepkg to append file:// when installing built packages.
    In the end, though, I'd really prefer to have the packages end up in the actual pacman cache because my root and home partitions are set up with the consideration that the root partition will get full of stuff like cached packages, and not my home partition.
    Last edited by gilmoreja (2015-03-03 22:12:51)

Maybe you are looking for

  • Why can't i access my CC apps on more than one computer?

    I installed a trial version of Adobe CC on my LAPTOP. I used my work email address as i knew we would be ugrading with Adobe CC for teams. After 30 days my license expired. I also installed a trial version on my DESKTOP. Our IT guy then transferred a

  • How to display a button in every row in a report page

    I am new to HTML DB.I created an application based on an existing table. In the report page I want to include a button in every row that is returned. Can anyone suggest as to how this needs to be done. Thanks, Srini.

  • Credit card entry problem

    Had to make a new apple id and when I go to enter my credit card it keeps telling me to contact support. It is the same card that I used on my old apple id

  • Question about saving calendar and contatcs

    Good Morning to all , I have a question about how to transfer the calendar and contatcs files to the new outlook profile after a imap migration. So what is the best way to transfer this kind of data before the imap migration for example to prep 70 ma

  • Microsoft Indexing Service and Dynamic PDF (XFA)

    I setup Indexing Services in Windwos Server 2003, and I confirmed that it can index (out of the box) Static PDF (created LiveCycle Desinger), in addition to other file types. However, the Dynamic PDF (also created with LiveCycle Desinger) are not ind