Run optipng png optimizer on several cores - optipng_multicore.sh

Question

So. It's a wrapper. I got bored when I have HEXA core on my server and I usually want to maximize (almost maximize) png optimization, but optipng as great tool it is - it lacks feature of running on multiple cores at the same time. So utilizing 1 of 6 of the cores in my case... led to this little implementation.
This script runs many optipng processes in unison/parallel to archieve threading-like support for optipng.
Secret lies within xargs.
Note that for now this script goes trough all the possible combinations/trials. I'm planning to change that to user defined.
It requires bash and xargs that are found from almost every Arch install. And, of course, optipng itself.
Here's the code (for you all to improve?):
#!/bin/bash
# optipng_multicore.sh
# optipng wrapper to use all the cores at maximum settings.
# Code: [email protected] Zucca@IRC (several networks)
# Licence: I guess MIT would be ok. Just mention my contact info if you want to fork/take over.
# Version 0.2.5
HELPMSG="
= optipng spawner for multiple CPU cores =
-f|--force Force overwriting files
-c|--num-cpu|-t|--threads Specify number of threads (default: automatic detection)
-h|--help Umm... It's a surprise?!?
Use '--' to stop searching switches."
# The dirty way to get CPU count (counts HT cores as two)
CPUCOUNT="$(grep -c ^processor /proc/cpuinfo)"
## Parse cli args ##
while [ "${1:0:1}" == "-" ]
do
case $1 in
-h|--help)
echo "$HELPMSG"
exit 0
-f|--force)
FORCE=true
-c|--num-cpu)
if egrep -q '^[0-9]+$' <<< "$2"
then
CPUCOUNT="$2"
else
echo '--num-cpu option must be followed by a number argument!' >&2
exit 1
fi
shift
--) # Stop searching switches
shift # break forces to exit the loop right here
# that's why we must shift here before
break
echo -e "DUDE! I don't know what to do with that switch \"$1\" of yours." >&2
exit 1
esac
shift
done
for PNGIN in "$@"
do
COUNTER=0
TMPDIR="$(mktemp -d)"
# Doing this "YO dawg" -loop is because one might want to change possible combinations
# ... and for the future development. ;)
for ZC in 1 2 3 8 9 # Compression levels
do
for ZM in {1..9}
do
for ZS in {0..3}
do
for F in {0..5}
do
for I in {0..1}
do
echo "-zc${ZC} -zm${ZM} -zs${ZS} -f${F} -i${I} -zw${ZW} -out \"${TMPDIR}/${COUNTER}.png\" \"$PNGIN\"" >> "$TMPDIR/list.txt"
let "COUNTER=$COUNTER+1" # It would be nice to have leding zeros but... meh.
done
done
done
done
done
# Start the compression
xargs -n 8 -P $CPUCOUNT -a "$TMPDIR/list.txt" optipng -quiet
# FIXME: a dirty way to remove all but the smallest size png. Maybe use find + stat
ls -Ss --block-size=1 "$TMPDIR"/*.png | head -n -1 | egrep -o "${TMPDIR}/[0-9]+\.png" | xargs rm
# Then move the rest (the only one)
let "WONBYTES=$(stat -c %s "$PNGIN")-$(stat -c %s "$TMPDIR"/*.png)"
echo "$PNGIN size reduced by $WONBYTES bytes."
test "$FORCE" == true && mv -f "$TMPDIR"/*.png "${PNGIN%.*}.png" || mv -i "$TMPDIR"/*.png "${PNGIN%.*}.png"
# Delete temp directory
rm -fr "$TMPDIR"
done
I could put this into AUR if I get enough ... um followers?
Last edited by Zucca (2013-04-13 12:08:35)

Answer 1

brunogm0 wrote:Nice wrapper, can you explain if xargs can auto detect  cpu cores or not ?
Afaik if you specify -P 0 (zero) xargs spans as many processes as it can. So bertter to determine cpu count by ourselves.
brunogm0 wrote:You can improve yours with my tunings and i can use your CPUCOUNT.
Thast's what the FOSS is for!
And thanks! I'll look if I can improve my script and I'll share everyting I discover/code here.
I really need to limit the number of trials on my script. -o5 level with few tweaks usually lead into desired size reduction.
I will also make my script more idiot-proof (like catching ^C and reporting possible errors).
Last edited by Zucca (2013-01-13 14:08:52)