On-demand scanning for only new files or scan a list from a file

Question

I run a free to use file hosting service and with it are the risks of abuse for malware. In the past I've used ClamAV with a bash script to search for any files that are as old or newer than X minutes, usually about 5 minutes and put the found files listed line by line in a file and ClamAV will scan those files only. Put on a Cron for every 5 minutes to run. ClamAV's detection is pathetic though and has not caught most of the stuff it has needed to find. So switching over to Sophos Linux, it has done a great job already detecting multiple files of the uploaded files folder that ClamAV had already missed in the past. It takes just 3 1/2 minutes for it to scan through 3500+ files and archives totaling 17 gigs. I have it set to scan the whole directory every 5 minutes. This takes a while, even if it is under 3 1/2 mins. Ideally I want Sophos to only scan new files so it can be set to run every minute, or every 5 minutes, whether that be from it's command line or via it reading a file list. This way newly uploaded malware is taken care of as fast as possible for the security of the end users.  Here is the script file I use for Sophos already: #! /bin/bash
PATH=/home/user:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
if [[ $UID -ne 0 ]]; then sudo "$0"; exit 0; fi
# Directories to scan
scan_dir="/home/user/directory/files/"
# Virus relocation folder
virus_vault="/home/user/directory/viruses/files/"
# Create new log folder if it doesn't exist
mkdir -p /home/user/directory/viruses/logs/$(date +%Y-%m)
# Location of log file
tmp_log_file="/home/user/directory/viruses/logs/$(date +%Y-%m)/sophos-scan-$(date +%Y-%m-%d)-tmp.log"
log_file="/home/user/directory/viruses/logs/$(date +%Y-%m)/sophos-scan-$(date +%Y-%m-%d).log"
# Scan files and remove (--remove) infected
savscan $scan_dir -nc -ss -archive -all -suspicious -pua --quarantine -move=$virus_vault -p=$tmp_log_file
# Format log
echo " " >> "$log_file"
echo " " >> "$log_file"
echo "-----------------" >> "$log_file"
echo " " >> "$log_file"
date >> "$log_file"
echo " " >> "$log_file"
echo "-----------------" >> "$log_file"
echo " " >> "$log_file"
# move consolidate log to daily file
cat $tmp_log_file >> $log_file
# remove temporary file
rm -f $tmp_log_file
# make sure virus files are right owners
chown -R user:user $virus_vault Here is what it's based off, what I did for ClamAV to find new files and scan those only. #! /bin/bash
if [[ $UID -ne 0 ]]; then sudo "$0"; exit 0; fi
# Directories to scan
scan_dir="/home/user/example.com/files"
# Virus relocation folder
virus_vault="/home/user/example.com/viruses/files"
# Temporary file
list_file=$(mktemp -t clamscan.XXXXXX) || exit 1
# Create new log folder if it doesn't exist
mkdir -p /home/user/example.com/viruses/logs/$(date +%Y-%m)
# Location of log file
log_file="/home/user/example.com/viruses/logs/$(date +%Y-%m)/clamscan-$(date +%Y-%m-%d).log"
# Make list of new files
if [ -f "$log_file" ]
then
# use newer files then logfile
find "$scan_dir" -type f -cnewer "$log_file" -fprint "$list_file"
else
# scan last 5 minutes
find "$scan_dir" -type f -cmin -5 -fprint "$list_file"
fi
if [ -s "$list_file" ]
then
# Scan files and remove (--remove) infected
clamscan -f "$list_file" --move=/"$virus_vault" >> "$log_file"
rm -f "$list_file"
else
# remove the empty file, contains no info
rm -f "$list_file"
fi
exit 

Answer 1

Quick comment...
Sophos has 'decision caching' to speed up files that haven't changed since the last scan, while also allowing files to be re-checked against the latest threat definitions.