Check website for changes and send message

Question

I maintain several websites for others and would like to use a script to check to see if any of the files have changed in the last day.  In other words, go to http://www.mysite.com and if the date modified property of the index.html file is equal to today, then send an email to me.  I'm familiar with AppleScript enough to send the email message but am kinda stuck on how to accomplish the checking the date modified aspect of the files on the remote Unix server.
Is this a job that AutomatorAppleScript should handle?  Or would something else be more effective?  Any help would be greatly appreciated.

Answer 1

I'd probably look to use RSS, if you're maintaining the web sites.  That's what an RSS feed is set up to do.
If you're looking at a web site that's based on a content-management system (CMS), those usually have RSS options or features available, and those also often have databases that can be queried.  Scraping is, well, ugly and resource-intensive, and best avoided.  But if you have to...
The following is mostly Terminal.app Unix-level stuff...
If you want to look at the lower level (the hard way), then a tool such as cURL would be one typical tool.
cURL is installed on OS X by default.
Flags:
  -z/--time-cond <date expression>|<file>
  -I, --head
rsync can provide a completely different approach here.  With rsync, you can synchronize a local cache of the files with those on the remote servers.
AppleScript is an excellent choice for scripting OS X GUI applications.  But if you're not doing that, then a bash script, or Python, php, or some other scripting tool or scripting language would probably be my choice; those are better suited to your particular task.  As for some of these other tools, Python has a pycurl library, php has a curl library, and bash can call the curl utility.  These are all web-centered languages, so poking around via HTTP is standard fare.  And each has other approaches available.   I'd expect there's a scraping library or three around, too.  This is hardly a new or unusual requirement.  There's the Python scrapy tool, for instance.
details: man curl, man rsync, or the available bash or Python or Perl documentation.  And bash scripting is cryptic, but it's a gonzo powerful tool.  Among the TLDP guides are an introduction and an advanced bash shell scripting guide.