How to automatically google a string periodically from a shell script?

Question

I am trying to figure out the best way to automatically google search on a certain given string, periodically (say once per hour). I think I can set up the launchd script properly, but am wondering how one might use curl or wget to search a specific string (say, for example, the Fourth Amendment to the US Constitution).
I figure it might be nice to have that firmly in my database.

Answer 1

Hey Bill,
There are many ways to do this in unix. Here's a wget example:
Note, first I got the URL that google would use to search for the 4th amendment, this is:
http://www.google.com/search?client=opera&rls=en&q=4th+amendment&sourceid=opera& ie=utf-8&oe=utf-8
Next I shrunk it down to a more manageable size using www.tinyURL.com to:
http://tinyurl.com/9fb35
now for wget you could run, say:
google_search=`wget -q -O- -U opera "http://tinyurl.com/9fb35"`
(I'm using bash here, so you'll have to change the assignment syntax for a different shell)
What's important here is the "-U opera" flag to wget. This sets wget to identify as a specific user:agent, and without it, Google search will return a 404. Also, instead of making a tinyURL out of that whole URL, I could have used:
http://www.google.com/search?q=4th+amendment
but I like using tinyURLs, and with a long-ish seach query it becomes useful very quickly.
Now, with curl you could use:
google_search=`curl -s -A opera "http://www.google.com/search?q=4th+amendment"`
The user:agent flag is important when using curl with the Google search also, since you'll get a 404 (albeit a strange one) if you don't. Also, curl doesn't seem to want to play well with the tinyURL redirection.
What you could do instead of use the tinyURL is set the URL to a variable, like;
searchurl="http://www.google.com/search?q=4th+amendment"
then you could simply run the command:
google_search=`curl -s -A opera "${searchurl}`
Hope this helps!
Powerbook G4   Mac OS X (10.4.4)   1.25 GHz, 1GB RAM