How do I search a spreadsheet for a list of URL's | then search those URL's for keywords | save the output? (Oh yeah..., and schedule it)

Fist, I should mention I am not a programmer but am eagerly learning powershell!
I am looking for an automated solution to accomplish what I am currently doing manually.  I need a script that would combine the following:
Reach out to a list of websites (probably a loop of some sort since the list will come out of a spreadsheet which could contain 1 or 100 different sites)
Search each page for a specific word or words (not contained in the spreadsheet though that may make it more scalable)
Save the URL of the site(s) that contained the keywords to one text file (versus the multiple .html files I am creating today)
Have the output contain which words it found on which site.
If not overly complicated, I would like to schedule this to recur once a week.
A working script would be ideal, but even the resources that show me how to incorporate each element would suffice.
I have had success pulling down the full content of the listed pages and saving them to a directory, which requires manual intervention.
So far this works, but it's not scalable:
     Set-ExecutionPolicy RemoteSigned
     $web = New-Object Net.WebClient
     $web.DownloadString("http://sosomesite/54321.com") | Out-File "C:\savestuffhere\54321.html"
     $web.DownloadString("http://sosomesite/54321.com") | Out-File "C:\savestuffhere\65432.html"
     Get-ChildItem -Path "C:\savestuffhere\" -Include *.html -Recurse | Select-String -Pattern "Keyword 1"
In otherwords, I have to manually replace the "http://sosomesite/54321.com" and "C:\savestuffhere\54321.html" when the URL changes to .\65432.com and the output name to match.  That works fine when it's a couple sites, but again,
is not scalable.  
Then, to see if any of the saved file's contain the keyword(s), I have to search the directory for the keyword which I am using:
Get-ChildItem -Path "C:\savestuffhere\54321.html" -Include *.html -Recurse | Select-String -Pattern "Keyword 1"

Hi Sure-man,
Sorry for the delay reply.
To automatically Reach out to all urls, you can list all urls in a txt file "d:\urls.txt" like this:
http://sosomesite/54321.com
http://sosomesite/65432.com
Then please try the script below to save the URL of the site(s) that contained the keywords to one text file "d:\outputurls.txt":
$urls = get-content d:\urls.txt
foreach($url in $urls){
$results = $web.DownloadString("$url")
$matches = $results | Select-String -Pattern "keyword1","keyword2"
#Extract the text of the messages, which are contained in segments that look like keyword1 or keyword2.
if ($matches.Matches){
$Object = New-Object PSObject
$Object | add-member Noteproperty keyword $matches.Matches.value
$Object | add-member Noteproperty URL $url
$output+=$Object}
$output|Out-File d:\outputurls.txt
If you want to schduled this script in Task Scheduler once a week, please save the script above as .ps1 file, and follow this article:
Weekend Scripter: Use the Windows Task Scheduler to Run a Windows PowerShell Script
If I have any misunderstanding, please let me know.
I hope this helps.

Similar Messages

  • HT1904 How can I get a refund for apps in the apple App Store that do not work and are not what they said they wer

    How can I get a refund for apps in the apple App Store that do not work and are not what they said they wer

    You've tried deleting and redownloading them and seeing if they then work and tried contacting the developers ? If you have and they can't/won't help then try the 'report a problem' page to contact iTunes Support : http://reportaproblem.apple.com
    If the 'report a problem' link doesn't work then you can try contacting iTunes support via this page : http://www.apple.com/support/itunes/contact/- click on Contact iTunes Store Support on the right-hand side of the page, then Purchases, Billing & Redemption

  • How can I create a form for users wherein the text field will expand to accommodate additional text?

    How can I create a form for users wherein the text field will expand to accommodate additional text?

    You need to use LiveCycle (PC Only) to create a dynamic form like that.
    The best you can do with Acrobat to view all of the text in a field is to set the field to multiline, and set the size to "Auto" (If you don't set the size to 'Auto', you can enter as much text as you wish, but the user will need to use the scrollbar to view all of the text.)

  • HT4191 iPhone Local Storage "My iPhone" - How do you create this folder for use by the Notes app on a iPhone or iPad?  If I want to keep some notes only on my device and not in a cloud environment associated with an e-mail account.

    iPhone Local Storage "My iPhone" - How do you create this folder for use by the Notes app on a iPhone or iPad?  If I want to keep some notes only on my device and not in a cloud environment associated with an e-mail account.  I've seen reference to the  "My iPhone" local storage put no mention on how you create this folder or access this folder within the Notes app.  I realize storing information in a local storage like this provides no syncing between other iDevices but that is exactly what I'm looking for.  I'm running iOS7.0.4 on a iPhone 5S, and a iPad Air.  Any help would be greatly appreciated.

    If you go to Settings > Notes > Default Account you will see "On My iPhone" as the default account and the only choice if you have not enabled syncing Notes in Settings >iCloud or Settings > Mail, Contacts, Calendars. If you have enabled syncing you can still select "On My iPhone" as the default account. When you are in the Notes app you won't see any accounts listed if you have not enabled syncing because they are all in the On My iPhone account and that is the only place possible. It is not a folder that you create.

  • I purchased a song through iTunes on my MacBook which I have owned for years using the same Apple ID and all of the sudden I am not authorized to listen to this song only? Why? How do I fix it cause it's making me very mad...

    Just purchased a song through iTunes on my MacBook which I have owned for years using the same Apple ID and all of the sudden I am not authorized to listen to this song only? Why? How do I fix it cause it's making me very mad...

    Hi E_WM7,
    Welcome to Apple Support Communities!
    Take a look at the article linked below, it provides suggestions that may be able to resolve the authorization issue.
    iTunes repeatedly prompts to authorize computer to play iTunes Store purchases
    http://support.apple.com/kb/ts1389
    -Jason

  • How do I buy CS6 Standard for Mac from the Adobe website?

    How do I buy CS6 Standard for Mac from the Adobe website? There doesn't appear to be an option for Mac although all the FAQs insist the suite will work on a Mac. Bought a Windows version by mistake and had it refunded, the Adobe help dude said you had to add something to the basket during checkout but can't find the way into that either. Is it just me or is the system completely rubbish?...

    You need to contact Adobe Support either by chat or via phone when you have serial number and activation issues.
    Here is a link to a page with options to help make contact:
    http://www.adobe.com/support/download-install/supportinfo/

  • How do I get a refund for an app I purchased in error and never actually installed? downloaded fully

    How do I get a refund for an app I purchased in error and never actually installed?

    How to report an issue with Your iTunes Store purchase - http://support.apple.com/kb/HT1933 - working through iTunes application and iTunes Store interface.
    iTunes Customer Service Contact - http://www.apple.com/support/itunes/contact.html

  • How to save the output of a concurrent proram automatically

    Hi Friends,
    We have a requirement to save the output of a concurrent program to a specific folder for a reporting purpose. whenever we run that concurrent program it should save the out file in the network folder \\out_files\con_prg like that
    instead of storing it in its default folder in unix.(or I should recoganize which out file is for my concurrent program from the default folder and to move it to windows shared network folder). If it is one time , I can do it manually. But I need to schedule that concurrent program to run every day and transfer the output to the windows folder.
    Our system is Oracle Apps 11i , Data base 9i in Solaris.
    If you have any idea or sugesstion for this , please share it with me here.
    Thanks in advance,
    Vimal...

    Hi
    Thanks for the reply. I can get the name from the fnd_concurrent_requests table, But how can I identify which is the one submitted by me. Because there is possibility of multiple submission by the users with different parameters.
    I can give you more details.
    We are having a BI report (not oracle) for AR Aging Report details. It has a staging table to get the data from Oracle EBS.
    My job is to run Our Customized AR Aging Report with 2 different parameters like sysdate and the same date for last month.
    Get the output file and processs it (for this we are using VB Script) to populate the staging table from where the BI report get the data.
    So far I am doing this manually for more than a month.
    The Users wants to have all the above mentioned process to be combined as a single concurrent program , so that they can submit easily and populate the data any time of the day like other Oracle Standard reports.
    The BI report can be viewed any time of the day by more than 15 users and also AR Aging report can be run so many times in a day by another set of users.
    In this situation how can I get the exact output file of the request submitted by me.
    Thanks,
    Vimal...

  • Just connected apple tv. Everything works perfectly but the screen is not in color. I have tried reversing the HDMI cable and changing the output in settings and restarting. Any ideas of how to fix this?

    Just connected apple tv. Everything works perfectly but the screen is not in color. I have tried reversing the HDMI cable and changing the output in settings and restarting. The screen is still black and white. Any ideas of how to fix this?

    Hi im,
    Thanks for using Apple Support Communities.  This article has a few troubleshooting ideas for issues with color on your TV when using Apple TV:
    Apple TV (2nd and 3rd generation): Troubleshooting video issues
    http://support.apple.com/kb/TS5148
    Cheers,
    - Ari

  • Find the SAP Script name for the Output type RD00 and Appli V3

    Hi Gurus,
    My requirement was to copy the Std SAP Script for the output type RD00 and application name V3 and to do some modification.
    Please any one suggest me how to find the Std SAP script based on the above Output type and Appl ?
    Regards
    paul

    Hi,
    The script Name is LB_BIL_INVOICE
    You can find byNACE>V3>Output Types>RD00-->Processing Routines
    In thje output type Press Change button and select New entries.
    Regards
    Sandipan
    Edited by: Sandipan Ghosh on Mar 31, 2008 12:06 PM

  • How to write code to print and save the output in my GUI?

    I had been searching on code to program print and save commands to print and save the output from the GUI but to no avail. Can someone help me?

    The output will be link from the previous GUI page. Hence the output is a page with Jtable and a button for print to print the information in the JTable.

  • I use to administrate my DSL modem via an ip-address. When I enter it into FF8 I am asked where to save the file. Why and how can I prevent FF8 from doing that?

    I use to administrate my DSL modem via an ip-address. When I enter it into FF8 I am asked where to save the file. Why and how can I prevent FF8 from doing that?
    And now anytime I am entering an ip-address I am asked if I want to download the file.

    It happens when the modem server doesn't send the file as text/html, but with another MIME type.<br />
    I tried the index.html addition in case the server might send that file as text/html.<br />
    If your DSL modem has a support website then you can try to ask there for advice about how to configure the modem server.

  • How can I sync my new iPad with my itunes account on my 4 yr old macbook? I plugged the iPad in once and all music synced. The 2nd time it said update needed. Mac says no update available.

    How can I sync my new iPad with my itunes account on my 4 yr old macbook? I plugged the iPad in once and all music synced. The 2nd time it said update needed. Macbook says no update available.

    " how can I sync my iphone to my mac book without losing all the music and pictures on it?"
    You cannot.  Copy everything you want form the old computer to the new one, then sync it.
    it basically says you have to erase your iphone to sync to a new itunes!
    Correct Iphone will sync, or manually manage, with one and only one computer/library at a time. Syncing, or manually managing, to another will indeed erase the current content.

  • How to save the output of sap script to pdf document in sap

    hi abapers
    how to save the output of sap script in sap so that can retrieve the saved document later.
    i have to save the rcia output from sap script in pdf document in sap so that it can be retrieved later
    how to use dms

    Hi deepika,
    This thread will solve ur problem OTF  -> PDF
    Regards,
    Pravin

  • How to save the output of sap script in sap so that can be retrieved later

    hi abapers
    how to save the output of sap script in sap so that can retrieve the saved document later.
    i have to save the rcia output from sap script in pdf document in sap so that it can be retrieved later
    how to use dms
    <MOVED BY MODERATOR TO THE CORRECT FORUM>
    Edited by: Alvaro Tejada Galindo on Dec 26, 2008 11:01 AM

    Hi Deepika,
    Look into this link https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/401cca81-b861-2910-ada2-f416dfb3b3fe
    Regards,
    Pravin

Maybe you are looking for

  • 500 Internal Server Error - Working with Web Dynpro

    Hi, I need some help regarding "Web Dynpro's" When i try and and run the test from :- Support Desk- Web Dynpro Test Tools- Portal Navigation, the following error occurs:- 500 Internal Server Error While processing the current request, an exception oc

  • TS1702 Why does iPad not support adobe flash player

    Why does the Ipad not support Adobe Flash Player ?

  • Timeout during long JCO runtimes

    Hi all, I have a webdynpro application that connects to backend SAP systems. If the R/3 functional module or BAPI runs for a long time, my webdynpro screen goes blank after 60-70 seconds. Where can I set the timeout setting to be greater? Is this a J

  • Encrypting a file using PGP

    Hi All, In my scenario i need to pick up a file,encrypt it using PGP and then send it destination folder. As i am new to XI ,I don't have any idea how to encrypt the file using PGP.it would be helpful for me if anyone could help me. Regards, Hemanth.

  • HT5312 What if I forgot my security questions

    How can I change my security questions? I forgot the answers to the ones I had so now I can't but anything on the app store