[Bash] Wget doesn't download the right page

Hi, I'm writting a little bash script under Ubuntu, but I'll use it under Arch ^^
I got a problem, and I don't know how can I fix it.
$ wget -O boulet.html http://twitter.com/justinbieber
--2010-12-22 12:05:53-- http://twitter.com/justinbieber
Resolving twitter.com... 128.242.240.212, 168.143.162.116, 128.242.245.180
Connecting to twitter.com|128.242.240.212|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 52909 (52K) [text/html]
Saving to: `boulet.html'
100%[==============================================================================================================================>] 52,909 57.1K/s in 0.9s
2010-12-22 12:05:54 (57.1 KB/s) - `boulet.html' saved [52909/52909]
Then I open boulet.html (local disk) and the online twitter page:
Why does the two pages aren't the same ? boulet.html was downloaded one minut ago, and it lacks a lot of tweets... ?
And less important, why does the downloaded page (boulet.html) is in French while I have my OS in English ?

Lowra wrote:Why does the two pages aren't the same ? boulet.html was downloaded one minut ago, and it lacks a lot of tweets... ?
And less important, why does the downloaded page (boulet.html) is in French while I have my OS in English ?
Content negotiation. Your browser sends in each http request some information the server can use to tailor the response to each request.
For example, you probably configured your browser to prefer the English language, so in each request it adds the header
Accept-Language: en
The twitter server reads that information and sends you the English page.
I assume twitter uses some sort of geolocation which resolves your ip to a French speaking country (Quebec?), and thus, when you omit Accept-Language (like when you use a raw tool like wget), you are sent the localized page.
And lastly, it seems to me localized pages lag behind the original ones.
So, you should try
wget --header="Accept-Language: en" -O boulet.html http://twitter.com/justinbieber
and for the love of god, Justin Bieber???

Similar Messages

  • Hi I'm running Addressbook and cannot clear previous entry easily when searching my data base of around 5,000 contacts.    I prefer to view in All contacts on a double page spread with details on the right page.  Searching doesn't seem to work correctly i

    Hi I'm running Addressbook and cannot clear previous entry easily when searching my data base of around 5,000 contacts. 
    I prefer to view in All contacts on a double page spread with details on the right page.  Searching doesn't seem to work correctly in this view.
    It's always the second search that is problematic.
    I've tried typing over and all it seems to do is confine the search to the the entries that have come up for the previous search.
    I've tried to use the x to clear the previous entry and then type the next search, same problem.  The only way seems to be to move from "All Contacts" to "Groups".  Then the searched name appears and I can return to All Contacts to see full details.
    Surely three key press' are not the way it's supposed to work?
    FYI
    Processor  2.7 GHz Intel Core i7
    Memory  8 GB 1333 MHz DDR3
    Graphics  Intel HD Graphics 3000 512 MB
    Software  Mac OS X Lion 10.7.3 (11D50d)
    Address book Version 6.1 (1083)
    MacBook Pro, Mac OS X (10.7.1), 8Mb RAM 2.7Ghz i7

    AddressBook experts are here:
    https://discussions.apple.com/community/mac_os/mac_os_x_v10.7_lion#/?tagSet=1386

  • Browser doesn't display the right web page

    Hey everyone,
    I have this problem with a web page that doesn't display
    right. What happen was my client has a news web page that is being
    updated once every week. The way I update it is changing the
    content on Dreamweaver and copy the codes and pasting them on the
    server interface, and resave them. Thus, over riding the old news
    page.
    Everything display right on my computer. The only thing is I
    have to refresh my browser in order for the right page to show up.
    Once it did, then it stays as that.
    The thing is, my client still sees the old news page even
    after I've replaced it with the new codes.
    I've done this for him for over 3 months, and this problem
    has never occur before.
    I don't understand why it's like this. If this is happening
    to him, wouldn't it be happening to other people's computer as
    well. If it is, then it's bad news, cause no one would like to
    always have to refresh their browser for the right news page to
    show.
    Could anyone explain why this is happening? And how would it
    affect other users? Or is this even possible.
    Thanks in advance, everyone.
    Jen

    It sounds like a cache issue. When you view the page, that
    page is loaded into your (or your client's) browser's cache memory.
    This is done so that when you return to that page it will load
    faster (since it is already loaded into memory).
    This will only happen if a person has been to the page
    recently. There is a limit on the amount of information your
    browser will store in cache memory. The more you surf, the more
    likely it is that the page will be replaced in the cache memory
    with some other information.
    You (and your client) can, depending on what browser you are
    using, set your preferences so that all web pages will be
    completely reloaded every time you visit them. In IE 6 for Win:
    Tools>Internet Options>Settings> Select 'Every visit to
    the page'. You can also set your cache to be the minimum amount of
    space alotted. Note: This doesn't really work very well in IE 6 for
    WIN. I use firefox for reviewing changes because I have found IE 6
    is completely unreliable for viewing the latest.
    Most other browsers will allow you to hold down Shift and hit
    'refresh' to throw out the cache for that page and view the latest.
    As far as I know this is standard operating procedure for proofing
    new pages.
    If there is a coding technique to force a browser to use only
    the latest information, I don't know what it is.
    Good luck.

  • Some browser doesn't display the MW page correctly

    after installing Maintenance Wizard 2.17, some browser doesn't display the MW page correctly, some menu function is missing. The display problem is not for everyone, some user's browser have no problem. Some user's IE are displaying it correctly. i.e. one user has both IE and firefox working, however, after she clear her java cache, both browser refuses to display the MW page correctly.
    The resolution of PCs is set to the same value. however, the size of the text isn't. after changing the size of the text, some of our pc start to work. The display web page for MW doesn't show the right most menu item. After resize the screen, we find it is because of the blue frame for the MW page that stop to dynamical stretch up to 20 inches.
    The menu item is been streched out of the blue frame, they are no longer displayable
    any ideas?

    We have the same problem.
    And find it is caused by a combination of your screen resolution, font size and browser window size.
    The blue frame that displays the Oracle Mainentance Wizard banner doesnt dynamical strech to the full window size.
    After experiment with it, MW can display much better in our case.

  • Preview doesn't remember the last page opened

    I have noticed that Preview doesn't remember the last page of the PDF documents after 30 mins of usage (or opening up several PDF files). Is there any way to make Preview always remember the last page of files irrespective of time the file was last used and number of PDF files opened? I think in SL-ML era this was probably the case but since Mavericks it seems to have changed.

    Deleting "browser.download.lastDir" makes firefox unusable.
    It's very annoying. I'm sick of the update to 3.6. It's getting worse with each minor update.
    Hour after hour researching about this odd behavior sucks!

  • I downloaded the recent Pages update but now I can not open some documents done last year.  I get an error saying that I need to download a newer version of Pages.  I am running version 4.3

    I downloaded the recent Pages update but now I can not open some documents done in the past year.  I get an error saying that I need to download a newer version of Pages.  I am running version 4.3.  Do I have to purchase an update of Pages?

    You are not on Mountain Lion anymore, right? You are on Mavericks and have downloaded Pages 5 and then created docuemnts in it which can't be opened in Pages 09.
    No, you don't have to buy a newer Pages but you have to be particular about which version you use.

  • I have downloaded the latest pages  update on my iPad, but my iMac still has version 4.3 and I cannot find out how to update it only desktop. When I try to send something from my iPad pages to via email to be opened on my desktop in Pages, it won't open P

    I have downloaded the latest pages update on my iPad, but my iMac still has Pages version 4.3, and I cannot find out how to update it on my desktop. When I try to send something from my iPad pages via email to be opened on my desktop in Pages, it won't open in Pages. I get a message that it can't be opened because the version is too old. I sent the document as a PDF and it worked. But I want to be able to use Pages back and forth. HOw do I update the Pages version on my desktop iMac?

    The file format used by the iOS versions of the iWork apps can only be read by the new iWork apps on your Mac - i.e. Keynote 6, Pages 5 & Numbers 3. Those versions for the Mac require Mavericks, OS X 10.9. The "too old" error on a Mac comes if you are trying to open an iWork '08 or earlier file in the newest apps. If you do have the newest apps you can open the files from your iPad.
    If you can't or don't want to upgrade to Mavericks & the newest iWork apps your best option would be to export/share the files from the iPad to a type an older version of iWork can read such as Word, text, Excel, etc.
    Or contact AppleCare. They made this mess, let them fix it.

  • I forgot my ipod 5 passcode. i do all the steps itunes tells my to do. when i get to the right page and it says "restore ipod" i click it. it works for about 11minutes left and then it says like 'error cod erXX011210021" PLEASE HELP ME!!!!!!!!!!!

    my iPod is I 5th generation. I forgot my passcode and typed in the wrong thing too many times, so now on the lock screen it says "connect to iTunes" I did all the steps your supposed to. I unplugged all cord from iPod connecting to my computer, I held the home button and plugged my iPod back in, and then I keep holding the home button until the iTunes sign showed up. I got to the right page and then clicked restore. A small window came up saying " restore and update" I clicked it, I agreed to the terms and agreements and the updating started. The updated finished with =out a problem. I t was when I clicked "restore " that I had a problem. The restoring lasted until I had about 10 or 11 minutes left. that is when I saw a alert of my screen saying "iTunes could not finish restoring,. an unknown error occurred. error code: 0xE8000065" I tried restoring and updating my iPod about three or four times now! PLEASE, PLEASE, PLEASE HELP ME!!!!!!!!!!!!!!!!!!!!!!!!!!!

    See this support document, iTunes: Resolve USB-related alerts - Apple Support or this one regarding the same error and Windows. iOS: Unknown error containing '0xE' when connecting to a Windows PC - Apple Support

  • New tabs vanish when I view another one, MOST URL's vanish after a second and that is a safety issue if you can see if you are on the right page!

    A while ( couple weeks ) before the last update and change I noticed my URL's were vanishing a second after I arrived on a page. Not all URL's but most. This is a safety issue if I can't see I'm on the right page and have not been diverted.
    ALSO, I can't copy them to send a link via email if they aren't there! I found a way in the old FF click down list, but even that feature is gone on this new version! This needs fixed at once.
    AND when I make a new tab it vanishes! Or worse does not display like a tab. I can't go back and forth between pages if the tab vanishes. This started with the last FF update and change. This is poor design!!! When I updated the restore previous session grayed out. All my tabs were gone and I don't have them all in history. I was researching some things and it was important, now I can't remember them all. This is not a good thing!
    Unless you pin all your tabs immediately they will vanish??????? Stupid design. It does not even show the new tab for THIS page!!! "but is does show your URL....hmmmmm"
    What is the reason for having an arrow go through open tabs if they vanish???
    PLEASE FIX THIS or I'll have to change browsers, I'll have no choice. How can you do anything if FF erases your URL's or tabs!

    Hey Fox1User,
    Thank you for your message. I believe that this will happen when you have a dark theme applied to the new version of Firefox. It has been resolved in the 29.0.1 update that went out yesterday and would love to hear if this has improved the issue for you. I understand that the tabs are disappearing and it sounds like they may be blending into the tab bar.
    However! if this is not the case, can you please show a screenshot of the tabs [[How do I create a screenshot of my problem?]] after trying to produce this in Safe Mode [[Troubleshoot Firefox issues using Safe Mode]].
    I hope we find the culprit, looking forward to your reply.
    I also noticed you have 4 versions of Flash installed.

  • I duplicated a document created originally in new pages 5.1 and exported that copy to the older pages '09. I see the document listed in the old pages but my computer refuses to open it demanding I download the newer pages app... which I have. Now what?

    I am able to open the document through finder, but I can't with Pages '09.

    I have the latest operating system on my '09 macbook pro. My old pages is pages '09.
    When this document was created I didn't realize I did it under Pages 5.1, which I now realize is limited in capabilities so it can speak to iOS devices through iCloud.
    I duplicated that file, renamed it "Yardening copy" and exported to Pages '09. I see the file in the pages listing, but when I try to open it there I get a warning that I need to download the new Pages 5.1... which I have on my computer. I even tried to email it to myself hoping that might work on a lark, but the file is too larg being that it includes pictures.
    I can find the copy via finder, and yes, double click it and it opens... but I'm assuming it is using Pages 5.1 to do so.
    Do I really have to copy and past all the info on this document onto a new Pages '09 document where I want to maintain my  master library of documents?

  • We purchased an Acrobat 9 Professional and no longer have the installation media. Where we can download the right software according with the license we have?

    We purchased an Acrobat 9 Professional and no longer have the installation media. Where we can download the right software according with the license we have? We want to install it on Windows 7 64 bits

    http://helpx.adobe.com/acrobat/kb/acrobat-downloads.html. Be sure to make a backup of this download to a CD or backup HD for the future. You will still need your S/N.

  • Help- Flex Start Page doesn't open the Start Page

    Help -> Flex Start Page doesn't open the Start Page.
    I see multiple bug entries regarding this resolved, but no description of what I can do to fix it.
    I'm running Flex Builder Professional on Windows XP

    I've heard this can be caused in IE when the pop-up blocker is enabled. Disable it so pop-ups are allowed, and the Flex Start Page may load.
    If this post answers your question or helps, please mark it as such.

  • TS1538 I'm on the right page for support, but nothing works.

    I'm on the right page for support, but the description given is underwhelming to be polite. I've been doing what is written for 2 hours and nothing has happened as if I've done nothing. I don't have time for this to not be working.

    See
    iOS: Device not recognized in iTunes for Windows
    - I would start with
    Removing and Reinstalling iTunes, QuickTime, and other software components for Windows XP
    or              
    Removing and reinstalling iTunes and other software components for Windows Vista, Windows 7, or Windows 8
    However, after your remove the Apple software components also remove the iCloud Control Panel via Windows Programs and Features app in the Window Control Panel. Then reinstall all the Apple software components
    - New cable and different USB port
    - Run this and see if the results help with determine the cause
    iTunes for Windows: Device Sync Tests
    Also see:
    iPod not recognised by windows iTunes
    Troubleshooting issues with iTunes for Windows updates
    - Try on another computer to help determine if computer or iPod problem

  • Custom abap iview need to double click to get in to the right page

    Hi experts,
    I create custom Abap Webdynpro Iview,
    but it has to double click to get into the service.
    If we only do one click then it got internet error.
    how to solve this?
    thanks in advance

    Hi,
    sorry for lack  of information.
    the error is "the webpage cannot be found"
    http 404. (This error (HTTP 404 Not Found) means that Internet Explorer was able to connect to the website, but the page you wanted was not found. It's possible that the webpage is temporarily unavailable. Alternatively, the website might have changed or removed the webpage)
    but, when I click the link page again (for second time), it will link to the right page.
    is there any missing configuration?
    thanks

  • My dad can't remember his iCloud password, and doesn't have the right answers to his security questions. Any other ways to reset a password?

    My dad can't remember his iCloud password, and doesn't have the right answers to his security questions. Any other ways to reset a password?

    Open > http://iforgot.apple.com and choose Email authentication

Maybe you are looking for