Get content of web page in java. Result - bad chars.

My common task is - get content of page in java. And parse data on this page.
When i am opening page in browser - i have a good look of all data. But when i am using java code for getting page content i have a bad content with bad chars.
How to do it right?
Thanx in advance!
public static void main(String[] args) {
             String url_get_page = "http://www.pai.pt/search.ds?activeSort=name+-maindocflag|asc&distSort=false&encodedRefinement=namechar1..%3d..^A%24..%26..A&what=Advogados&startingPageNumber=1&stageName=What+search&originalOffset=1&expandWWWSearch=false&myplaces=false&distance=50&searchType=www&phoneNumberSearch=false&advancedSearch=true&alphaRefineable=AN4683|BN317|CN2373|DN696|EN704|FN1549|GN456|HN539|IN670|JN3704|KN8|LN1433|MN4143|NN606|ON318|PN1727|QN9|RN1383|SN1522|TN484|UN14|VN677|WN16|XN8|YN4|ZN23|[0-9]N0&excludeZone=false&restoSearch=false&firstMaxRank=43522&previousPath=search";
             StringBuffer result = new StringBuffer();
             URL url;
          try {
                  url = new URL(url_get_page);               
                     HttpURLConnection connection = null;
                         connection = (HttpURLConnection) url.openConnection();                       
                 connection.setRequestMethod("GET");
                 connection.setDoOutput(true);
                    connection.setReadTimeout(10000);           
                    connection.setRequestProperty("Host", "www.pai.pt");
                    connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2) Gecko/20100115 Firefox/3.6");
                connection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
                connection.setRequestProperty("Accept-Language", "ru,en-us;q=0.7,en;q=0.3");
                connection.setRequestProperty("Accept-Encoding", "gzip,deflate");
                connection.setRequestProperty("Accept-Charset", "windows-1251,utf-8;q=0.7,*;q=0.7");
                connection.setRequestProperty("Keep-Alive", "115");
                connection.setRequestProperty("Connection", "keep-alive");
                connection.setRequestProperty("Referer", "http://www.pai.pt/search.ds");
                 connection.setRequestProperty("Cookie", "MfPers=12678646695048a98819027298bf50127329f8c315e8f; vuid=8a98819027298bf50127329f8c315e8f; ptkn=40EAFA18-5758-F374-F570-A0480F306222; WT_FPC=id=174.142.104.57-1456441520.30063880:lv=1267888167073:ss=1267888167073; __utma=76091412.2059393411.1267864686.1267878351.1267891770.4; __utmz=76091412.1267864686.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); BFHost=wd-web04.osl.basefarm.net; JSESSIONID=20C8FD4414F50F3AE361C487D0E3C719; MfTrack=12678917654148a98819027298bf50127329f8c315e8f; BIGipServerwd-web-pt=285284362.20480.0000; __utmb=76091412.1.10.1267891770; __utmc=76091412");           
                 connection.connect();
                 BufferedReader rd = new BufferedReader(new InputStreamReader(connection.getInputStream(),"UTF8"));
                 String line;
                 while ((line = rd.readLine()) != null) {
                     result.append(line).append("\n");
                 connection.disconnect();
               } catch (MalformedURLException e) {
                    e.printStackTrace();
               } catch (IOException e) {
                    e.printStackTrace();
            System.out.println(result.toString());            
     }

Now the code is
public static void main(String[] args) {
             String url_get_page = "http://www.pai.pt/search.ds?activeSort=name+-maindocflag|asc&distSort=false&encodedRefinement=namechar1..%3d..^A%24..%26..A&what=Advogados&startingPageNumber=1&stageName=What+search&originalOffset=1&expandWWWSearch=false&myplaces=false&distance=50&searchType=www&phoneNumberSearch=false&advancedSearch=true&alphaRefineable=AN4683|BN317|CN2373|DN696|EN704|FN1549|GN456|HN539|IN670|JN3704|KN8|LN1433|MN4143|NN606|ON318|PN1727|QN9|RN1383|SN1522|TN484|UN14|VN677|WN16|XN8|YN4|ZN23|[0-9]N0&excludeZone=false&restoSearch=false&firstMaxRank=43522&previousPath=search";
             StringBuffer result = new StringBuffer();
             URL url;
          try {
                  url = new URL(url_get_page);               
                 HttpURLConnection connection = null;
                connection = (HttpURLConnection) url.openConnection();                       
                 connection.setRequestMethod("GET");
                 connection.setDoOutput(true);
                connection.setReadTimeout(10000);           
                connection.setRequestProperty("Host", "www.pai.pt");
                connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2) Gecko/20100115 Firefox/3.6");
                connection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
                connection.setRequestProperty("Accept-Language", "ru,en-us;q=0.7,en;q=0.3");
                connection.setRequestProperty("Accept-Encoding", "gzip,deflate");
                //connection.setRequestProperty("Accept-Charset", "windows-1251,utf-8;q=0.7,*;q=0.7");
                connection.setRequestProperty("Keep-Alive", "115");
                connection.setRequestProperty("Connection", "keep-alive");
                connection.setRequestProperty("Referer", "http://www.pai.pt/search.ds");
                 connection.setRequestProperty("Cookie", "MfPers=12678646695048a98819027298bf50127329f8c315e8f; vuid=8a98819027298bf50127329f8c315e8f; ptkn=40EAFA18-5758-F374-F570-A0480F306222; WT_FPC=id=174.142.104.57-1456441520.30063880:lv=1267888167073:ss=1267888167073; __utma=76091412.2059393411.1267864686.1267878351.1267891770.4; __utmz=76091412.1267864686.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); BFHost=wd-web04.osl.basefarm.net; JSESSIONID=20C8FD4414F50F3AE361C487D0E3C719; MfTrack=12678917654148a98819027298bf50127329f8c315e8f; BIGipServerwd-web-pt=285284362.20480.0000; __utmb=76091412.1.10.1267891770; __utmc=76091412");           
                 connection.connect();
                 BufferedReader rd = new BufferedReader(new InputStreamReader(connection.getInputStream(),"UTF8"));
                 String line;
                 while ((line = rd.readLine()) != null) {
                     result.append(line).append("\n");
                 connection.disconnect();
               } catch (MalformedURLException e) {
                    e.printStackTrace();
               } catch (IOException e) {
                    e.printStackTrace();
            System.out.println(result.toString());            
     }I have checked in FF encoding of page is - UTF8.
The problem still exists.

Similar Messages

  • ABAP to get content of web-page

    Hello everyone! _
    Can somebody help me with subject?
    I want to get a content of web page in Internet, for example strings 'Hello world', and put in data base or on a screen.
    May be you can give me some hints?
    Thanks

    Hello,
    Try with These Fm's
    Name of function module        Short text for function module
    RMPO
    RM_GET_WEBLNR_FROM_PRTNR
    SADA
    GET_WEBDBMSRV_URL_ADA
    SFES
    GET_WEBGUI_HELP_MODE
    SHTM
    GET_VERSION_WEB_OBJECT
    SWRXML
    SWR_GET_WEB_SERVER
    W3_API
    W3_GET_WEBAPP_DEFINITIONX
    W3_API_MINIAPP
    W3_GET_FGROUPS_FOR_WEBAPP
    W3_GET_WEBAPP_DEFINITION
    Regards,
    Vasanth

  • Safari (and other browsers) very slow loading "content rich" web pages

    Hi,
    I'm a very "light" user of my mac, using it basically for surfing the web, uploading pictures and music, etc... and recently any web page that has a lot of content (examples are nfl.com, nba.com, nhl.com, mlb.com, united.com, continental.com) takes a very long time to load (most other web pages load normally). Safari (and Firefox as well) gets to the web page in an instant, but then sits there trying to load content (mostly high res pictures and ads I suppose), which sometimes takes minutes or just never fully downloads the content. I'm connected to the internet via ethernet (high speed cable modem from Time Warner), and they've already come out and verified that everything on their end is fine. I've followed the instructions posted on some of the other similar posts (emptying cache, deleting cookies, disenabling add on's, reseting safari, logging in as another user, etc..), and this has not helped. Since this problem is persistent with both browsers, I'm wondering if this may be an OS issue as opposed to a browser issue. If someone can walk me through a solution (keep in mind that I'm a novice), that would be much appreciated.
    Thanks!
    Message was edited by: Mr. Cady

    HI and welcome....
    Since this problem is persistent with both browsers, I'm wondering if this may be an OS issue as opposed to a browser issue
    Could be.
    Since the problems exist with multiple browsers, login to another user account on your Mac. Try Safari there. If you see the same behavior with Safari there, then follow the instructions here. You will need your install disc. The startup disk may need repairing. Even if Safari is ok in another account, it's still a good idea to check the startup disk. That way you can rule that out.
    Insert your install disk and Restart, holding down the "C" key until grey Apple appears.
    Go to Installer menu and launch Disk Utility.
    (In Mac OS X 10.4 or later, you must select your language first from the installer menu)
    Select your HDD (manufacturer ID) in the left panel.
    Select First Aid in the Main panel.
    (Check S.M.A.R.T Status of HDD at the bottom of right panel. It should say: Verified)
    Click Repair Disk on the bottom right.
    If DU reports disk does not need repairs quit DU and restart.
    If DU reports errors Repair again and again until DU reports disk is repaired.
    When you are finished with DU, from the Menu Bar, select Utilities/Startup Manager.
    Select your startup disk and click Restart
    While you have the Disk Utility window open, look at the bottom of the window. Where you see Capacity and Available. Make sure there is always 15% free space.
    Carolyn

  • HT1338 In my e mails, I cannot get to a web page that is mentioned in the body of the e mail

    In my e mails, I cannot get to a web page that is mentioned in the body of the e mail.

    That's probably a bookmark you have saved in Favorites.
    Tap bookmark icon>Favorites>Edit>Delete the bookmark

  • Dynamic content of web pages sample

    Dear ALL,
    I WANT TO BUILD DYNAMIC CONTENTS OF WEB PAGES/SITE BY
    JDEVELOPER 9I/10G. PLS TELL ME WHERE HAVE BEST SAMPLES/EXAMPLES/TUTORIALS.
    I KNOW OTN HAVE VSM TUTORIALS. BUT NOT ENOUGH.
    BEST REGARDS
    BORIS

    Boris
    BC4J Toy Store is a very good example.
    http://otn.oracle.com/sample_code/products/jdev/bc4jtoystore/bc4jtoystore.zip
    success
    -Jan

  • Summarizing content of web page

    Right now i am doing my final project which is summarizing the content of web page. After summarize, the system can read aloud the summary text.
    Anyone have any idea how to summarize text?
    Can you give me some source code or reference?
    Please... HELP ME...

    You could try using the summary="" attribute of tables in HTML, but that will only work if the site is compliant with web accessibility (Section 508) requirements. Other than that, it's probably not an easy thing to do at all. Apple tried to do this with documents on OS X, but they failed miserably as well.

  • How to embed 3D models in web page using JAVA?

    Hello
    I'm doing a project at University to create a virtual tour of a village. I am creating the buildings in MAYA using polygon planes and cubes and then exporting to VRML.
    I plan to embed each individual model into a separate web page. Then when the user clicks on a location in a 2D map they will be taken to the web page containing that building.
    However, I can't work out how to embed these 3D models into web pages using JAVA. Can anyone help me with this, please?
    Also, I've been told to use JAVA 3D to interact with the project but am unclear as to how JAVA 3D would be used in this project - aqny ideas/suggestions welcome!
    M.

    If you use Java-plugin, you can embed Java3D in a web page.
    Here is the URL(http://www.geovista.psu.edu/products/demos/LorenzAttractor3D/index.html) which used Java3D.
    I hope this helps.
    Masa

  • HT1349 How do I get rid of web pages I'm through viewing?

    How do I get rid of web pages I'm through viewing?

    Not sure exactly what you mean but while you are using Safari, just tap the X on the left side of the tab to close a it. Only the active tab shows the X. You have to tap on a tab to make it active or the foremost window.
    If you want to clear the history - Settings>Safari>Clear History.
    These are instructions for Safari in iOS 5. What version are you running?

  • I was referred to this site because Internet Explorer would not allow me to navigate to web page. Therefore the tech-support staff sugested I try your site to get to the web page

    upon logging into this program open road x3 from the site realeflow.com. I get a msg "navigation to web page has been canceled" so I contacted the tech-support staff. They informed me that in the past this problem has occurred with internet explorer, so they referred your site to me, as a option to get into the web page from their site. I have never used your site to get into a web page before. So I do not exactly know how this works??????????

    In Firefox 3.6.4 and later the default connection settings have been changed to "Use the system proxy settings".
    See "Firefox connection settings" in [[Server not found]]
    You can find the connection setting here: Tools > Options > Advanced : Network : Connection
    If you do not need to use a proxy to connect to internet then select No Proxy
    Another possible cause is security software (firewall) that blocks or restricts Firefox without informing you about that,.
    Remove all rules for Firefox from the permissions list in the firewall and let your firewall ask again for permission to get full unrestricted access to internet for Firefox.
    See [[Server not found]] and [[Firewalls]] and http://kb.mozillazine.org/Firewalls

  • Web pages in java

    i want to make a web page in java.what r the ways to do this?

    u need one one web server like tomcat, and one servlet.jar and u should about to code servlets, enough u can develop a website

  • Content Search Web Part displaying wrong Results for anonymous Users.

    HI Forum Group,
    I am getting Wrong results for my content search web part. The requirement is to show the News Description for the selected news item.
    I have a catalog site which stores News like 
    News1
    News2
    News3
    as Items. and i have connected this catalog in publishing site which is anonymous. In the publishing site created one page "News.aspx"added search results webpart which shows all the news item. Added one page "Description.aspx" to show
    description to show the selected news item.
    When ever user selects any news from news.aspx page it will redirected to description.aspx with the selected item ID
    The "Description.aspx" the search results page gets the data based on the URL by QueryString parameter as shown below
    The problem is, if i multiple items to open in tabs all the items are showing the data same as the first selected item, though the article ID is different.
    Thanks
    Sithender

    Hi,
    Thank you for your feedback on how you were successful in resolving this issue.
    Your solution will benefit many other users, and we really value having you as a Microsoft customer.
    Have a nice day!
    Best Regards,
    Lisa Chen
    TechNet Community Support
    Please remember to mark the replies as answers if they help, and unmark the answers if they provide no help. If you have feedback for TechNet Support, contact
    [email protected]

  • Content Search Web Part Has No Results

    Hello,
    I have been working on a site for a few weeks now and have been trying to make a simple Image Slider. I have found many tutorials and all are helpful -- except I get stuck at the same part every time. I am trying to use a Content Search Web Part in order
    to pull images from a library/list and display them. I go through almost the same instructions every time but my query never shows any results.
    I have went in to 'Change Query' and set the following:
    Select a query: Pictures (System)
    Restrict By App: Specify a URL --> Then changed to mysite/myimagelibrary
    I have tried a few things. I have changed the Result Source to multiple different things, including the default 'Recently Changed Items.' I have also tried to put the images in a list rather than a Picture Library. I have also tried MANY different ways of
    putting in the 'specific URL.'
    I don't know why I can't seem to get results from my query. Any help or suggestions would be appreciated. Thank you.

    Hi,According to your post, my understanding is that you used a Content Search Web part to search images from a library/list , it didn’t display any
    Results.
    Please check which content type you used for these images . Maybe its content type is document rather than picture. Please make sure your images’ content type is picture.
    You can go in to 'Change Query' and set the following:
    Select a query: Items matching a content type(System)
    Restrict by App: Current site
    Restrict by content type: Picture
    As you have mentioned in your description “Specify a URL --> Then changed to mysite/myimagelibrary”.
    Here is a similar issue, you can use as a reference:
    http://social.technet.microsoft.com/forums/sharepoint/en-US/4a683be5-354d-4497-88da-672dd6335358/content-by-search-web-part-shows-no-results
    If the issue still exists, please feel free to let me know.
    Best Regards,
    Lisa Chen

  • OMG! These web pages are java classes! :) A simple question!

    I imported a web page fragment as "header". Now I'm trying to display the header before "page1".
    I tried putting this into the page1.java, but no luck.
    public HtmlForm getHeader() {
    doug2005.header hd = new doug2205.header();
    return hd.getForm1();
    This has to be simple to do. How should I do the method?
    Thanks!

    Thanks! That worked very well. I think I'm getting the idea here. I'm working with something called "JSP" pages which is there own language like "ASP" pages.
    You don't happen to know the current links to these directives and such. My searches tend to bring up outdated material.
    Thanks much!

  • Web page in Java

    So I am brand new to Java but I have developed in C++. I dont have alot of experience building websites or webpages. I am starting an individual graduate project for my degree. It involves building a web page that will be hosted by my teacher. My question to you is, Can i use java to do this? Basically I would use java instead of html, if it is possible. If it is, how do I get started. If not, what do you recommend? I figured since I know C++ already, its time for me to learn Java. What better way to do it than with a project. All advice is appreciated. Thank You.

    Basically I would use java instead of htmlBrowsers only understand html. You need to know html to create web pages. What java can help you do is respond to information that users enter into forms on html pages. You can also use java to create an html page to send back to the browser as a response, which the browser will display to the user.
    An html page (HelloWorld.html) is typically a static web page to display a document with no dynamic
    abilty such as submitting a textfield (that a user entered information into such as a password) to the
    server (although you can probably force it to be). Html pages frequently contain forms which users fill out and then submit. Html pages that contain forms were around long before java servlets + jsp's were invented. If you want the user to enter some info, then you have to send an html page with a form on it to the browser. How you create the html page is up to you. It can be a static file on the server which your java code just relays to the browser, or you can use servlets + jsp to retrieve some data from a database and then create an html page on the fly that contains the data, and then send that "dynamically created" html page back to the browser. But no matter how the page is created, the browser receives an html page, which the browser then displays. And the html page will look exatcly the same to the browser no matter how the page was created.
    I am starting an individual graduate project for my degree. It involves building a web page that
    will be hosted by my teacher. My question to you is, Can i use java to do this?Yes, but you are a thousand miles away from being able to do that in java. First, you have to learn the java language, just like you learned C++. You could read a short beginners book on java to get a cursory grasp of the basics. But after you learn some basic java, then you have to learn about a software application written in java called "servlets + jsp". The application "servlets + jsp" gives you the means to do web programming. You will speak to that application using the java programming language. But in order to learn how the application "servlets + jsp" works, you will have to read another book, and servlets + jsp is not an easy application to figure out. You might need a couple of months of studying to achieve your goal, which is a big time comittment for a student, so that is pretty unrealistic. You might be able to read a bunch of tutorials and throw something together, but you won't have any idea what's going on, and you'll probably need a lot of help.
    If you want to get your project done in a reasonable amount of time, you should probably learn php. A beginning php book will teach you the basics of html and forms, as well as teach you how to use databases to store information that users submit, e.g. login names and passwords. In my experience, some php books get you up to speed very quickly in 3 or 4 chapters and teach you to do some neat stuff immediately.
    Another thing you need to consider is: what type of software does your "host" have? If your host doesn't have Tomcat installed, then you can't use servlets + jsp. If you decide to use php as your "server side web programming language", then your host needs to have php installed.

  • Get parameters from html page from java application standalone ...

    Hi all,
    I work in one solution that i have values in Html Page and i want get the parameters values from html and cath they in java application standalone.
    The Html page is in same host than de java application.
    I want know if this is possible. I wnat know if without HttpServlet i can get the parameters from Html Page pure.
    Thanks in Advance for the ideas,
    Antonio.

    Hi Abdul,
    The problem is my client want one solution where i have one page simple page Html and one application java standalone. This application runs in one machine, but we don't have web server. So the question is: Is possible without web server i can get the parameters values that is inside the html page from java application. I remember you that the application java is one .jar that run's with one command line from crontab "java -jar teste.jar".

Maybe you are looking for

  • Does Migration Asst thwart a truly "clean" install?

    I decided to do a clean install this time when 10.6 was released because one lookinside my preferences folder and libraries found a swarm of unknown, ancient or irrelevant support files. I want these gone. No amount of AppZapper will clear this stuff

  • Unlock Iphone4 (AT&T)

    My Iphone4 was supposed to be eligible for unlocking by AT&T as I've bought it 2 years ago. But I had my phone's screen broken by the end of last year and when I went to an Apple store to fix it, they said they had to replace it. At that time nobody

  • Ipad or ipad mini for web, email, you tube

    Which ipad is better for web, email, you tube & powerpoints?  I know ipad has the retina display but I like the portability of the mini.

  • I would like to setup Apple TV in a business environment on a guest Wi-Fi and cannot get out to iTunes with Apple ID?

    I would like to get Apple TV working in a business environment but when I try to login to iTunes or iCloud using Apple ID no connection is made. I have verified network settings and am getting a DHCP address but is there something in the Access Point

  • Batch not showing at QA32

    Hi, We are facing one problem regarding batch.... Material is maintained in batches, batch entry 1, inspection type 03. At process order stage we are entering the batch no before releasing the order. Batch no is generating and going to the database.