Extracting info from a web page

Hi,
I m not sure if i m asking this question at the right forum.
Can anyone tell me if there is a way to extract data from a web page.
This means, say for example a web site Yahoo displays stock quotes
updated or NASDAQ values almost in real time.
Now if i want to get that information from the web page into one
of my applications ,say, something that uses that data. Is there
a way to do it?
Just curious

Yes, it's possible. You can use the java.net.URL object to connect to websites and download the html. Doing the coding is not that easy, and you should also be mindful of not redistributing data you've gotten from another site without permission

Similar Messages

Extracting text from a web page

I want to extract information about livestock from a certain web page, and add these data to a database.
The web page in question allows me to enter an animal's ID number, and then it displays various data about that individual animal. The format of the page is always the same, but the data vary depending on the particular ID/animal.
What I'm hoping for is a way to quickly extract information about the current animal (date of birth, color, breeding status, etc.) from the open page, and add these to my database.
I realize I can "cut and paste", but that is awfully slow for large numbers of data and animals.
Since the data I need are always in the same places on the page, it seems like there must be a way to automate the extraction.
Any ideas? Thanks in advance.
Eric

Here's a very rough, untested way to do it:
(I'm at work and not on a Mac..so..Milage May Vary)
It would be run as script.pl -i input -o output
eg ./script.pl -i myfilea -o myfileb
oh and the input would be a list of valid web page addresses
eg http://www.foo.com
--bEGIN cODE---
#!/usr/bin/perl
use Getopt::Std;
use LPW::Simple;
getopts("i:o:");
#allows for using -i -o cl options
open(IN,"$opt_i")||die "Cant open input";
open(OUT,">$opt_o")||die "Cant open output";
#opens 2 files
while ($line = <IN>){
#READ IN line by line
@Doc=();
#clear the variable Doc
@Doc=LPW::Simple::get($line);
#open the web page and store all the html into Doc
foreach $docline (@Doc){
#go through every line of text in Doc
if ($docline =~ m/:/){
#does the line have a : on it..if so..
$docline =~ s/ / /;
$docline =~ s/
$docline =~ s///;
$docline =~ s/<\/b>//;
#these 4 lines remove or reformat html formatting
print OUT "$docline\n";
#ok its simplistic, but for now just spits out the line it read to OUT
close (IN);
close (OUT);

How can I extract information from a web page

I wanna read some information from a webpage, and then put those information together into a table. Since I only know the url, not the file path, when I use BufferedReader(new FileReader(FilePath)), there is a FileNotFoundException. How can I do that? Thanks a lot.

You can do it in following way....
First you need to create the object of URL by specifying the URL name.
URL myurl = new URL("http://www.xyz.com/index.html"); //specify your URL path here
URLConnection conn = null;
DataInputStream data = null;
String line;
StringBuffer buf = new StringBuffer();
try {
conn = myurl.openConnection();
conn.connect();
data = new DataInputStream(new BufferedInputStream(
conn.getInputStream()));
while ((line = data.readLine()) != null) {
buf.append(line + "\n");
data.close();
catch (IOException e) {
System.out.println("IO Error:" + e.getMessage());
So, at the end, you have the data in your string buffer...you can use it wherever u want.
Hope this helps

How can I use Automator or AppleScript to get text from a web page and paste it in execl?

I don't know how to make scripts or complexed automator workflows... that's why I'm asking.
I'm trying to make a simple app or script to ask me what text to extract from a web page, like name, address and phone number of a web page and paste each one of these data in the righ cell of excel.
I was thinking to promt a request from automator or an applescript to ask me which text to extract from the page or to look throught the HTML of the page to search for specific html tags, from which extracting text and then importing it, or paste it to the specified execl cell. Name in the name cell, address in the address cell and so on.
Can somebody help me to make this script?
If you know an alternative, like a software that already do this or another language to use, please tell.

Try holding down the alt key as you mark the text to be copied. You can then copy columns to table text.

Posting an XML Variable to a BLS Transaction from a web page

I am working in xMII 11.5 with all the latest service packs. I know this is probably a really basic question but I am stumped.
I am trying to pass a multi row and multi column XML data set from a web page into a BLS transaction (actually, two of them) in order to populate the parameter. I want to use the Web Service interface to the transaction. I have tried using parameters on an Xacute Query in an Applet as well with an equal lack of success. I cannot persuade the transaction to see the incoming variable as an XML data type. I have tried encoding and decoding and string to xml conversions and nothing seems to successfully allow the data set to be seen withing the BLS as anything but a string. The String to XML action will not handle the number of columns in the dataset though it seems to work if the data set has only one column. The data set is formatted in the proper "Rowsets/Rowset/Row" format. I have considered writing the data to an XML file on the server (I know I can deal with that) but that is not acceptable in this application.
Can someone share the secret with me?
...Sparks

Parameter value:
r1d1,r1d2,r1d3;r2d1,r2d2,r2d3;r3d1,r3d2,r3d2
Pass thru String List to Xml Parser with delim ";"
<Row>
<Item>r1d1,r1d2,r1d3</Item>
</Row>
<Row>
<Item>r2d1,r2d2,r2d3</Item>
</Row>
<Row>
<Item>r3d1,r3d2,r3d2</Item>
</Row>
Repeat on each row/item and pass thru String List to Xml Parser with delim ","
<Row>
<Item>r1d1</Item>
</Row>
<Row>
<Item>r1d2</Item>
</Row>
<Row>
<Item>r1d3</Item>
</Row>
Of course, your columns aren't flat but they are easy to ref, to get "column 2" for example:
StringListToXml_1.Output{/Rowsets/Rowset/Row[2]/Item}
So now you have rows and columns. Assign your data to your BAPI structured as needed.
We have passed complex XML via the SOAP interface in 11.5, but it involved some "hacks". Basically we passed the sterilized XML via a String Type Parameter, and then unserialized it inside the BLT.
I have been told on this board that there is a solution to passing XML data vie the SOAP interface using ref docs, but i have never personally seen a working example.

How to read text from a web page

I want to read text from a web page. Can any body tell me how to do it.

Ok i tell you detail. visit the site " http://seriouswheels.com/" you will a index from A to Z which are basically car name index i want to read each page get car name and its model and store it in data base. I you can provide me the code i will be very thankful.

How to get the return values from a web page

Hi all :
how to get the return values from a web page ? I mean how pass values betwen webflow and web page ?
thank you very much
Edited by: jingying Sony on Apr 15, 2010 6:15 AM
Edited by: jingying Sony on Apr 15, 2010 6:18 AM

Hi,
What kind of web page do you have? Do you have possibility to for example make RFCs? Then you could trigger events (with parameters that could "return" the values) and the workflow could react to those events. For example your task can have terminating events.
Regards,
Karri

How to download an animated .gif from a web page

how do i download to my mac an animated gif that is on a web page....when i want a picture or text or something like that i just ...apple shift 4 and "take a picture of it" ...but i want the animation..i have never had to do that on this mac.....a windoze person from the site(chat site) said just right click but my wireless mac mouse doesnt do that....i would appreciate any help.....i do know how to down load from a site that has downloadable .gifs...you just click the download sign they have and no problem....but this is just a random animated .gif from a web page ....i hope my ques is clear....thank you

Drag and drop it from the webpage onto your desktop. Note that it won't remain "animated" while you're looking at the file's icon, but it should work if you build it into a new website.
And you can right-click with a Mac mouse - you just have to set it up to do so in System Preferences. (Set the right side of the mouse to be a "secondary click".)
Matt

New Mac, fresh install of Mountain Lion When I click to open a .Pdf from a web page,while in Safar, I get a black window Nothing opens in Preview or in Acrobat No option to download

New Mac, fresh install of Mountain Lion
When I click to open a .Pdf from a web page,while in Safar, I get a black window
Nothing opens in Preview or in Acrobat
No option to download

Open the Finder. From the Finder menu bar click Go > Go to Folder
Type of copy paste the following:
/Library/Internet Plug-Ins
Click Go. If you see this file: AdobePDFViewer.plugin
Drag it to the Trash, empty the Trash.
Quit and relaunch Safari.

How to Open an Oracle Apps Screen from a web page

Hi,
We have requirement for Opening an Oracle Application screen (say sales order form) directly from a web page.
I could get the URL of the required screen, but the URL contains an ICX_TICKET number, which is generated dynamically by Oracle Apps. So I can't use a static URL for this.
Do you know how I can use or generate an ICX_TICKET in runtime? My user will have an active Oracle Application screen opened along with web page. He want to navigate to Oracle Apps screen from Web page. Hopes this makes the requirement more clear.
Thanks for your time,
Aneesh

Hi Helios,
I have identified a function to generate ICX_ticket. By appending this ticket, I am able to open the Oracle Apps screen. Now, is there in implication on the security side, if I go ahead this way?
Function
fnd_gfm.one_time_use_store(icx_sec.GetSessionCookie(CZ_CF_API.ICX_SESSION_TICKET),300,'FORMS_APPLET')
Anyways, I am raising an SR as u suggested.
Thank you,
Aneesh

Silent print a PDF from a web page using Flex. We are targeting Windows and Mac with Arcobat reader installed

We are planning to Silent print a PDF from a web page using Flex. We are targeting Windows and Mac with Arcobat reader installed. I have tried using the AIR appliaction and it worked, But our requirement is NO INSTALL to the user machine for the silent printing. It is just from web page and silent printing to the default printer to the desktop/Laptop. Can anyone share your thoughts and experience with us. It will be very helpful..
For AIR : I tried the thread Re: AIR and PDF showing/silent printing

Hey CodeMonkey & Ross,
Did you either of you ever find a solution? I'm stuck too, it seems I can get remote printing on all these PDFs to work but it just prints a blank page since I've been using Javascript in the browser, not Adobe's (they are Engineering drawings that I do not have permission to edit so I can't just insert code into them but I need to make work instructions). I've been scouring the internet for hours now, it seems that this thread is the only relevant/useful one at this point. No one else was trying to achieve this for similar reasons to mine.
Thanks guys,
Lox

Why does Acrobat suddenly not work on my Mac (won't open pdf files from a web page), yet I can open them from my iPhone? Everything worked fine yesteray.

Hello:
I have a Mac running OS 10.8.5 and I use Acrobat Pro and Acrobat Reader. Today for some reason, any time I try and access a pdf file from a web page, it won't open. I get a blank window. PDF files already on my computer open fine and I can create PDF files from Word docs, but I can't download or view any pdf file from any web page. Strangely, PDF files on web pages open just fine on my iPhone.
I checked that both my copy of Acrobat Pro and Acrobat Reader are up-to-date. They are.
I checked that my copy of Firefox is up-to-date. It is. And nothing changed with Firefox within the last 24 hours.
I am a humble computer end-user. I am baffled as to why this would suddenly not work. I have not changed any settings, etc. Any help or suggestions would be greatly appreciated.

Thank you.
I checked your instructions you sent and as far as I can tell all of my settings, etc for Firefox (plugin updates and preferences) are correct but I am having the same problem.
However, everything works in Safari. I don't have time to attempt to diagnose why Firefox no longer works. I will just switch to Safari.
Many thanks.
Charles
Charles Deister
(503) 949-5762
[email protected]<applewebdata://81CB4171-226F-49DF-BD59-A38A7360B3FB/[email protected]>
PO Box 5032
Salem, OR 97304
http://www.pilotstrat.com<http://www.pilotstrat.com/>
This transmission (including any attachments) may contain confidential information, privileged material, or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

I did a COPY of some text from a web page, and then did a PASTE into notepad.exe (Windows). The text from each line was duplicated -- on the line! Instead of "Fred", it became "Fred Fred".

I just recently installed Firefox for the first time. It seems nice and quick. The version is reported as: "10.0.1".
I wanted to save some text from a web page, so navigated to that page, selected the text, and pressed the Control-C combination to COPY the selected text to the buffer. For example, the text I selected looked something like this:
Harry
Ron
Hermione
Hagrid
Albus
NOTE: Each line of text has a small icon to the left of the text.
It is not reasonable to COPY and PASTE each line individually, as there can be hundreds of lines of data. I recall, however, that
doing a COPY and PASTE on this data into Microsoft's Excel will produce cells which have the icons included in the cell, but unfortunately one cannot can't get rid of them! At least I've never found a way to remove them, but that's another issue. :)
Once I'd done the COPY operation I switched to a Notepad window and did a PASTE operation. To my surprise, the text from each line was duplicated. It looked like this:
Harry Harry
Ron Ron
Hermione Hermione
Hagrid Hagrid
Albus Albus
Thinking that there might be something unusual about the web page I looked at the source, but it appeared "normal" -- that is, as expected.
Note: I have done this operation several times before, and have never seen this occur before.
Note: In the actual data some of the lines have quoted text in them. Curiously there is weird behavior on these lines. In some cases the entire line is shown only once. (These occur at the top of the line, and the quoted text is at the beginning of the name.)
When quoted text appears "later" in the name, in some cases the quoted text is duplicated, and in other cases the quoted text is missing altogether! I have also noticed an error with the quoted text, and so will be reporting that to the web site which generates the HTML.
Note that each line of "text" is "anchor text", so if I click on a name the browser navigates to a page for that name.
I believe that the problem is that the COPY operating in Firefox is not simply copying the visible text, but also the ALT=
Below is a sample of what the source HTML looks like:
<a class="lnk" target="_blank" href="http://details.aspx?id=Harry">
<img width="16" height="16" alt="Harry" class="tb_icon" src="http://.../Harry.gif"/>
<span>Harry</span></a>
<br/>
(Because of the true length of the lines in the source HTML, I have stripped out the actual URL of the site.)
To make sure I wasn't imaging this difference I repeated the process within Internet Explorer. In that browser I did not get duplicated data.

Try:
*Extended Copy Menu (fix version): https://addons.mozilla.org/firefox/addon/extended-copy-menu-fix-vers/

Printing a selection from a web page, word document etc.

It embarrasses me to say, I cannot find a way to print a selection--a paragraph for example--from a web page or a Word document. In Windows it's done with a few mouse clicks: file/print/selection. I'm sure there must be a simple way to do it in Leopard, but what is it?

While there is no "print selection" command, you can use one of the globally available services to open a new window in textedit containing your selection, then print it.
Select something and then go to Safari>Services>TextEdit>New Window Containing Selection
and a new textedit window will open with your selection pasted in. Now you can print it.
I suppose if you do this a lot, you could write a short applescript that'll do this, print it, and quit textedit, and then stick the script in the script menu. Just a thought.

How to retrieve data from a web page through php scripts..........

kindly suggest me the php parsing script so that i can fetch the data from a web page.....
suppose we have a url.........
http://abc.com/news/companydetails.aspx?sskicode=x&Exchange=y
and the page contains the various fields.........like
xyz 10
xyz1 20
xyz2 30 etc...
then we have to retrive data from this page trough php script and insert it into database.....
value of xyz , xyz1 n xyz2 should be retrived and further inserted into database.......
thanx ......

Should be nice..
But its not working i think..

Extracting info from a web page

Similar Messages

Maybe you are looking for