Extract Data From Website

Hello,
I have a task to extract data from a website, on the below website you can find a table "Performance Summary" and i need to extract that entire table which you see in the middle of the page.
http://www.tadawul.com.sa/wps/portal/!ut/p/c1/04_SB8K8xLLM9MSSzPy8xBz9CP0os3g_A-ewIE8TIwN3Q0tDA0_v4EDLUCNHIwMvc6B8JJK8QbCpgYGniU9YiLOPu7GBgQFJut0DwkxBuoONggO8jA08jQjo9vPIz03VJ-RmVDe5O5rqB6fm6RfkhkYYG1Q6AgCbZOXy/dl2/d1/L3dDb1ZBQSEhL3dHa0FKREFOZ0EhIS9ZQkpKdzQ1dy83X04wQ1ZSSTQyMEcxOTEwSUtTUTlVMkEyMEI1/?symbol=1330&tabOrder=2&chart_type=chart_oneDay&announcmentNumber=&isAnnual=
Extract and load the data into a table.
Can you please share what is the best approach to do this?

Not exactly for SSIS folks VJ
See if you can parse the HTML in C#
Arthur
MyBlog
Twitter

Similar Messages

Is it possible to Extract the Data from Website?

Dear Experts,
Is it possible to Extract the Data from Website and Upload it in the SAP Table?.It is very Urgent.Please help me.
Thanks & Regards,
Ashok.

Hi Friend,
Generaly if it is one time activity you can do it manually by copy paste in excel.
But if it is weekly or monthly activity then you have to ask website provide to provide data in excel or CSV file and upload it into SAP.
I have a situtation like this, and they (Rapaport price) provide it in CSV format with defined columns.
So it is better to raise the issue to website provider.
Regards
Krishnendu

How to extract Live data from website

Dear Java Specialist,
I am wondering how to write a Java program to regularly extract certain important live data such as share price on a daily basis:
( i ) This program will need to step through a few submenus, possibly accept certain condition to proceed to where the stock price of the interested company which will be displayed on a particular pane/window.
( ii ) These data can come in the form of Excel/HTML/PDF format.
Could Web Service or Java script do the job? Obviously the simpler a solution the better. Please provide some direction on which technology to a achieve this objective with the least effort. Note that these websites could use any form of technology and hence our solution needs to be technology independent.
Any assistance would be much appreicated.
Thanks,
George

I tried the following codes but got a connection refused error:
import java.io.*;
import java.net.*;
public class DnldURL {
   public static void main (String[] args) {
      URL u;
      InputStream is = null;
      DataInputStream dis;
      String s;
      try {
//         u = new URL("http://www.homepriceguide.com.au/auction_results/index.cfm?action=view&suburbORpostcode=2010&st_locale=Darlinghurst&source=apm");
//         u = new URL("http://localhost:8080/index.html");
           u = new URL("http://www.yahoo.com.au/index.html");
         is = u.openStream();         // throws an IOException
         dis = new DataInputStream(new BufferedInputStream(is));
         BufferedReader br = new BufferedReader(new InputStreamReader(dis));
      String strLine;
      //Read File Line By Line
      while ((strLine = br.readLine()) != null)      {
      // Print the content on the console
          System.out.println (strLine);
      //Close the input stream
      dis.close();
      } catch (MalformedURLException mue) {
         System.out.println("Ouch - a MalformedURLException happened.");
         mue.printStackTrace();
         System.exit(1);
      } catch (IOException ioe) {
         System.out.println("Oops- an IOException happened.");
         ioe.printStackTrace();
         System.exit(1);
      } finally {
         try {
            is.close();
         } catch (IOException ioe) {
} // end of class definition
The output from JDK 1.6.0_03 and Netbeans 6.0 on Windows XP platform is as follows:
Execute:Java13CommandLauncher: Executing 'C:\Program Files\Java\jdk1.6.0_03\jre\bin\java.exe' with arguments:
'-classpath'
'C:\Documents and Settings\htran\DnldURL\build\classes'
'DnldURL'
The ' characters around the executable and arguments are
not part of the command.
Oops- an IOException happened.
java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
at sun.net.www.http.HttpClient.New(HttpClient.java:306)
at sun.net.www.http.HttpClient.New(HttpClient.java:323)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:788)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:729)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:654)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:977)
at java.net.URL.openStream(URL.java:1009)
at DnldURL.main(DnldURL.java:45)
Java Result: 1
BUILD SUCCESSFUL (total time: 1 second)
It appears to have partially worked when u = new URL("http://localhost:8080/index.html");. However, my purpose is to be able to extract data from various pane/screen when u = new URL("http://www.homepriceguide.com.au/auction_results/index.cfm? action=view&suburbORpostcode=2010&st_locale=Darlinghurst&source=apm");Likewise, neither http://www.yahoo.com:80/index or http://www.yahoo.com:8080/index worked. Was it because Yahoo was using a different port all together? If so, what port are they or the industry in general are using?
Many thanks,
George

How to extract data from XML file with JavaScript

HI All
I am new to this group.
Can anybody help me regarding XML.
I want to know How to extract data from XML file with JavaScript.
And also how to use API for XML
regards
Nagaraju

This is a Java forum.
JavaScript is something entirely different than Java, even though the names are similar.
Try another website with forums about JavaScript.
For example here: http://www.webdeveloper.com/forum/forumdisplay.php?s=&forumid=3

How to extract data from database to XSLT?

I want to generate a report by XSLT, but the data is extracted from the database. I will use Access/SQL server for my database. Can i write SQL in XSLT to extract data from database? Have any sample code or reference website to show how it work?
THX

for example: "SELECT code, name FROM TABLE FOR XML RAW"
String xml = null;
if(rs.next()){
xml = rs.getString(1);
}You will get xml string something like this:
<row empID="1234"/><row empID="1235"/>
You can construct a DOM using this xml data and operate on it.
However you may explore more on resulting xml format.
A different SQL Query (rather than using XML RAW) may give output in a more desired format.

Extract data from database to xml ?

is there any API, classes or tools ready to extract data from oracle database and generate xml files, Instead to implement a DOM class and build little by little the xml file??
thanks for every info in advance.

Yes, download the Oracle XML Development Kit from Oracle's website and look at the documentation of the XML-SQL Utility.

Unable to extract data from an AS/400 system.

Hello experts.
We are trying to extract data from an AS/400 system but not having any success until now.
I´ll write down you the stepts that we have followed until now:
1.- Create a DB Connect between both systems
2.- Create a Source System from AS400 in the workbench under DB Connect Directory
3.- Generate datasources from tebles specified in the schema of the connection
break point -
At this point, we had a problem with some tables with at least one fieldname containing character "Ñ".
After asking some possible solutions to SAP, the told us this is not supported, as the system can´t have any object with character "Ñ", so the transfer structure was unable to activate with this fields in the datasource.
--- end of brek point --
4.- After those issues, we´ve decided to implement, in another schema, views from those tables which had the fieldnames with that character "Ñ", changing them to an "N".
5.- We´ve created another source system with that schema, and user than can see that schema.
6.- To be able to see those views, in transaction RSDBC, we had to deactivate the two checkboxes in the first window ( Choose tables and Choose views) .
7.- Right afeter, we could generate correctly the datasources from this logical tables.
8.- We have designed the hole dataflow for this datasources and everithing went rigth.
9.- But wen we tried to execute the infopackage to extract data from those logic tables, we cannot get any registers. Acctualy the charge remains yellow after the job have finished.
Please, I would appreciate any help you could give us on this problem.
Thank you very much
Regards
Joaquin

I´d like ti add something to this thread, and maybe clarify a littel bit the question.
The only way that the BW system recognizes those logical tables, through transaction RSDBC is checking out the two boxes on this transaction, "Select Tables" and "Select Views".
I don´t know haw these logical tables have been created, bus does this mean that the are not neither tables or views as BW understand them.
Please, if someone knows anythin about this, answer to this thread.
Thank you very much.
Joaquin Sobrido

Extracting Data from APO PP/DS to BW

Hi Gurus,
I'm trying to extract data from APO PP/DS (SCM 5.1) to BW (BI 7.0). I'm new to SCM and am not sure how the extraction from SCM to BI happens other than that we need to read data from LiveCache.
The extractor we are interested are
0APO_PPDS_RESCAPREQ_01
0APO_PPDS_PROD_CUST_01
0APO_PPDS_OPERATION_01
0APO_PPDS_ORDER_01
Pls kindly help me out with the procedure.
Thanks in advance.

Hi
Have a look at the below urls..
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/5f229690-0201-0010-84ba-9ee5a8958a05
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/4fe5d590-0201-0010-6c8d-ada86492cf11
Re: APO to BW Design Question
Re: APO BW Integration
Hope it helps
Thanks
Teja

What key fields should i set in DSO extracting data from 2LIS_02_ITM

hi experts
i extract data from 2LIS_02_ITM into a DSO, i know the DSO isn't a must, becoz the 2LIS_02_ITM delta type is ABR, but i want to keep the info in change log.
so, what the key fields should i set in the dso? just ebeln and ebelp is enough?
hunger for ur advice and thanks a lot!

If you extract ITM toa DSO you cannot maintain a log of every change....the data will come ....but when the data must be activated the reference of the ebelp ebeln will remain only a single record....if you want to maintain all the data you must create another field in extractor with you can difference all the changes for one single ebelp ebeln...
Regards

When I extracting data from DSO to Cube by using DTP.

When i am extracting data from DSO to Cube by using DTP.
I am getting following erros.
Data package processing terminated (Message no. RSBK229).
Error in BW: error getting datapakid cob_pro (Message no. RS_EXCEPTION105).
Error while extracting from source 0FC_DS08 (type DataStore) - (Message no. RSBK242).
Data package processing terminated. (Message no. RSBK229).
Data package 1 / 10/04/2011 15:49:56 / Status 'Processed with Errors'. (Message no. RSBK257).
This is the brand new BI 7.3 system. Implementing the PSCD and TRM.
I have used the standard business content objects in FI-CA (dunning history header, item,activities) and standard Datasource (0FC_DUN_HEADER ,0FC_DUN_ITEMS, 0FC_DUN_ACTIVITIES). I have extracted data till the DSO level . when I try to pull the data to info provider level(cube) using DTP . I am getting fallowing error.
my observation: when ever I use the DSO as source to any target like another DSO or cube. its throwing same kind of error for any flow including simple Flat file .
please suggest any one whether do I need to maintain basic settings since its a brand new BI 7.3.
please help me out on this issue . I am not able to move forward .its very urgent

hello
Have you solved the problem ?
I have the same error...
as you solve this error,
can you help me please I have the same error
yimi castro garcia
[email protected]

How can I join 3 tables while extracting data from SAP R/3?

I have 3 tables with the following columns
Emp table (emp)
emp_id
emp_name
emp_add
Dept table (dept)
dept_id
dept_name
dept_loc
Location table (loc)
loc_id
loc_name
Now. If I want to select data from loc_id = 10 and emp_id between 2000 and 3000
How to join these three tables while extracting data from R/3
join condition
loc.loc_id = dept.loc_id
and dept.dept_id = emp.dept_id
and loc.loc_id =10
and emp.emp_id between 2000 and 3000.
Could any one let me know the procedure to extract this data into BW system.

Hi,
shouldn't your join condition be:
loc.loc_id = dept.DEPT_LOC
and dept.dept_id = ??
If you can join the three tables then create a generic datasource (RSO) based on a view (create your view with your join in SE11).
Enable the loc_id and the emp_id as selectable in the datasource so you can then select the values from a BW IPack.
hope this helps...
Olivier.

Extracting data from Z-table from SAP R/3 to BW

Hi all
I want to extract data from a Z-table from SAP R/3 system to Bw system. Currently I am on BW 3.5. Since it is a Z table I dont have a standard extractor for it & I dont knw how to create it. Can anyone provide me with the step-by-step documentation of how to extract data from a non standard SAP table????

Hi
You need to create Generic Datasource on the Z-Table you want to get data from
Go to RSO2 transaction to create generic datasource .
You need to give technical name of datasource under datasource type you want and click on create. Then you can give descrption and Application component under which u want see the datasource,
enter the z table name under view/ table and save.
here you can click on check boxes to make fields hidden or selection fields.
Regards
Ravi
Edited by: Ravi Naalla on Aug 25, 2009 8:24 AM

Generate Insert Statement Script to Extract Data from Table in Oracle 7i

Hi all, I have an old Oracle legacy system that is running for over 15 years.Every now and then we need to extract data from this table@ ORacle 7i to be imported back to Oracle 10G.
My thoughts are to create a script of Insert statements in oracle 7 and that to be deployed back to Oracle 10G.
I found this scripts in Google and not sure how exactly this works.Any explanation on thsi scripts , would be greatly appreciated.I find this scripst may help to generate a set of insert statements from that table to the latest table at 10G.
<pre>
-- Step 1: Create this procedure:
create or replace Function ExtractData(v_table_name varchar2) return varchar2 As
b_found boolean:=false;
v_tempa varchar2(8000);
v_tempb varchar2(8000);
v_tempc varchar2(255);
begin
for tab_rec in (select table_name from user_tables where table_name=upper(v_table_name))
loop
b_found:=true;
v_tempa:='select ''insert into '||tab_rec.table_name||' (';
for col_rec in (select * from user_tab_columns
where
table_name=tab_rec.table_name
order by
column_id)
loop
if col_rec.column_id=1 then
v_tempa:=v_tempa||'''||chr(10)||''';
else
v_tempa:=v_tempa||',''||chr(10)||''';
v_tempb:=v_tempb||',''||chr(10)||''';
end if;
v_tempa:=v_tempa||col_rec.column_name;
if instr(col_rec.data_type,'CHAR') > 0 then
v_tempc:='''''''''||'||col_rec.column_name||'||''''''''';
elsif instr(col_rec.data_type,'DATE') > 0 then
v_tempc:='''to_date(''''''||to_char('||col_rec.column_name||',''mm/dd/yyyy hh24:mi'')||'''''',''''mm/dd/yyyy hh24:mi'''')''';
else
v_tempc:=col_rec.column_name;
end if;
v_tempb:=v_tempb||'''||decode('||col_rec.column_name||',Null,''Null'','||v_tempc||')||''';
end loop;
v_tempa:=v_tempa||') values ('||v_tempb||');'' from '||tab_rec.table_name||';';
end loop;
if Not b_found then
v_tempa:='-- Table '||v_table_name||' not found';
else
v_tempa:=v_tempa||chr(10)||'select ''-- commit;'' from dual;';
end if;
return v_tempa;
end;
show errors
-- STEP 2: Run the following code to extract the data.
set head off
set pages 0
set trims on
set lines 2000
set feed off
set echo off
var retline varchar2(4000)
spool c:\t1.sql
select 'set echo off' from dual;
select 'spool c:\recreatedata.sql' from dual;
select 'select ''-- This data was extracted on ''||to_char(sysdate,''mm/dd/yyyy hh24:mi'') from dual;' from dual;
-- Repeat the following two lines as many times as tables you want to extract
exec :retline:=ExtractData('dept');
print :retline;
exec :retline:=ExtractData('emp');
print :retline;
select 'spool off' from dual;
spool off
@c:\t1
-- STEP3: Run the spooled output c:\recreatedata.sql to recreate data.
Source:http://www.idevelopment.info/data/Oracle/DBA_tips/PL_SQL/PLSQL_5.shtml
</pre>

Thanks Justin.
I get what you are saying,i really wanted to see the output of the codes, because the furtherst i could get from that code is
SELECT EXTRACTDATA('MYTABLE') FROM MYTABLE;
and it generated this:
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
"select 'insert into MYTABLE ('||chr(10)||'DATE1,'||chr(10)||'TIME1,'||chr(10)||'COUNTS) values ('||decode(DATE1,Null,'Null','to_date('''||to_char(DATE1,'mm/dd/yyyy hh24:mi')||''',''mm/dd/yyyy hh24:mi'')')||','||chr(10)||''||decode(TIME1,Null,'Null',TIME1)||','||chr(10)||''||decode(COUNTS,Null,'Null',COUNTS)||');' from MYTABLE;
select '-- commit;' from dual;"
I was expecting a string of
insert into mytable values (19/1/2009,1,1);
insert into mytable values (19/10/2008,5,10);
Thanks for the explanation .

Extract data from Essbase with HAL to a flat file

Hello,
I would like to extract data from Essbase with HAL.
I use the essbase Adapter with the "Extract Data" Method.
All is working when I use only the essbase Adaptater and configure it.
But I want to have a variable on my Years dimension.
I have to fill "Years member port" and "Years Criteria port" of my essbase Adapter with variable.
I have a "v_year" and "v_year_criteria" variable. I fill "v_year" with "FY05" but I don't know how fill my "v_year_criteria" variable.
Thanks a lor for answer.

If you are in 9.3x, I would recommend using the DATAEXPORT calc script function. You just FIX on what you want to export (including substitution variables), and then fill in some parameters and it writes a file. Look in the DBAG.

Extract data from Query to a Flat File - scheduling in the background

Hi All,
We need to extract data from a query to a flat file periodically, but each time 10 separate executions based on 10 profit centers, to get the separate files.
So, we have PC1, PC2, ... PC10 Profit centeres.
We have Query1, with profit center variable.
We have created 10 variants for Query1 so that we can execute for 10 profit centers.
Now, we have used RSCRM_BAPI to execute the Query1 and write to a flat file. Now we are having a problem in getting this feature used to execute the query automatically for 10 different varaints.
Thanks in advance,
_ Shashi

Shashi,
You can try using Information Broadcasting or Reporting Agent. You can schedule this in Information Broadcasting.
check this link for Information Broadcasting... <a href="http://help.sap.com/saphelp_nw04s/helpdata/en/a5/359840dfa5a160e10000000a1550b0/content.htm">http://help.sap.com/saphelp_nw04s/helpdata/en/a5/359840dfa5a160e10000000a1550b0/content.htm</a>
Nagesh Ganisetti.

Extract Data From Website

Similar Messages

Maybe you are looking for