Data Load from Source to Destination

Hello Forum Members,
I have to databases on different solaris servers.
I have to load data from Source Table[Server 1] to Target Current Table[truncate existing data and load from Source Table] [Server 2]and on every Wednesday from Target Current Table into Target History Table[Data always appended][Server 2].
Any advice how to solve the issue is highly appreciated.
Thanks and Regards,
Suresh

Unless there is a history table on the remote server, the history table would not appear to be an appropriate candidate for a materialized view.
You could, in theory, create the live table as a materialized view rather than doing a TRUNCATE (or DELETE) and INSERT INTO SELECT in your stored procedure. Since it sounds like you need to have some coordination between the refreshing of the materialized view and the population of the history table, it's not obvious that creating the live table as a materialized view buys you anything.
Justin

Similar Messages

Data loading from source system takes long time.

Hi,
I am loading data from R/3 to BW. I am getting following message in the monitor.
Request still running
Diagnosis
No errors could be found. The current process has probably not finished yet.
System response
The ALE inbox of the SAP BW is identical to the ALE outbox of the source system
and/or
the maximum wait time for this request has not yet run out
and/or
the batch job in the source system has not yet ended.
Current status
in the source system
Is there anything wrong with partner profile maintanance in the source system.
Cheers
Senthil

Hi,
I will suggest you to check a few places where you can see the status
1) SM37 job log (In source system if load is from R/3 or in BW if its a datamart load) (give request name) and it should give you the details about the request. If its active make sure that the job log is getting updated at frequent intervals.
Also see if there is any 'sysfail' for any datapacket in SM37.
2) SM66 get the job details (server name PID etc from SM37) and see in SM66 if the job is running or not. (In source system if load is from R/3 or in BW if its a datamart load). See if its accessing/updating some tables or is not doing anything at all.
3) RSMO see what is available in details tab. It may be in update rules.
4) ST22 check if any short dump has occured.(In source system if load is from R/3 or in BW if its a datamart load)
5) SM58 and BD87 for pending tRFCs and IDOCS.
Once you identify you can rectify the error.
If all the records are in PSA you can pull it from the PSA to target. Else you may have to pull it again from source infoprovider.
If its running and if you are able to see it active in SM66 you can wait for some time to let it finish. You can also try SM50 / SM51 to see what is happening in the system level like reading/inserting tables etc.
If you feel its active and running you can verify by checking if the number of records has increased in the data tables.
SM21 - System log can also be helpful.
Thanks,
JituK

Error is data loading from 3rd party source system with DBCONNECT

Hi,
We have just finished an upgrade of SAP BW 3.10 to SAP NW 7.0 EHP1.
After the upgrade, we are facing a problem with data loads from a third party Oracle source system using DBConnect.
The connection is working OK and we can see the tables in the source system. But we cannot load the data.
The error in the monitor is as follows:
'Error message from the source system
Diagnosis
An error occurred in the source system.
System Response
Caller 09 contains an error message.
Further analysis:
The error occurred in Extractor .
Refer to the error message.'
But, unfortunately, the error message has no further information.
If we look at the job log in sm37, the job finished with the following log -
27.10.2009 12:14:19 Job started                                                                                00           516          S
27.10.2009 12:14:19 Step 001 started (program RSBATCH1, variant &0000000000119, user ID RXSAHA)                    00           550          S
27.10.2009 12:14:23 Start InfoPackage ZPAK_4FMNJ2ZHNNXC6HT3A2TYAAFXG                                              RSM1          797          S
27.10.2009 12:14:24 Element NOAUTHORITYCHECK is not available in the container                                     OL           356          S
27.10.2009 12:14:24 InfoPackage ZPAK_4FMNJ2ZHNNXC6HT3A2TYAAFXG created request REQU_4FMXSQ6TLSK5CYLXPBOGKF31G     RSM1          796          S
27.10.2009 12:14:24 Job finished                                                                                00           517          S
In a BW 3.10 system, there is no message related to element NOAUTHORITYCHECK. So, I am wondering if this is something new in NW 7.0.
Thanks in advance,
Rajib

There will be three things to get the errors like this
1.RFC CONNECTION FAILED
2.CHECK THE SOURCE SYSTEM
3.CHECK IT OUT WITH Oracle Consultants WEATHER THEY ARE FILLING UP THE LOADS.TELL THEM TO STOP
4.CHECK I DOC PROCESSING
5.FINALLY MEMORY ISSUES.
6.CATCH THE DATA SOURCE FIRST CHANGE IT AND THEN ACTIVATE AND RUN THE LOAD
7.LAST IS MEMORY ISSUE.
and also Check the RFC connection in SM59 If it is ok then
check the SAP note : 692195 for authorization
Santosh

Data Load from a DSO to a Cube

Hello,
I am facing a problem
Case:
There is a DSO with following Char/Key Figure:
Characteristics     ( Value-1 )     ( Value-2 )     ( Value-3 )
A(Key)          ( A1 )     ( A2 )     ( A3 )
B(Key)          ( B1 )     ( B2 )     ( B3 )
C          ( C1 )     ( C1 )     ( C1 )
D          ( D1 )     ( D1 )     ( D1 )
E          ( E1 )     ( E1 )     ( E1 )
F          ( F1 )     ( F1 )     ( F1 )
KF1(Key Figure)     ( 10 )     ( 20 )     ( 30 )
I got below data from the DSO to a Cube(structure is below with data) details are below:
Dimention          ( Values )
C          ( C1 )
D          ( D1 )
E          ( E1 )
F          ( F1 )
KF1(Key Figure)     ( 50 )
My Question is why i am not getting KF1 value as 60.
When i tried with including Key Fields of the DSO, i got all the records in the Cubes.
Where i am mistaking?
Can some one correct me please?
Thanks.
Avinash.

Below are the basic steps which we follow in any BI 2004S system:
1)Create datasource. Here u can set/check the Soucre System fields.
2)Create Transformation for that datasource. (no more update rules/transfer rules)
2.1) While creating transformation for DS it will ask you for data target name, so just assign where u want to update ur data.
DataSource -> Transformation -> Data Target
Now if you want to load data into data target from Source System Datasource:
1) Create infopackage for that data source. If you are creating infopackage for new datasources, it will only allow you update upto PSA, all other options u can see as disabled.
2)Now Create DTP (Data Transfer Process) for that data source.
3) NOw schdule the Infopackage, once the data is loaded to PSA, you can execute your DTP which will load data to data target.
If you are loading data from one one data target to other, no need to use PSA, you can directly execute DTP in that case.
Data Source -> Transformation (IP/DTP) -> Data Target1 -> DTP ->Data Target 2
Use the below link for detailed example:
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/fc61e12d-0a01-0010-2883-e2fc63ef729b
Infosources are no more mandatory with BI 7.0, below is the link to scenarios where we use infosources:
http://help.sap.com/saphelp_nw04s/helpdata/en/44/0243dd8ae1603ae10000000a1553f6/content.htm
Full or delta depends on your requirement...
chk the below thread to know better
difference between the various loads
hope it helps
Message was edited by:
        sriram viswanathan

How to schedule Job for data uploading from source to BI

Hi to all,
How to schedule Job for data uploading from source to BI,
Why we required and how we do it.
As I am fresher in BI, I need to know from bottom.
Regards
Pavneet Rana

Hi.
You can create [process chain |http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502b2998-1017-2d10-1c8a-a57a35d52bc8?quicklink=index&overridelayout=true]for data loading pocess and schedule start process to any time/date etc ...
Regadrs.

Move files from source to destination directory

I would like to copy files from source directory to destination directory, which have a creation date lower than p_date. p_date is a entered value of a forms field.
source:
c:\tmp\my_b.pdf 10.10.06 04:21
c:\tmp\my_c.pdf 10.10.06 04:20
c:\tmp\my_d.pdf 09.10.06 01:35
c:\tmp\my_e.pdf 05.10.06 02:13
c:\tmp\my_f.pdf 01.10.06 00:20
User enter the value: 09.10.06 09:35
destination should be:
f:\tmp\my_d.pdf 09.10.06 01:35
f:\tmp\my_e.pdf 05.10.06 02:13
f:\tmp\my_f.pdf 01.10.06 00:20
win_api_utility.move_file move all files from source to destination, but only I need a part of them. I have installed Forms 6i and Windows 2000. Does anybody have an idea?
Best regards,
tom

Hi Tom,
In a Forms 10g environment you would probably use a Java component. On a c/s environment I would issue a host command. Although the windows 'copy' statement has no 'date' flag, the windows 'xcopy' statement has. Type 'help xcopy' in a command/dos box for the syntax/flags for the 'xcopy' statement.
Hope this helps,
Matthieu de Graaf

Number of parallel process definition during data load from R/3 to BI

Dear Friends,
We are using Bi7.00. We have a requirement in which i should increase the number of parallel process during data load from R/3 to BI. I want to modify this for a particular data source and check.Can experts provide helpful answers for the following question.
1) When load is taking place or have taken place, where can we see how many parallel process that particular load has taken.
2) Where should i change the setting for the number of parallel process for data load (from R/3 to BI) and not within BI.
3) How system works and what will be net result of increasing or decreasing the number of parallel process.
Expecting Experts help.
Regards,
M.M

Dear Des Gallagher,
Thank you very much for the useful information provided. The following was my observation.
From the posts in this forum, i was given to understand that the setting for specific data source can be done in the infopackage and DTP level, i carried out the same and found that there is no change in the load, i.e., system by default takes only one parallel process even though i maintained 6.
Can you kindly explain about the above mentioned point. i.e.,
1) Even though the value is maintained in the infopackage level , will system consider it or not. -> if not then from which transaction system is able to derive the 1 parallel process.
Actually we wanted to increase the package size but we failed because i could not understand what values have to be maintained -> can you explain in detail
Can you calrify my doubt and provide solution?
Regards,
M.M

Doesn't load from source system to PSA and DSO

Hi Gurus,
We have launched a Delta load from Source System (POSDM) to DSO. In the source system there are 5000000 rows but none was loaded into DSO or PSA. The delta load was ok, but any data was loaded.
If the delta load was lunch again, would be loaded those 5000000 rows. Or those 5000000 rows wouldn't be able to load again.
Any idea abut this issue? Any feedback will be really appreciated.
Thanks in advance.

Hi David,
Are you sure this 5 million records are delta records and should be pulled into BW as delta??
did you count the number of records in the underlying tables??
which data source you are using??
Delta loads are suppose to bring the new records or changed records since the last data load and not all the records.
Since the request is green as you said and still it shows 0 records then it is possible that nothing has changed since last delta.try to see the details of the data load... if it failed...may be thats the reason you are not able to see any records.
If you schedule the delta again it will bring the records changed or created after last delta.
If the delta was unsuccessfull then turn the QM status and overall status of request to red manually in the monitor of the request and delete it from PSA and all the targets in which its loaded and schedule the delta again and it should bring the delta.
Thanks
Ajeet
Thanks
Ajeet

Data loading from flat file to cube using bw3.5

Hi Experts,
                   Kindly give me the detailed steps with screens about Data loading from flat file to cube using bw3.5
       ...............Please

Hi ,
Procedure
You are in the Data Warehousing Workbench in the DataSource tree.
       1.      Select the application components in which you want to create the DataSource and choose Create DataSource.
       2.      On the next screen, enter a technical name for the DataSource, select the type of DataSource and choose Copy.
The DataSource maintenance screen appears.
       3.      Go to the General tab page.
                            a.      Enter descriptions for the DataSource (short, medium, long).
                            b.      As required, specify whether the DataSource builds an initial non-cumulative and can return duplicate data records within a request.
                            c.      Specify whether you want to generate the PSA for the DataSource in the character format. If the PSA is not typed it is not generated in a typed structure but is generated with character-like fields of type CHAR only.
Use this option if conversion during loading causes problems, for example, because there is no appropriate conversion routine, or if the source cannot guarantee that data is loaded with the correct data type.
In this case, after you have activated the DataSource you can load data into the PSA and correct it there.
       4.      Go to the Extraction tab page.
                            a.      Define the delta process for the DataSource.
                            b.      Specify whether you want the DataSource to support direct access to data.
                            c.      Real-time data acquisition is not supported for data transfer from files.
                            d.      Select the adapter for the data transfer. You can load text files or binary files from your local work station or from the application server.
Text-type files only contain characters that can be displayed and read as text. CSV and ASCII files are examples of text files. For CSV files you have to specify a character that separates the individual field values. In BI, you have to specify this separator character and an escape character which specifies this character as a component of the value if required. After specifying these characters, you have to use them in the file. ASCII files contain data in a specified length. The defined field length in the file must be the same as the assigned field in BI.
Binary files contain data in the form of Bytes. A file of this type can contain any type of Byte value, including Bytes that cannot be displayed or read as text. In this case, the field values in the file have to be the same as the internal format of the assigned field in BI.
Choose Properties if you want to display the general adapter properties.
                            e.      Select the path to the file that you want to load or enter the name of the file directly, for example C:/Daten/US/Kosten97.csv.
You can also create a routine that determines the name of your file. If you do not create a routine to determine the name of the file, the system reads the file name directly from the File Name field.
                              f.      Depending on the adapter and the file to be loaded, make further settings.
■       For binary files:
Specify the character record settings for the data that you want to transfer.
■       Text-type files:
Specify how many rows in your file are header rows and can therefore be ignored when the data is transferred.
Specify the character record settings for the data that you want to transfer.
For ASCII files:
If you are loading data from an ASCII file, the data is requested with a fixed data record length.
For CSV files:
If you are loading data from an Excel CSV file, specify the data separator and the escape character.
Specify the separator that your file uses to divide the fields in the Data Separator field.
If the data separator character is a part of the value, the file indicates this by enclosing the value in particular start and end characters. Enter these start and end characters in the Escape Charactersfield.
You chose the; character as the data separator. However, your file contains the value 12;45 for a field. If you set u201C as the escape character, the value in the file must be u201C12;45u201D so that 12;45 is loaded into BI. The complete value that you want to transfer has to be enclosed by the escape characters.
If the escape characters do not enclose the value but are used within the value, the system interprets the escape characters as a normal part of the value. If you have specified u201C as the escape character, the value 12u201D45 is transferred as 12u201D45 and 12u201D45u201D is transferred as 12u201D45u201D.
In a text editor (for example, Notepad) check the data separator and the escape character currently being used in the file. These depend on the country version of the file you used.
Note that if you do not specify an escape character, the space character is interpreted as the escape character. We recommend that you use a different character as the escape character.
If you select the Hex indicator, you can specify the data separator and the escape character in hexadecimal format. When you enter a character for the data separator and the escape character, these are displayed as hexadecimal code after the entries have been checked. A two character entry for a data separator or an escape sign is always interpreted as a hexadecimal entry.
                            g.      Make the settings for the number format (thousand separator and character used to represent a decimal point), as required.
                            h.      Make the settings for currency conversion, as required.
                              i.      Make any further settings that are dependent on your selection, as required.
       5.      Go to the Proposal tab page.
This tab page is only relevant for CSV files. For files in different formats, define the field list on the Fields tab page.
Here you create a proposal for the field list of the DataSource based on the sample data from your CSV file.
                            a.      Specify the number of data records that you want to load and choose Upload Sample Data.
The data is displayed in the upper area of the tab page in the format of your file.
The system displays the proposal for the field list in the lower area of the tab page.
                            b.      In the table of proposed fields, use Copy to Field List to select the fields you want to copy to the field list of the DataSource. All fields are selected by default.
       6.      Go to the Fields tab page.
Here you edit the fields that you transferred to the field list of the DataSource from the Proposal tab page. If you did not transfer the field list from a proposal, you can define the fields of the DataSource here.
                            a.      To define a field, choose Insert Row and specify a field name.
                            b.      Under Transfer, specify the decision-relevant DataSource fields that you want to be available for extraction and transferred to BI.
                            c.      Instead of generating a proposal for the field list, you can enter InfoObjects to define the fields of the DataSource. Under Template InfoObject, specify InfoObjects for the fields in BI. This allows you to transfer the technical properties of the InfoObjects into the DataSource field.
Entering InfoObjects here does not equate to assigning them to DataSource fields. Assignments are made in the transformation. When you define the transformation, the system proposes the InfoObjects you entered here as InfoObjects that you might want to assign to a field.
                            d.      Change the data type of the field if required.
                            e.      Specify the key fields of the DataSource.
These fields are generated as a secondary index in the PSA. This is important in ensuring good performance for data transfer process selections, in particular with semantic grouping.
                              f.      Specify whether lowercase is supported.
                            g.      Specify whether the source provides the data in the internal or external format.
                            h.      If you choose the external format, ensure that the output length of the field (external length) is correct. Change the entries, as required.
                              i.      If required, specify a conversion routine that converts data from an external format into an internal format.
                              j.      Select the fields that you want to be able to set selection criteria for when scheduling a data request using an InfoPackage. Data for this type of field is transferred in accordance with the selection criteria specified in the InfoPackage.
                            k.      Choose the selection options (such as EQ, BT) that you want to be available for selection in the InfoPackage.
                              l.      Under Field Type, specify whether the data to be selected is language-dependent or time-dependent, as required.
       7.      Check, save and activate the DataSource.
       8.      Go to the Preview tab page.
If you select Read Preview Data, the number of data records you specified in your field selection is displayed in a preview.
This function allows you to check whether the data formats and data are correct.
For More Info: http://help.sap.com/saphelp_nw70/helpdata/EN/43/01ed2fe3811a77e10000000a422035/content.htm

Data loading from DSO to Cube

Hi,
I have a question,
In book TBW10 i read about the data load from DSO to InfoCube
" We feed the change log data to the InfoCube, 10, -10, and 30 add to the correct 30 value"
My question is cube already have 10 value, if we are sending 10, -10 and 30 Values(delta), the total should be 40 instead of 30.
Please some one explaine me.
Thanks

No, it will not be 40.
It ll be 30 only.
Since cube already has 10, so before image ll nullify it by sending -10 and then the correct value in after immage ll be added as 30.
so it ll be like this 10-10+30 = 30.
Thank-You.
Regards,
Vinod

How to find the data loaded from r/3 to bw

hi
how to find the data loaded from r/3 to bw is correct . i am not able to find which feild in the query is connected to which feild in the r/3 . where i am geting the data from r/3 . is there any process to find which feild and table the data is comming from . plz help
thanks in advance to u all

Hi Veda ... the mapping between R/3 fields and BW InfoObjects should take place in Transfer Rules. Other transformation could take place in Update Rule.
So you could proceed this way: look at InfoProvider Data Model and see if the Query does perform any calculation (even with Virtual keyfigures / chars). Than go back to Update Rules and search for other calculation / transformation. At least there are Tranfer Rule and eventually DataSource / Extraction Enhancements.
As you can easily get there are many points where you have to look for ... it's a quite complex work but very usefull.
Once you will have identified all mappings / transfromation see if BW data matchs R/3 (considering calculations ...)
Good job
GFV

How we can automate the data loading from BI-BPC

Dear Guru's
Thanks for watching this thread,my question is
How we can load the data from BI7.0 to BPC.My environment is SAP-BI 7.0 and BPC is 7.5 MS version and 2008SQL.
How we can automate the data loading from BI- BPC Ms version.Is manual flat file load is mandatory in ms version.
Thanks in Advance,
Srinivasan.

Here are some options
1) Use standars packages and schedule them :
A) Openhub masterdata file into a flat file/ BPC App server and Schedule the package - Import Master Data from a Data File and other relevent packages.
2 ) Using Custom Tasks in Custom Packages ( SSIS)
Procedure
From the Microsoft SQL Server Business Intelligence Developer Studio, open the Microsoft SSIS folder.
Create a new package, or select an existing package to modify.
Choose Task Register Custom Task .
In the Task Location field, browse for the target .dll file.
Note
By default, the .dll files are stored in BPC/Websrvr/bin.
End of the note.
Enter a task description, select an appropriate icon, then click OK.
Drag the icon to the designer window. Enter data as required.
Save the package.

Data load from R3 is slow and gives rfc connection error

Hi all, is there any debug capabilties available on BI 7.0 when it comes to debug data load from R3 using rfc connection ( aleremote and bwiremote users )
any ideas where can I look for solution.
thanks.

Hi,
Check the connection between the R/3 and BW.
or It may be the network problem contact system admin or basis people.
thanks,
dru

Summing up key figure in Cube - data load from ODS

Hello Gurus,
I am doing a data load from ODS to cube.
The records in ODS are at line-item level, and all needs to be summed up at header level. There is only one key-figure. ODS has header and line-item fields.
I am loading only header field data ( and not the item-field data) from ODS to Cube.
I am expecting only-one record in cube ( for all the item-level records) with all the key-figures summed up. But couldn't see that.
Can anyone please explain how to achieve it.
I promise to reward points.
=====
Example to elaborate my point.
In ODS
Header-field item-field   quantity
123                301          10
123                302           20
123                303           30
Expected record in Cube
Header-field       Quantity
   123                    60
====================
Regards,
Pramod.

Hello Oscar and Paolo.
Thanks for the reply. I am using BW 7.0
Paolo suggested:
>>If you don't add item number to cube and put quantity as adition in update rules >>from ODS to cube it works.
I did that still I get 3 records in cube.
Oscar Suggested:
>>What kind of aggregate do you have for your key figure in update rules (update >>or no change)?
It is "summation". And it cannot be changed. (Or at least I do not know how to change it.)
There are other dimensions in the cube - which corresponds to the field(s) in ODS.
But, I just mentioned these two (i.e. header and item-level) for simplicity.
Can you please help?
Thank you.
Pramod.

BPC:: Master data load from BI Process chain

Hi,
we are trying to automatize the master data load from BI.
Now we are using a package with:
PROMPT(INFILES,,"Import file:",)
PROMPT(TRANSFORMATION,%TRANSFORMATION%,"Transformation file:",,,Import.xls)
PROMPT(DIMENSIONNAME,%DIMNAME%,"Dimension name:",,,%DIMS%)
PROMPT(RADIOBUTTON,%WRITEMODE%,"Write Mode",2,{"Overwirte","Update"},{"1","2"})
INFO(%TEMPNO1%,%INCREASENO%)
INFO(%TEMPNO2%,%INCREASENO%)
TASK(/CPMB/MASTER_CONVERT,OUTPUTNO,%TEMPNO1%)
TASK(/CPMB/MASTER_CONVERT,FORMULA_FILE_NO,%TEMPNO2%)
TASK(/CPMB/MASTER_CONVERT,TRANSFORMATIONFILEPATH,%TRANSFORMATION%)
TASK(/CPMB/MASTER_CONVERT,SUSER,%USER%)
TASK(/CPMB/MASTER_CONVERT,SAPPSET,%APPSET%)
TASK(/CPMB/MASTER_CONVERT,SAPP,%APP%)
TASK(/CPMB/MASTER_CONVERT,FILE,%FILE%)
TASK(/CPMB/MASTER_CONVERT,DIMNAME,%DIMNAME%)
TASK(/CPMB/MASTER_LOAD,INPUTNO,%TEMPNO1%)
TASK(/CPMB/MASTER_LOAD,FORMULA_FILE_NO,%TEMPNO2%)
TASK(/CPMB/MASTER_LOAD,DIMNAME,%DIMNAME%)
TASK(/CPMB/MASTER_LOAD,WRITEMODE,%WRITEMODE%)
But we need to include these tasks into a BI process chain.
How can we add the INFO statement into a process chain?
And how can we declare the variables?
Regards,
EZ.

Hi,
i have followed your recomendation, but when i try to use the process /CPMB/MASTER_CONVERT, with the parameter TRANSFORMATIONFILEPATH and the root of the transformation file as value, i have a new problem. The value only have 60 char, and my root is longer:
\ROOT\WEBFOLDERS\APPXX\PLANNING\DATAMANAGER\TRANSFORMATIONFILES\trans.xls
How can we put this root???
Regards,
EZ.

Data Load from Source to Destination

Similar Messages

Maybe you are looking for