Data loading into Data warehouse Tables

Hi all,
I am having scenario like this. I am having table with lacks of records. Every day, new data will come with some records DELETE ed, some new records
INSERT ed, some records UPDATE ed. I have to load this new data into parent table.
Currently, we are DELETING all parent data and inserting new data. It is taking a lot of time. There is NO PRIMARY KEY on this parent
table.
What are the best options to load the new data. Actually, there are several tables are there. I have mentioned one table for example.
Oracle version is 9i.
Thanks,
Pal

I need to know which is a better way of loading flat
files data into Warehouse tables. SQL*Loader or
Oracle DataMart Suite. Doesn't make much odds. The advantage of using the
Delimited or Fixed Length transform is that it keeps
everything within Data Mart Suite.
1. What all types of databases are supported by DataMart
ODBC. DataMart Suite will support most RDBMS's as an
ODBC source. Sybase, SQL Server, Informix, DB2, Redbrick,
Teradata. Note this is only a source for the
data, not a store for the DataMart
2. Can DataMart suite be programmed for auto operation of
loading data.There is some limited scheduling functionality based on
the NT scheduler service. Where you want to run one plan
following another, you may need to get a bit clever, by
including some preprocessing to check that the previous
plan has completed or use a third party scheduler which
would not start the plan until the previous plan had
completed.
3. What is the extend of errors handling in loading operation,
supported by this suite. Pretty good. All the stuff from SQL Loader, plus each of the
transforms has it's own error handling.
===
Manish Jain (guest) wrote:
: Hello,
: I need to know which is a better way of loading flat
: files data into
: Warehouse tables. SQL*Loader or Oracle DataMart Suite.
: 1. What all types of databases are supported by DataMart ODBC.
: 2. Can DataMart suite be programmed for auto operation of
: loading data.
: 3. What is the extend of errors handling in loading operation,
: supported by this suite.
: regards,
: Manish.
Oracle Technology Network
http://technet.oracle.com
null

Similar Messages

How to delete the data loaded into MySQL target table using Scripts

Hi Experts
I created a Job with a validation transformation. If the Validation was failed the data passed the validation will be loaded into Pass table and the data failed will be loaded into failed table.
My requirement was if the data was loaded into Failed database table then i have to delete the data loaded into the Passed table using Script.
But in the script i have written the code as
sql('database','delete from <tablename>');
but as it is an SQL Query execution it is rising exception for the query.
How can i delete the data loaded into MySQL Target table using scripts.
Please guide me for this error
Thanks in Advance
PrasannaKumar

Hi Dirk Venken
I got the Solution, the mistake i did was the query is not correct regarding MySQL.
sql('MySQL', 'truncate world.customer_salesfact_details')
error query
sql('MySQL', 'delete table world.customer_salesfact_details')
Thanks for your concern
PrasannaKumar

Transaction data can be loaded into the Fact table without loading the

Transaction data can be loaded into the Fact table without loading the corresponding master data (Example : Sales analysis transaction data can be loaded without populating any of its dimensions master data)
a. True
b. False

Hi Kutti,
True - You need to select the option in the infopackage - alwyas load even if no master data exists.
Bye
Dinesh

Adding leading zeros before data loaded into DSO

Hi
In below PROD_ID... In some ID leading zeros are missing before data loaded into BI from SRM into PROD_ID. Data type is character. If leading zeros are missing then data activation of DSO is failed due to missing zeros and have to manually add them in PSA table. I want to add leading zeros if they're missing before data loaded into DSO.... total character length is 40.. so e.g. if character is 1502 then there should be 36 zeros before it and if character is 265721 then there should be 34 zeros. Only two type of character is coming either length is 4 or 6 so there will be always need to 34 or 36 zeros in front of them if zeros are missing.
Can we use CONVERSION_EXIT_ALPHPA_INPUT functional module ? As this is char so I'm not sure how to use in that case.. Do need to convert it first integer?
Can someone please give me sample code? We're using BW 3.5 data flow to load data into DSO.... please give sample code and where need to write code either in rule type or in start routine...

Hi,
Can you check at info object level, what kind of conversion routine it used by.
Use T code - RSD1, enter your info object and display it.
Even at data source level also you can see external/internal format what it maintained.
if your info object was using ALPHA conversion then it will have leading 0s automatically.
Can you check from source how its coming, check at RSA3.
if your receiving this issue for records only then you need to check those records.
Thanks

Data load into SAP ECC from Non SAP system

Hi Experts,
I am very new to BODS and I have want to load historical data from non SAP source system into SAP R/3 tables like VBAK,VBAP using BODS, Can you please provide steps/documents or guidelines on how to achieve this.
Regards,
Monil

Hi
In order to load into SAP you have the following options
1. Use IDocs. There are several standard IDocs in ECC for specific objects (MATMAS for materials, DEBMAS for customers, etc., ) You can generate and send IDocs as messages to the SAP Target using BODS.
2. Use LSMW programs to load into SAP Target. These programs will require input files generated in specific layouts generated using BODS.
3. Direct Input - The direct input method is to write ABAP programs targetting on specific tables. This approach is very complex and hence a lot of thought process needs to be applied.
The OSS Notes supplied in previous messages are all excellent guidance to steer you in the right direction on the choice of load, etc.,
However, the data load into SAP needs to be object specific. So targetting merely the sales tables will not help as the sales document data held in VBAK and VBAP tables you mentioned are related to Articles. These tables will hold sales document data for already created articles. So if you want to specifically target these tables, then you may need to prepare an LSMW program for the purpose.
To answer your question on whether it is possible to load objects like Materials, customers, vendors etc using BODS, it is yes you can.
Below is a standard list of IDocs that you can use for this purpose to load into SAP ECC system from a non SAP system.
Customer Master - DEBMAS
Article Master - ARTMAS
Material Master - MATMAS
Vendor Master - CREMAS
Purchase Info Records (PIR) - INFREC
The list is endless.........
In order to achieve this, you will need to get the functional design consultants to provide ETL mapping for the legacy data to IDoc target schema and fields (better to ahve sa tech table names and fields too). You should then prepare the data after putting it through the standard check table validations for each object along with any business specific conversion rules and validations applied. Having prepared this data, you can either generate flat file output for load into SAP using LSMW programs or generate IDoc messages to the target SAPsystem.
If you are going to post IDocs directly into SAP target using BODS, you will need to create a partner profile for BODS to send IDocs and define the IDocs you need as inbound IDocs. There are few more setings like RFC connectivity, authorizations etc, in order for BODS to successfully send IDocs into the SAP Target.
Do let me know if you need more info on any specific queries or issues you may encounter.
kind regards
Raghu

*HOW TO INSERT DATA MANUALLY INTO A BW TABLE*

Dear experts,
I'm working in BW 3.5 version.
Since I need to test some tables which are going to be load manually, please could anyone explain me which are the steps to insert data manually into a BW table.
Thank you very much in advance,
Jorge

Hi Jorge,
You can maintain the TMG(Table maintenance generator) and then enter the data manually. TMG creation Tcode is se55. and to view and maintain TMG it is sm30 .
Or if you have the data in excel. You can write a simple excel uploading ABAP program which will load your excel data to the table .
Hope the above reply was helpful.
Thanks & Regards,
Ashutosh Singh
Edited by: Ashutosh Singh on Apr 30, 2011 6:45 AM

How can i add the dimensions and data loading into planning apllications?

Now please let me know how can i add the dimensions and data loading into planning apllication without manuallly?

you can use tools like ODI or DIM or HAL to load metadata & data into planning applications.
The data load can be done at the Essbase end using rules file. But metadata changes should flow from planning to essbase through any of above mentioned tools and also there are many other way to achieve the same.
- Krish

Using external parameters within MS Query returning that data right into a pivot table Excel 2010

In Excel 2010 i was able to use the external parameters within MS Query returning that data right into a pivot table. But the parameter on the worksheet will not save in the paremeter setting of the data connection. And if you save it and open
it and try to refresh the pivot table Excel stops responding and you have to force the file closed. I found out the reason to crash is that it did not keep the parameter saved.
Can this be fixed?

Hi,
Just
checking in to see if the information of Oskar was helpful. Please
let us know if you would like further assistance.
Jaynet Zhang
TechNet Community Support

Data loads into multiple InfoObjects from 0EHS_PHRASE_TEXT DataSource.

Dear Experts,
I am working on SAP HCM-BW 7.0 Implementation and am trying to load data into 8 different InfoObjects (Texts) through 0EHS_PHRASE_TEXT (Phrases) extractor.
There are many InfoPackages in place created during the previous project loading into different set of Standard BCT InfoObjects.
After migrating the 0EHS_PHRASE_TEXT DataSource from 3.x to 7.0 version, I have created different InfoObjects with Transformations and DTPs upon business requirements to load them. The relevant data is available in 0EHS_PHRASE_TEXT when checked in Extractor Checker (RSA3). However, when I create InfoPackages to load these different InfoObjects, I don't see the Data Targets listed and it makes sense since the data is first loaded into PSA and then will be loaded into Data Targets (in this case InfoObjects with Text) using DTPs.
My concern is... when I create a Process Chain to load Master Data into these Objects, the following action is happening:
1. Load 10 Records into PSA and then into InfoObject A. All 10 Records are loaded into A.
2. Load 5 Records into PSA and then into InfoObject B. Previous 10 + New 5 Records are loaded into B.
3. and it continues... for 8 InfoObjects.
My question is... is there are way that I can see the corresponding Data Target tab in InfoPackages so the Data is immediately picked from PSA of 0EHS_PHRASE_TEXT DataSource and then loaded into the corresponding InfoObjects.
Or should I make use of Transfer Rules by Re-storing the 0EHS_PHRASE_TEXT DataSource from 7.0 to 3.x version...!!
Your help is much appreciated.
Thanks,
Chandu

Hi Andreas
Thanks for replying and sorry for the confusion.
The extractor is delivering 10 and then 5 because of different selection parameters in the InfoPackage "selecting" which InfoObject I wish to load the texts for.
Your understanding of my requirements is correct.
Could you please elaborate Option 1 bit more..!
Regarding your Option 2, I don't want the DTPs to extract directly in full from the datasource because by doing so, all the data apart from the relevant data is also loaded into each and every InfoObject. And I don't want that to happen. I want to load only relevant data into respective InfoObjects from the PSA using DTPs.
I have tried to setup filters in DTPs with different Selection Parameters and tried to load relevant InfoObjects. For example, I am trying to load ZEHS_SUBS InfoObject with EHS_INJ_SUB_SUBSTANCE as the Selection Parameter. There are 54 records in both Source System and PSA for this parameter.
And the request is showing 54 in Transferred Records but only the very last 1 in Added Records. When I check in the Target (ZEHS_SUBS) InfoObject, it is showing only 1 Record. I have tried many combinations. They are:
1. Loaded only EHS_INJ_SUB_SUBSTANCE data from Source System into PSA and then tried to execute the relevant DTP.
2. Checked the 'Do Not Extract from PSA but Access Data Source (for Small Amounts of Data) and tried it.
3. Tried with both Delta and Full Extraction Mode.
4. Set Filter on Single Value as 'EHS_INJ_SUB_SUBSTANCE' to extract this data only.
5. Set Filter Excluding all the other Single Values.
6. Checked the 'Handle Duplicate Record Keys'.
Still only the last 1 records out of 54 is showing up in the Target InfoObject.
Please let me know if you have any idea as to why this is happening so.
Thanks for your time.
Chandu

Aggregating data loaded into different hierarchy levels

I have some problems when i try to aggregate a variable called PRUEBA2_IMPORTE dimensinated by time dimension (parent-child type).
I read the help in DML Reference of the OLAP Worksheet and it said the follow:
When data is loaded into dimension values that are at different levels of a hierarchy, then you need to be careful in how you set status in the PRECOMPUTE clause in a RELATION statement in your aggregation specification. Suppose that a time dimension has a hierarchy with three levels: months aggregate into quarters, and quarters aggregate into years. Some data is loaded into month dimension values, while other data is loaded into quarter dimension values. For example, Q1 is the parent of January, February, and March. Data for March is loaded into the March dimension value. But the sum of data for January and February is loaded directly into the Q1 dimension value. In fact, the January and February dimension values contain NA values instead of data. Your goal is to add the data in March to the data in Q1. When you attempt to aggregate January, February, and March into Q1, the data in March will simply replace the data in Q1. When this happens, Q1 will only contain the March data instead of the sum of January, February, and March. To aggregate data that is loaded into different levels of a hierarchy, create a valueset for only those dimension values that contain data. DEFINE all_but_q4 VALUESET time
LIMIT all_but_q4 TO ALL
LIMIT all_but_q4 REMOVE 'Q4'
Within the aggregation specification, use that valueset to specify that the detail-level data should be added to the data that already exists in its parent, Q1, as shown in the following statement. RELATION time.r PRECOMPUTE (all_but_q4)
How to do it this for more than one dimension?
Above i wrote my case of study:
DEFINE T_TIME DIMENSION TEXT
T_TIME
200401
200402
200403
200404
200405
200406
200407
200408
200409
200410
200411
2004
200412
200501
200502
200503
200504
200505
200506
200507
200508
200509
200510
200511
2005
200512
DEFINE T_TIME_PARENTREL RELATION T_TIME <T_TIME T_TIME_HIERLIST>
-----------T_TIME_HIERLIST-------------
T_TIME H_TIME
200401 2004
200402 2004
200403 2004
200404 2004
200405 2004
200406 2004
200407 2004
200408 2004
200409 2004
200410 2004
200411 2004
2004 NA
200412 2004
200501 2005
200502 2005
200503 2005
200504 2005
200505 2005
200506 2005
200507 2005
200508 2005
200509 2005
200510 2005
200511 2005
2005 NA
200512 2005
DEFINE PRUEBA2_IMPORTE FORMULA DECIMAL <T_TIME>
EQ -
aggregate(this_aw!PRUEBA2_IMPORTE_STORED using this_aw!OBJ262568349 -
COUNTVAR this_aw!PRUEBA2_IMPORTE_COUNTVAR)
T_TIME PRUEBA2_IMPORTE
200401 NA
200402 NA
200403 2,00
200404 2,00
200405 NA
200406 NA
200407 NA
200408 NA
200409 NA
200410 NA
200411 NA
2004 4,00 ---> here its right!! but...
200412 NA
200501 5,00
200502 15,00
200503 NA
200504 NA
200505 NA
200506 NA
200507 NA
200508 NA
200509 NA
200510 NA
200511 NA
2005 10,00 ---> here must be 30,00 not 10,00
200512 NA
DEFINE PRUEBA2_IMPORTE_STORED VARIABLE DECIMAL <T_TIME>
T_TIME PRUEBA2_IMPORTE_STORED
200401 NA
200402 NA
200403 NA
200404 NA
200405 NA
200406 NA
200407 NA
200408 NA
200409 NA
200410 NA
200411 NA
2004 NA
200412 NA
200501 5,00
200502 15,00
200503 NA
200504 NA
200505 NA
200506 NA
200507 NA
200508 NA
200509 NA
200510 NA
200511 NA
2005 10,00
200512 NA
DEFINE OBJ262568349 AGGMAP
AGGMAP
RELATION this_aw!T_TIME_PARENTREL(this_aw!T_TIME_AGGRHIER_VSET1) PRECOMPUTE(this_aw!T_TIME_AGGRDIM_VSET1) OPERATOR SUM -
args DIVIDEBYZERO YES DECIMALOVERFLOW YES NASKIP YES
AGGINDEX NO
CACHE NONE
END
DEFINE T_TIME_AGGRHIER_VSET1 VALUESET T_TIME_HIERLIST
T_TIME_AGGRHIER_VSET1 = (H_TIME)
DEFINE T_TIME_AGGRDIM_VSET1 VALUESET T_TIME
T_TIME_AGGRDIM_VSET1 = (2005)
Regards,
Mel.

Mel,
There are several different types of "data loaded into different hierarchy levels" and the aproach to solving the issue is different depending on the needs of the application.
1. Data is loaded symmetrically at uniform mixed levels. Example would include loading data at "quarter" in historical years, but at "month" in the current year, it does /not/ include data loaded at both quarter and month within the same calendar period.
= solved by the setting of status, or in 10.2 or later with the load_status clause of the aggmap.
2. Data is loaded at both a detail level and it's ancestor, as in your example case.
= the aggregate command overwrites aggregate values based on the values of the children, this is the only repeatable thing that it can do. The recomended way to solve this problem is to create 'self' nodes in the hierarchy representing the data loaded at the aggregate level, which is then added as one of the children of the aggregate node. This enables repeatable calculation as well as auditability of the resultant value.
Also note the difference in behavior between the aggregate command and the aggregate function. In your example the aggregate function looks at '2005', finds a value and returns it for a result of 10, the aggregate command would recalculate based on january and february for a result of 20.
To solve your usage case I would suggest a hierarchy that looks more like this:
DEFINE T_TIME_PARENTREL RELATION T_TIME <T_TIME T_TIME_HIERLIST>
-----------T_TIME_HIERLIST-------------
T_TIME H_TIME
200401 2004
200402 2004
200403 2004
200404 2004
200405 2004
200406 2004
200407 2004
200408 2004
200409 2004
200410 2004
200411 2004
200412 2004
2004_SELF 2004
2004 NA
200501 2005
200502 2005
200503 2005
200504 2005
200505 2005
200506 2005
200507 2005
200508 2005
200509 2005
200510 2005
200511 2005
200512 2005
2005_SELF 2005
2005 NA
Resulting in the following cube:
T_TIME PRUEBA2_IMPORTE
200401 NA
200402 NA
200403 2,00
200404 2,00
200405 NA
200406 NA
200407 NA
200408 NA
200409 NA
200410 NA
200411 NA
200412 NA
2004_SELF NA
2004 4,00
200501 5,00
200502 15,00
200503 NA
200504 NA
200505 NA
200506 NA
200507 NA
200508 NA
200509 NA
200510 NA
200511 NA
200512 NA
2005_SELF 10,00
2005 30,00
3. Data is loaded at a level based upon another dimension; for example product being loaded at 'UPC' in EMEA, but at 'BRAND' in APAC.
= this can currently only be solved by issuing multiple aggregate commands to aggregate the different regions with different input status, which unfortunately means that it is not compatable with compressed composites. We will likely add better support for this case in future releases.
4. Data is loaded at both an aggregate level and a detail level, but the calculation is more complicated than a simple SUM operator.
= often requires the use of ALLOCATE in order to push the data to the leaves in order to correctly calculate the aggregate values during aggregation.

How to make data loaded into cube NOT ready for reporting

Hi Gurus: Is there a way by which data loaded into cube, can be made NOT available for reporting.
Please suggest. <removed>
Thanks

See, by default a request that has been loaded to a cube will be available for reporting. Bow if you have an aggregate, the system needs this new request to be rolled up to the aggregate as well, before it is available for reporting...reason? Becasue we just write queries for the cube, and not for the aggregate, so you only know if a query will hit a particular aggregate at its runtime. Which means that if a query gets data from the aggregate or the cube, it should ultimately get the same data in both the cases. Now if a request is added to the cube, but not to the aggregate, then there will be different data in both these objects. The system takes the safer route of not making the 'unrolled' up data visible at all, rather than having inconsistent data.
Hope this helps...

AWM Newbie Question: How to filter data loaded into cubes/dimensions?

Hi,
I am trying to filter the amount of data loaded into my dimensions in AWM (e.g., I only want to load like 1-2 years worth of data for development purposes). I can't seem to find a place in AWM where you can specify a WHERE clause...is there something else I must do to filter data?
Thanks

Hi there,
Which release of Oracle OLAP are you using? 10g? 11g?
You can use database views to filter your dimension and cube data and then map these in AWM
Thanks,
Stuart Bunby
OLAP Blog: http://oracleOLAP.blogspot.com
OLAP Wiki: http://wiki.oracle.com/page/Oracle+OLAP+Option
OLAP on OTN: http://www.oracle.com/technology/products/bi/olap/index.html
DW on OTN : http://www.oracle.com/technology/products/bi/db/11g/index.html

Incremental load into the Dimension table

Hi,
I have the problem in doing the incremental load of the dimension table.Before loading into the dimension table,i would like to check the data in the dimnesion table.
In my dimension table i have one not null surrogate key and the other null dimension tables.The not null surrogate key, i am populating with the Sequence Generator.
To do the incremental load i have done the following.
I made lookup into the dimension table and looked for a key.The key from the lookup table i have passed to the expression operator.In the expression operator i have created one field and hard coded one flag based on the key from the lookup table.I passed this flag to the filter operator and rest of the fields from the source.
By doing this i am not able to pass the new records to the dimension table.
Can you please help me.
I have another question also.
How do i update one not null key in the fact table.
Thanks
Vinay

Hi Mark,
Thanks for your help to solve my problem.I thought i share more information by giving the sql.
I am giving below the 2 sqls, i would like to achieve through OWB.
Both the following tasks need to be accomplished after loading the fact table.
task1:
UPDATE fact_table c
SET c.dimension_table_key =
(SELECT nvl(dimension_table.dimension_table_key,0)
FROM src_dimension_table t,
dimension_table dimension_table
WHERE c.ssn = t.ssn(+)
AND c.date_src_key = to_number(t.date_src(+), '99999999')
AND c.time_src_key = to_number(substr(t.time_src(+), 1, 4), '99999999')
AND c.wk_src = to_number(concat(t.wk_src_year(+), concat(t.wk_src_month(+), t.wk_src_day(+))), '99999999')
AND nvl(t.field1, 'Y') = nvl(dimension_table.field1, 'Y')
AND nvl(t.field2, 'Y') = nvl(dimension_table.field2, 'Y')
AND nvl(t.field3, 'Y') = nvl(dimension_table.field3, 'Y')
AND nvl(t.field4, 'Y') = nvl(dimension_table.field4, 'Y')
AND nvl(t.field5, 'Y') = nvl(dimension_table.field5, 'Y')
AND nvl(t.field6, 'Y') = nvl(dimension_table.field6, 'Y')
AND nvl(t.field7, 'Y') = nvl(dimension_table.field7, 'Y')
AND nvl(t.field8, 'Y') = nvl(dimension_table.field8, 'Y')
AND nvl(t.field9, 'Y') = nvl(dimension_table.field9, 'Y')
WHERE c.dimension_table_key = 0;
fact table in the above sql is fact_table
dimesion table in the above sql is dimension_table
source table for the dimension table is src_dimension_table
dimension_table_key is a not null key in the fact table
task2:
update fact_table cf
set cf.key_1 =
(select nvl(max(p.key_1),0) from dimension_table p
where p.field1 = cf.field1
and p.source='YY')
where cf.key_1 = 0;
fact table in the above sql is fact_table
dimesion table in the above sql is dimension_table
key_1 is a not null key in the fact table
Is it possible to achieve the above tasks through Oracle Warehouse builder(OWB).I created the mappings for loading the dimension table and fact table and they are working fine.But the above two queries i am not able to achieve through OWB.I would be thankful if you can help me out.
Thanks
Vinay

Latest PowerQuery issues with data load to data models built with older version + issue when query is changed

We have a tool built in excel + Powerquery version 2.18.3874.242 - 32 Bit (No PowerPivot) using data load to data model (not to workbook). There are data filters linked to excel cells, inserted in OData query before data is pulled.
The Excel tool uses organisational credentials to authenticate.
System config: Win 8.1, Office 2013 (32 bit)
The tool runs for all users as long as they do not upgrade to PowerQuery_2.20.3945.242 (32-bit).
Once upgraded users can no longer get the data to load to the Model. Data still loads to the Workbook but the model breaks down. Resetting load to data model erases all measures.
Here are the exact errors users get:
1. [DataSource.Error] Cannot parse OData response result. Error: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
2. The Data Model table could not be refreshed: There isn't enough memory to complete this action. Try using less data or closing other applications. To increase memory available, consider ......

Hi Nitin,
Is this still an issue? If so, can you kindly provide the details that Hadeel has asked for?
Regards,
Michael Amadi
Please use the 'Mark as answer' link to mark a post that answers your question. If you find a reply helpful, please remember to vote it as helpful :)
Website: http://www.nimblelearn.com, Twitter:
@nimblelearn

How to rectify the errors in master data loads & transactional data loads?

hy,
please any one tell me
How to rectify the errors in master data loads & transactional data loads?
thnQ
Ravi

Hi,
Please post specific questions in the forum.
Please explain the error you are getting.
-Vikram

Data loading into Data warehouse Tables

Similar Messages

Maybe you are looking for