Incremental Data Load

Hi, I am trying to load data to a 6.5 Application. In the file I have a number of lines of data with the exact same members but different data. We have to convert products that used to roll up to a total to only one member. For example ABC Total ABC DEF GHI Now all the data in DFE and GHI will need to be accumulated in ABC. I tried the "Add to Existing Values" in the data load rule, but that seems to only work with data that is there before the load. Any thoughts? I really appreciate it.

Never Mind! I am sorry, I had a mistake in my load rule. I tried to delete the prior post but could not figure out how. Very Sorry.

Similar Messages

Incremental Data loading in ASO 7.1

HI,
As per the 7.1 essbase dbag
"Data values are cleared each time the outline is changed structurally. Therefore, incremental data loads are supported
only for outlines that do not change (for example, logistics analysis applications)."
That means we can have the incremental loading for ASO in 7.1 for the outline which doesn't change structurally. Now what does it mean by the outline which changes structurally? If we add a level 0 member in any dimension, does it mean structrual change to that outline?
It also syas that adding Accounts/Time member doesn't clear out the data. Only adding/deleting/moving standard dimension member will clear out the data. I'm totally confused here. Can anyone pls explain me?
The following actions cause Analytic Services to restructure the outline and clear all data:
● Add, delete, or move a standard dimension member
● Add, delete, or move a standard dimension
● Add, delete, or move an attribute dimension
● Add a formula to a level 0 member
● Delete a formula from a level 0 member
Edited by: user3934567 on Jan 14, 2009 10:47 PM

Adding a Level 0 member is generally, if not always, considered to be a structural change to the outline. I'm not sure if I've tried to add a member to Accounts and see if the data is retained. This may be true because by definition, the Accounts dimension in an ASO cube is a dynamic (versus Stored) hierarchy. And perhaps since the Time dimension in ASO databases in 7.x is the "compression" dimension, there is some sort of special rule about being able to add to it -- although I can't say that I ever need to edit the Time dimension (I have a separate Years dimension). I have been able to modify formulas on ASO outlines without losing the data -- which seems consistent with your bullet points below. I have also been able to move around and change Attribute dimension members (which I would guess is generally considered a non-structural change), and change aliases without losing all my data.
In general I just assume that I'm going to lose my ASO data. However, all of my ASO outlines are generated through EIS and I load to a test server first. If you're in doubt about losing the data -- try it in test/dev. And if you don't have test/dev, maybe that should be a priority. :) Hope this helps -- Jason.

Incremental Data load in SSM 7.0

Hello all,
I once raised a thread in SDN which says how to automate data load into SSM 7.0.
Periodic data load for a model in SSM
Now my new requirement is not to upload the whole data again , but only the new data (data after the previous data load) . Is there a way to do the incremental data load in SSM 7.0 ? Loading the whole of the fact data again and again will take a hit on the performance of the SSM system. Is there a work around in case there is no solution ?
Thanks
Vijay

Vijay,
In your PAS model you can build a procedure to remove data and then load that data to the correct time period.
In PAS, to remove data but not the variable definitions from the database:
Removing data for a particular variable
REMOVE DATA SALES
or if there were particular areas only within
SELECT product P1
SELECT customer C1
REMOVE SELECTED SALES
or remove all data
REMOVE DATA * SURE
or just a time period
REMOVE DATA SALES BEFORE Jan 2008
Then you would construct or modify your Load Procedure to load the data for the new period
SET PERIOD {date range}
Regards.
Bpb
Then would

How to use incremental data load in OWB? can CDC be used?

hi,
i am using oracle 10g relese 2 and OWB 10g relese 1
i want know how can i implement incremental data load in OWB?
is it having such implicit feature in OWB tool like informatica?
can i use CDC concept for this/ is it viable and compatible with my envoirnment?
what could be other possible ways?

Hi ,
As such the current version of OWB does not provide the functionality to directly use CDC feature available. You have to come up with your own strategy for incremental loading. Like, try to use the Update Dates if available on your source systems or use CDC packages to pick the changed data from your source systems.
rgds
mahesh

Essbase 7.1 - Incremental data load in ASO

Hi,
Is there incremental data loading feature in ASO version 7.1? Let's say, I've the following data in ASO cube
P1 G1 A1 100
Now, I get the following 2 rows as per the incremental data from relational source:
P1 G1 A1 200
P2 G1 A1 300
So, once I load these rows using rule file with override existing values option, will I've the following dataset in ASO:
P1 G1 A1 200
P2 G1 A1 300
I know there is data load buffer concept in ASO 7.1. And this is the inly way to improve data load performance. But just wanted to check if we can implement incremental loading in ASO or not.
And one more thing, Can 2 load rules run in parallel to load data in ASO cubes? As per my understanding, when we start loading data, the cube is locked for any other insert/update. Pls correct me if I'm wrong!
Thanks!

Hi,
I think the features such as incremental data loads were available from version 9.3.1
In the whats new for Essbase 9.3.1 it contains
Incrementally Loading Data into Aggregate Storage Databases
The aggregate storage database model has been enhanced with the following features:
l An aggregate storage database can contain multiple slices of data.
l Incremental data loads complete in a length of time that is proportional to the size of the
incremental data.
l You can merge all incremental data slices into the main database slice or merge all
incremental data slices into a single data slice while leaving the main database slice
unchanged.
l Multiple data load buffers can exist on a single aggregate storage database. To save time, you
can load data into multiple data load buffers at the same time.
l You can atomically replace the contents of a database or the contents of all incremental data
slices.
l You can control the share of resources that a load buffer is allowed to use and set properties
that determine how missing and zero values, and duplicate values, in the data sources are
processed.
Cheers
John
http://john-goodwin.blogspot.com/

Incremental data loading

I've come across this question:
Identify the two true statements about incremental loading.
A. Allows for real time data access for end users.
B. Creates subscribes along the main slice in the database.
C. Materialization of slices is required to provide users the correct query results.
D. Different materialized views may exist within a slice as compared to the main slice of the database.
Can anyone state which of these two are correct and why so?
Thanks

Im not sure, but wish you good luck with your examination regardless.
JB

Urgent help require-ASO data loading

I am working on 9.3.0 and want to do incremental data loading so that it wont affect my past data.I am still not sure that it is doable or not .Now the question is
do I need to design a new aggragation after loading data.
Thanks in advance

Hi,
The ASO cube will clear off all the data , if you make any structural changes to the cube ( i.e if you change your outline ,and you can find out what exactly can clear off hte data ,for example , if you add a new member ,then ,it clears off , and in other case ,if you just add a simple formula to the existing member , it might not clear off the data ).
If you dont want to effect the past data ,and yet you want to incrementally load , then ensure that all the members are already in the otl( take care of time dimension , let all the time members be there in place) and then you can load in ,using the option of 'add to existing values' while you load .
But remember , you can only do ,without any structural changes. Else , you need to load all together again
Its good if you design aggregation , as it helps in retrieval performance, but its not mandatory.
Sandeep Reddy Enti
HCC

Incrementally Loading Data Using a Data Load Buffer

Hi
I am using Essbase 9.3.1. I am trying to use the feature "Replacing Database or Incremental Data Slice Contents" for my data loads to the ASO Cube.
I have 2 data sets, one of them is 2 years history. And another is last 3 months which would be changing on a daily basis. I looked at that DBAG and they have exact same scenario as an example for this feature. But I am not able to overwrite my valatile data set with my new file.
Here is what I do
alter database ${1}.${2} initialize load_buffer with buffer_id ${6} resource_usage 0.3 property ignore_missing_values, ignore_zero_values ;
import database ${1}.${2} data from data_file '${3}' using server rules_file '${4}' to load_buffer with buffer_id ${6} add values create slice on error write to '${5}' ;
alter database {1}.{2} merge incremental data;
alter database ${1}.${2} clear aggregates ;
execute aggregate process on database ${1}.${2} ;
In fact my data from my new (incremental file) does not even make it to the database. I checked that it does get rejected.
Am I doing something wrong over here. How do I use the concept of "data slice" and its incremental load feature.
Can anyone please explain ?
Thanks
Mandar Joshi

Hi,
Just wondering if anyone had any inputs or feedback on my query. Or is my question a really stupid one and does not deserve any attention :)
Can someone explain how the "data slice" concept works ??
Thanks
Mandar Joshi.

Data Load MAXLs in ASO

Hi All,
Greetings of the day !!!!
Want to understand the difference between "Add values create slice" and "override values create slice" used in data loading MAXLs
Suppose we initialized buffer and loaded data in buffer then we can use following two MAXLs
1)
import database AsoSamp.Sample data
from load_buffer with buffer_id 1
add values create slice;
2)
import database AsoSamp.Sample data
from load_buffer with buffer_id 1
override values create slice;
Q1
What i am thinking logically is if i am again loading the data in the same intersections from which slice is created ADD VALUE will add it and override value will overwrite it .... e.g if 100 was present earlier and we are again loading 200 then ADD will make 300 and overwrite will result 200. Let me know if my understanding is correct
Q2
Why do we use "create slice" ? What is the use? Is it for better performance for data loading? Is it compulsary to merge the slices after dataloading??
Cant we just use add value or override values if we dont want to create slice...
Q3
I saw two MAXLs for merging also ... one was Merge ALL DATA and other was MERGE incremental data ... Whats the diff ? In which case we use what?
Pls help me in resolving my doubts... Thanks a lot !!!!

Q1 - Your understanding is correct. The buffer commit specification determines how what is in the buffer is applied to what is already in the cube. Note that there are also buffer initialization specifications for 'sum' and 'use last' that apply only to data loaded to the buffer.
Q2 - Load performance. Loading data to an ASO cube without 'create slice' takes time (per the DBAG) proportional to the amount of data already in the cube. So loading one value to a 100GB cube may take a very long time. Loading data to an ASO cube with 'create slice' takes time proportional to the amount of data being loaded - much faster in my example. There is no requirement to immediately merge slices, but it will have to be done to design / process aggregations or restructure the cube (in the case of restructure, it happens automatically IIRC). The extra slices are like extra cubes, so when you query Essbase now has to look at both the main cube and the slice. There is a statistic that tells you how much time Essbase spends querying slices vs querying the main cube, but no real guidance on what a 'good' or 'bad' number is! See http://docs.oracle.com/cd/E17236_01/epm.1112/esb_tech_ref/aggstor_runtime_stats.html.
The other reason you might want to create a slice is that it's possible to overwrite (or even remove, by committing an empty buffer with the 'override incremental data' clause in the buffer commit specification) only the slice data without having to do physical or logical clears. So if you are continually updating current period data, for example, it might make sense to load that data to an incremental slice.
Q3 - You can merge the incremental slices into the rest of the cube, or you can merge multiple incremental slices into one single incremental slice, but not into the rest of the cube. Honestly, I've only ever wanted to use the first option. I'm not really sure when or why you would want to do the second, although I'm sure it's in there for a reason.

ODI data loading Speed

Hi All,
ODI data loading step's speed is too low.
I am using;
LKM=LKM SQL to Oracle,
CKM=CKM SQL,
IKM=IKM SQL Incremental Update for replication SQL Server to Oracle.
I don't use Flow control in interfaces.
SQL Server and Oracle database are installed on the same server.
How can I do it faster?

If the two database servers are on the same machine, and you are dealing with bulk data, you should use an LKM which uses bulk methods (BCP to extract the data from SQL Server and External tables to get the data into Oracle) - something like this KM https://s3.amazonaws.com/Ora/KM_MSSQL2ORACLE.zip (which is actually an IKM but the LKM is not so different.
Hope it help

Incremental Cube Load Error: job slave process terminated

Hi,
For performance reasons, we switched to Incremental Cube Loading i.e. only those partitions are autosolved whose data is made available.
Some times, the background submitted job terminates and the reason given in dba_scheduler_job_run_details is:
REASON="Job slave process was terminated"
There so no definits occurance pattren for this error.
The job submitted in background is killed.
The last entry the xml_load_log displayed is of Started Auto solving of a partition.
After this error occurs, we have to Full Aggregate the cube; which offcourse would autosolve all partitions.
We have been too much annoyed by this error as we did lot of package changes as part of a release to production to include Incremental cube loading, and once done, we see that incremental cube loading just terminates while autosolving a partitions.
Can any one assist please? Urgent?
thank you,

Hi,
There is a metalink note about this issue. Check note 443348.1
Thanks
Brijesh

How to Run ebs adaptor data load for 1 year using DAC

Hi,
iam trying to Run the ebs adaptor data load for 1 year for procurement and spend ,Please let me know the parameter to set in DAC.
last extract date is set as Custom Format(@DAC_SOURCE_PRUNED_REFRESH_TIMESTAMP
Thanks

You need to set $$INITIAL_EXTRACT_DATE to a year ago. The LAST EXTRACT DATE is something used for incremental loads and you do not manually set that.
if this helps, mark as correct or helpful.

Data load problem - BW and Source System on the same AS

Hi experts,
Im starting with BW (7.0) in a sandbox environment where BW and the source system are installed on the same server (same AS). The source system is the SRM (Supplier Relationship Management) 5.0.
BW is working on client 001 while SRM is on client 100 and I want to load data from the SRM into BW.
Ive configured the RFC connections and the BWREMOTE users with their corresponding profiles in both clients, added a SAP source system (named SRMCLNT100), installed SRM Business Content, replicated the data sources from this source system and everything worked fine.
Now I want to load data from SRM (client 100) into BW (client 001) using standard data sources and extractors. To do this, Ive created an InfoPackage in one standard metadata data source (with data, checked through RSA3 on client 100 source system). Ive started the data load process, but the monitor says that no Idocs arrived from the source system and keeps the status yellow forever.
Additional information:
BW Monitor Status:
Request still running
Diagnosis
No errors could be found. The current process has probably not finished yet.
System Response
The ALE inbox of the SAP BW is identical to the ALE outbox of the source system
and/or
the maximum wait time for this request has not yet run out
and/or
the batch job in the source system has not yet ended.
Current status
No Idocs arrived from the source system.
BW Monitor Details:
0 from 0 records
but there are 2 records on RSA3 for this data source
Overall status: Missing messages or warnings
- Requests (messages): Everything OK
o Data request arranged
o Confirmed with: OK
- Extraction (messages): Missing messages
o Missing message: Request received
o Missing message: Number of sent records
o Missing message: Selection completed
- Transfer (IDocs and TRFC): Missing messages or warnings
o Request IDoc: sent, not arrived ; Data passed to port OK
- Processing (data packet): No data
Transactional RFC (sm58):
Function Module: IDOC_INBOUND_ASYNCHRONOUS
Target System: SRMCLNT100
Date Time: 08.03.2006 14:55:56
Status text: No service for system SAPSRM, client 001 in Integration Directory
Transaction ID: C8C415C718DC440F1AAC064E
Host: srm
Program: SAPMSSY1
Client: 001
Rpts: 0000
System Log (sm21):
14:55:56 DIA 000 100 BWREMOTE D0 1 Transaction Canceled IDOC_ADAPTER 601 ( SAPSRM 001 )
Documentation for system log message D0 1 :
The transaction has been terminated. This may be caused by a termination message from the application (MESSAGE Axxx) or by an error detected by the SAP System due to which it makes no sense to proceed with the transaction. The actual reason for the termination is indicated by the T100 message and the parameters.
Additional documentation for message IDOC_ADAPTER 601 No service for system &1, client &2 in Integration Directory No documentation exists for message ID601
RFC Destinations (sm59):
Both RFC destinations look fine, with connection and authorization tests successful.
RFC Users (su01):
BW: BWREMOTE with profile S_BI-WHM_RFC (plus SAP_ALL and SAP_NEW temporarily)
Source System: BWREMOTE with profile S_BI-WX_RFCA (plus SAP_ALL and SAP_NEW temporarily)
Someone could help ?
Thanks,
Guilherme

Guilherme
I didn't see any reason why it's not bringing. Are you doing full extraction or Delta. If delta extraction please check the extractor is delta enabled or not. Some times this may cause problems.
Also check this weblog on data Load errors basic checks. it may help
/people/siegfried.szameitat/blog/2005/07/28/data-load-errors--basic-checks
Thanks
Sat

BI 7.0 data load issue: InfoPackage can only load data to PSA?

BI 7.0 backend extraction gurus,
We created a generic datasource on R3 and replicated it to our BI system, created an InfoSource, the Transformation from the datasource to the InfoSource, an ODS, the transformation from the InfoSource to the ODS.
After the transformation creation between the InfoSource and the ODS is done on this BI system, a new folder called "Data Transfer Process" is also created under this ODS in the InfoProvider view. In the Data Transfer Process, in the Extraction tab, picks 'Full' in the field Extraction Mode, in the Execute tab, there is a button 'Execute', click this button (note: so far we have not created InfoPackage yet) which sounds like to conduct the data load, but find there is no data available even if all the status show green (we do have a couple of records in the R3 table).
Then we tried to create an InfoPackage, in the Processing tab, find 'Only PSA' radio button is checked and all others like 'PSA and then into Data Targets (Package by Package)" are dimmed! In the Data Target tab, find the ODS as a target can't be selected! Also there are some new columns in this tab, 'Maintain Old Update Rule' is marked with red color 'X', under another column 'DTP(S) are active and load to this target', there is an inactive picture icon, that's weird since we have already activated the Data Transfer Process! Anyway, we started the data load in the InfoPackage, and the monitor shows the records are brought in, but since in the Process tab in the InfoPackage, 'Only PSA' radio button is checked with all others dimmed that there is no any data going to this ODS! Why in BI 7.0, 'Only PSA' radio button can be checked with others all dimmed?
Many new features with BI 7.0! Any one's idea/experience is greatly appreciate on how to load data in BI 7.0!

You dont have to select anything..
Once loaded to PSA in DTP you have the option of FULL or DELTA ,full loads all the data from PSA and DELTA loads only the last load of PSA.
Go through the links for Lucid explainations
Infopackage -
http://help.sap.com/saphelp_nw2004s/helpdata/en/43/03808225cf5167e10000000a1553f6/content.htm
DTP
http://help.sap.com/saphelp_nw2004s/helpdata/en/42/f98e07cc483255e10000000a1553f7/frameset.htm
Creating DTP
http://help.sap.com/saphelp_nw2004s/helpdata/en/42/fa50e40f501a77e10000000a422035/content.htm
Pre-requisite-
You have used transformations to define the data flow between the source and target object.
Creating transformations-
http://help.sap.com/saphelp_nw2004s/helpdata/en/f8/7913426e48db2ce10000000a1550b0/content.htm
Hope it Helps
Chetan
@CP..

Open HUB ( SAP BW ) to SAP HANA through DB Connection data loading , Delete data from table option is not working Please help any one from this forum

Issue:
I have SAP BW system and SAP HANA System
SAP BW to SAP HANA connecting through a DB Connection (named HANA)
Whenever I created any Open Hub as Destination like DB Table with the help of DB Connection, table will be created at HANA Schema level ( L_F50800_D )
Executed the Open Hub service without checking DELETING Data from table option
Data loaded with 16 Records from BW to HANA same
Second time again executed from BW to HANA now 32 records came ( it is going to append )
Executed the Open Hub service with checking DELETING Data from table option
Now am getting short Dump DBIF_RSQL_TABLE_KNOWN getting
If checking in SAP BW system tio SAP BW system it is working fine ..
will this option supports through DB Connection or not ?
Please follow the attachemnet along with this discussion and help me to resolve how ?
From
Santhosh Kumar

Hi Ramanjaneyulu ,
First of all thanks for the reply ,
Here the issue is At OH level ( Definition Level - DESTINATION TAB and FIELD DEFINITION )
in that there is check box i have selected already that is what my issue even though selected also
not performing the deletion from target level .
SAP BW - to SAP HANA via DBC connection
1. first time from BW suppose 16 records - Dtp Executed -loaded up to HANA - 16 same
2. second time again executed from BW - now hana side appaended means 16+16 = 32
3. so that i used to select the check box at OH level like Deleting data from table
4. Now excuted the DTP it throws an Short Dump - DBIF_RSQL_TABLE_KNOWN
Now please tell me how to resolve this ? will this option is applicable for HANA mean to say like , deleting data from table option ...
Thanks
Santhosh Kumar

Incremental Data Load

Similar Messages

Maybe you are looking for