How to Handle Data Loads in BW

Hi friends,
we are almost on the verge of completing the development, and are planning to go live very soon, but before that we need to make sure the number of records getting loaded (from R/3 to BW)are in sync. If the user says that there are so many records for the Area currently, how do we handle this to happen in BW?? when the number of data packets keeps increasing how do we update the same to our targets,
If there are any steps or procedures to habdle the data loads , after and before GOLIVE , can someone please let me know(Data sizing, Loading, and archiving),
If the client tells that these many records are expected, how should we handle these situations??
Your input is really appreciated.
Thanks alot in advance,

Hi,
Best way to ensure that is after your data has been loaded to BW, run a report and verify the results. They should match per the logic or any filter criteria with the R3 area numbers.
In most cases the number of records might not match so doing a comparision of how many records were sent and how many got uploaded through the monitor will not be a good comparision.
For increasing data volumes go through the following thread on Performance tuning :
performance tuning
Data Reconciliation :
Reconciliation of a BW and R3 data
Re: Reg -- reconciliation process.
Cheers,
Kedar

Similar Messages

Model of how APEX handles data (values)?

Reference {thread:id=2486655}
I'm trying to get a model of how APEX handles data (values). There are several places(?) that data (values) can exist. Or so it seems. I'm trying to understand how these work -- to put all the pieces together.
Question:
A) Where data can be?
1) In the database
2) In the session
3) Rendered -- and hence displayed on the screen, if a displayed value -- but not in the session
4) In perhaps(?) some working memory pool(?) but different from the page rendered values I see on the screen?
5) Other?
Why do I think this is relevant?
Question:
B) If there are rendered values and session values different from the rendered values, then when code executes, which of these values (rendered or session) is it executing against?
And maybe the answer is, "You don't understand what's going on." Yes, "Exactly!" Hence the question.
Best wishes,
Howard

Howard (DBA in Training) wrote:
I'm putting this together now. For example:
I set the value of a Page Item (say, :P2_DATA) to "BEFORE" in a Before Region process and change it to "AFTER" in an After Region Process. 'BEFORE' was displayed on the screen but 'AFTER' was in the session state. So I see that PL/SQL changes these values in the session state. But it doesn't "go back" to change the HTML after the region has been rendered -- hence we see BEFORE shown on the screen. (I'll have to think about what a Dynamic Action does.)
Also, if a page item has a default value, say 'DEFAULT' then that value doesn't get into the session state. At least, not with my current settings. Oh, that explains a lot of confusion!! I suppose there's a logical reason for this but who would expect default values to be differently from other PL/SQL operations on the item?
Questions:
1) So why does a default value not get put into the session state?Several reasons, among them:
*1. When/why is the value required?* Region items (and thus their source and default values) are rendered in region/item sequence order. The item's source/default value would therefore not be available in session state until it's rendering point. This is frequently later in page show processing than the value is actually required, such as the common use case of including a data value in the page or region title. Setting the value in a Before Header/Regions computation or process makes it available as required before the item is actually rendered.
*2. The rendered page might not be submitted.* Default values are typically set when a page will create a new row. Consider the situation where the user clicks a "Create" button and is presented with an order entry form containing several default values. They then decide they don't want to create a new order after all, and navigate away from the form page via some mechanism that uses a redirect rather than a submit (such as a default "Cancel" button). This means that no page submit processing is performed, and generally that no session state management is performed. If the default values were set in session state, the order entry page would contain the default values for the non-existent order despite the user never submitting it. This may have unwanted side-effects if these values are referenced elsewhere in the application, and therefore would require extra session state management to be built into the application.
*3. A rendered item might not be submitted with the page.* Per the HTML specification, the values of disabled controls unchecked checkboxes are not POSTed on page submit. Consider a situation where a checkbox item is set with several default values, but the application requires that the checkboxes be cleared and disabled by a dynamic action if the user chooses certain options from a select list. In this case, on page submission no values are submitted into session state for the checkbox item. If the defaults had been set in session state, then the application would be in an inconsistent state, with the user-selected option indicating that none of the checkboxes were checked, but with session state holding values showing that they were. This would require additional processing to ensure that the application's state was consistent before saving it to the database.<sup>1</sup>
2 & 3 are particularly significant in basic wizard-generated applications. Doing things in this way, the wizard-generated DML and navigation "just works", without the need for additional session state management.
2) Is there a Page Item setting that causes the default value to be saved to the session state?No. Hopefully the explanations above provide sufficient reason to show why this is genearlly not desirable. On the rare occasions when it is required, set the session state value using a computation or process.
<sup>1</sup> There are other reasons—mainly security related—that make it good practice to perform such checks in more complex and internet-facing applications anyway.

How to handle SQL loader in stream,

Hi friends,
ANy experience to handle SQL*loader in stream? How to handel space and mamory for SQL*Loader Direct path loading?
we have a bi_direction stream. how to load huge data into stream without brokken stream system?
Thanks for your help!!
Jim

see also note
Note:222769.1
I have not use this parameter before in sqlldr.. Do need to do some research..
STREAMSIZE
Specifies the size, in bytes,      250 KB
New parameter introduced in for direct path streams.
Oracle 9i.

How to tune data loading time in BSO using 14 rules files ?

Hello there,
I'm using Hyperion-Essbase-Admin-Services v11.1.1.2 and the BSO Option.
In a nightly process using MAXL i load new data into one Essbase-cube.
In this nightly update process 14 account-members are updated by running 14 rules files one after another.
These rules files connect 14 times by sql-connection to the same oracle database and the same table.
I use this procedure because i cannot load 2 or more data fields using one rules file.
It takes a long time to load up 14 accounts one after other.
Now my Question: How can I minimise this data loading time ?
This is what I found on Oracle Homepage:
What's New
Oracle Essbase V.11.1.1 Release Highlights
Parallel SQL Data Loads- Supports up to 8 rules files via temporary load buffers.
In an Older Thread John said:
As it is version 11 why not use parallel sql loading, you can specify up to 8 load rules to load data in parallel.
Example:
import database AsoSamp.Sample data
connect as TBC identified by 'password'
using multiple rules_file 'rule1','rule2'
to load_buffer_block starting with buffer_id 100
on error write to "error.txt";
But this is for ASO Option only.
Can I use it in my MAXL also for BSO ?? Is there a sample ?
What else is possible to tune up nightly update time ??
Thanks in advance for every tip,
Zeljko

Thanks a lot for your support. I’m just a little confused.
I will use an example to illustrate my problem a bit more clearly.
This is the basic table, in my case a view, which is queried by all 14 rules files:
column1 --- column2 --- column3 --- column4 --- ... ---column n
dim 1 --- dim 2 --- dim 3 --- data1 --- data2 --- data3 --- ... --- data 14
Region -- ID --- Product --- sales --- cogs ---- discounts --- ... --- amount
West --- D1 --- Coffee --- 11001 --- 1,322 --- 10789 --- ... --- 548
West --- D2 --- Tea10 --- 12011 --- 1,325 --- 10548 --- ... --- 589
West --- S1 --- Tea10 --- 14115 --- 1,699 --- 10145 --- ... --- 852
West --- C3 --- Tea10 --- 21053 --- 1,588 --- 10998 --- ... --- 981
East ---- S2 --- Coffee --- 15563 --- 1,458 --- 10991 --- ... --- 876
East ---- D1 --- Tea10 --- 15894 --- 1,664 --- 11615 --- ... --- 156
East ---- D3 --- Coffee --- 19689 --- 1,989 --- 15615 --- ... --- 986
East ---- C1 --- Coffee --- 18897 --- 1,988 --- 11898 --- ... --- 256
East ---- C3 --- Tea10 --- 11699 --- 1,328 --- 12156 --- ... --- 9896
Following 3 out of 14 (load-) rules files to load the data columns into the cube:
Rules File1:
dim 1 --- dim 2 --- dim 3 --- sales --- ignore --- ignore --- ... --- ignore
Rules File2:
dim 1 --- dim 2 --- dim 3 --- ignore --- cogs --- ignore --- ... --- ignore
Rules File14:
dim 1 --- dim 2 --- dim 3 --- ignore --- ignore --- ignore --- ... --- amount
Is the upper table design what GlennS mentioned as a "Data" column concept which only allows a single numeric data value ?
In this case I cant tag two or more columns as “Data fields”. I just can tag one column as “Data field”. Other data fields I have to tag as “ignore fields during data load”. Otherwise, when I validate the rules file, an Error occurs “only one field can contain the Data Field attribute”.
Or may I skip this error massage and just try to tag all 14 fields as “Data fields” and “load data” ?
Please advise.
Am I right that the other way is to reconstruct the table/view (and the rules files) like follows to load all of the data in one pass:
dim 0 --- dim 1 --- dim 2 --- dim 3 --- data
Account --- Region -- ID --- Product --- data
sales --- West --- D1 --- Coffee --- 11001
sales --- West --- D2 --- Tea10 --- 12011
sales --- West --- S1 --- Tea10 --- 14115
sales --- West --- C3 --- Tea10 --- 21053
sales --- East ---- S2 --- Coffee --- 15563
sales --- East ---- D1 --- Tea10 --- 15894
sales --- East ---- D3 --- Coffee --- 19689
sales --- East ---- C1 --- Coffee --- 18897
sales --- East ---- C3 --- Tea10 --- 11699
cogs --- West --- D1 --- Coffee --- 1,322
cogs --- West --- D2 --- Tea10 --- 1,325
cogs --- West --- S1 --- Tea10 --- 1,699
cogs --- West --- C3 --- Tea10 --- 1,588
cogs --- East ---- S2 --- Coffee --- 1,458
cogs --- East ---- D1 --- Tea10 --- 1,664
cogs --- East ---- D3 --- Coffee --- 1,989
cogs --- East ---- C1 --- Coffee --- 1,988
cogs --- East ---- C3 --- Tea10 --- 1,328
discounts --- West --- D1 --- Coffee --- 10789
discounts --- West --- D2 --- Tea10 --- 10548
discounts --- West --- S1 --- Tea10 --- 10145
discounts --- West --- C3 --- Tea10 --- 10998
discounts --- East ---- S2 --- Coffee --- 10991
discounts --- East ---- D1 --- Tea10 --- 11615
discounts --- East ---- D3 --- Coffee --- 15615
discounts --- East ---- C1 --- Coffee --- 11898
discounts --- East ---- C3 --- Tea10 --- 12156
amount --- West --- D1 --- Coffee --- 548
amount --- West --- D2 --- Tea10 --- 589
amount --- West --- S1 --- Tea10 --- 852
amount --- West --- C3 --- Tea10 --- 981
amount --- East ---- S2 --- Coffee --- 876
amount --- East ---- D1 --- Tea10 --- 156
amount --- East ---- D3 --- Coffee --- 986
amount --- East ---- C1 --- Coffee --- 256
amount --- East ---- C3 --- Tea10 --- 9896
And the third way is to adjust the essbase.cfg parameters DLTHREADSPREPARE and DLTHREADSWRITE (and DLSINGLETHREADPERSTAGE)
I just want to be sure that I understand your suggestions.
Many thanks for awesome help,
Zeljko

How to design data load process chain?

Hello,
I am designing data load process chains for the first time and would like to get some general information on best practicies in that area.
My situation is as follows:
I have 3 source systems (R3 and two for which I use flat files).
How do you suggest, should I define one big chain for all my loading process (I have about 20 InfoSources) or define a few shorter e.g.
1. Master data R3
2. Master data flat file system 1
3. Master data flat file system 2
4. Transaction data R3
5. Transaction data file sys 1
... and execute one after another succesful end?
Could you also suggest me any links or manuals on that topic?
Thank you
Andrzej

Andrzej,
My advise is to make separate chains for master & transaction data (always load in this order!) and afterwards make a 'master chain' where you insert these 2 chains one after the other (so: Start process -> Master data chain -> Transaction data chain).
Regarding the separate chains; paralellize as much as possible (if functionally allowed). Normally, the number of parallel ('vertical') chains equals the nr of CPU's available (check with basis-person).
Hope this provides you with enough info to start off with!
Regards,
Marco

How to handle data that got from SAP by Webservice?

Hi, Gurus.
I am working on IFbA with ABAP and Adobe LiveCycle Designer.
I've developed a function module in R/3 system and created webservice, connected the WSDL address to the form. The problem is how can i handle data that got from SAP by Webservice?
It's Ok if i bind the dataconnection node to the form element, but how can i handle data without binding to the form element? I mean that how can i use the data retrieved from Webserivce in the Javascript, please give me some sample code or script method.
Thanks.

Hi Yang,
If you are using Web Service, then the binding will be done automatically.
It can't grow automatically, what else should i do to achieve this?
1.Select the table(make sure you are not selecting subform) in Hierarchy view
2. Under table select the Row which is having the required fileds for data
3. In the Binding of that row, check the checkbox "Repeat Row for each item data" and under that check the Min Count which will set the default row **** to 1
Regards,
Sachin

How to confirm data loaded properly

Hi,
Could you please let me know how to confirm the data is loaded, where i can see the same, is there any preview data option is sytem 9.
Thanks,
sudhakar.

Hi,
It was loaded, I believe it was loaded using rule files.
I am in the process of testing, i know the application and database,
I need to confirm, whether the data loaded or not, plz tel me how i can firm that,
is there any preview data option. I know it is there in in v7.3,
Thanks,
sudhakar

How to handle data in a request object

I want to know the best way to handle data received from sql query to display on a jsp page and for page navigation. If I store data in a request object then the data is lost after displaying first page. When user hits Next link to view more data I need to do another query to database which I want to avoid. I display only 20 rows at a time. So if my query returns 100 rows I display 5 pages. I need to retain data when user hits next to view other pages. What is the best way to handle this?
- Raj

By caching your resultset. There are caching custom tags that perform this function. Check out http://www.servletsuite.com/jsp.htm.

How to make data loaded into cube NOT ready for reporting

Hi Gurus: Is there a way by which data loaded into cube, can be made NOT available for reporting.
Please suggest. <removed>
Thanks

See, by default a request that has been loaded to a cube will be available for reporting. Bow if you have an aggregate, the system needs this new request to be rolled up to the aggregate as well, before it is available for reporting...reason? Becasue we just write queries for the cube, and not for the aggregate, so you only know if a query will hit a particular aggregate at its runtime. Which means that if a query gets data from the aggregate or the cube, it should ultimately get the same data in both the cases. Now if a request is added to the cube, but not to the aggregate, then there will be different data in both these objects. The system takes the safer route of not making the 'unrolled' up data visible at all, rather than having inconsistent data.
Hope this helps...

AWM Newbie Question: How to filter data loaded into cubes/dimensions?

Hi,
I am trying to filter the amount of data loaded into my dimensions in AWM (e.g., I only want to load like 1-2 years worth of data for development purposes). I can't seem to find a place in AWM where you can specify a WHERE clause...is there something else I must do to filter data?
Thanks

Hi there,
Which release of Oracle OLAP are you using? 10g? 11g?
You can use database views to filter your dimension and cube data and then map these in AWM
Thanks,
Stuart Bunby
OLAP Blog: http://oracleOLAP.blogspot.com
OLAP Wiki: http://wiki.oracle.com/page/Oracle+OLAP+Option
OLAP on OTN: http://www.oracle.com/technology/products/bi/olap/index.html
DW on OTN : http://www.oracle.com/technology/products/bi/db/11g/index.html

How to find Data load time ?

Hi,
Where do i look for the Total data load time ? from data source to PSA ?

Hi Honar,
1) Goto monitor of IP, in header tab you can find the runtime of IP.
2) So you are loading data from source to BW.In IP header tab copy the req number, goto the source systems from which data is loading.
goto SM37 give the request number with BI as pre-fix, you will find the total run time of job with job log.
Hope this helps.

How to handle heavy load caused by master data in XI

Hi all,
We have a situation where masterdata has to be send through XI. As you all would agree, the initial load for master data (done through the BD10,12,14 etc) is heavy. Even though we have low priority queues defined in our XI server for master data, our servers performance is getting affected heavily. Any suggestion to improve the situation would be highly appreciated.
Thanks in advance
cheers
danus

Hi Danus,
We manage this on the R/3 side. Schedule multiple jobs to run BD10 at spaced intervals of time with different variants. For example:
10:00 - Materials 1 - 10000
10:15 - Materials 10001 - 20000
10:30 - Materials 20001 - 30000
Thanks,
J Wolff

How to handle DATE type problems in migrating from mysql to oracle.?

Hi,
I'm migrating only the data from mysql to oracle with the help of sql loader.But with this type i cannot able to insert the date values from mysql to oracle.In mysql i have defined date as "DATETIME" type and in oracle it is in TimeStamp.Whenever i'm inserting the values thru CTL file ,i'm getting an error invalid date format entered.How to solve this problem?
Thanks in Advance
JAI

you need to supply a mask to the timestamp entry. see http://download-west.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_field_list.htm#i1006714 for details on datatypes within the ctl file
B

How to control data load in Info-Package using ABAP?

Hello Gurus:
I am trying to resolve couple of issues.
1. I need to load data DAILY Full Load into a Planning cube. I have the 0NETDUEDATE   for selection in the
     Info-package. I am getting data from anothe base A/R cube. I want to Load Daily ONLY those records
     for which the "0NETDUEDATE" is Greater than "System Date". I think the logic would be IF 0NETDUEDAT
     .GT. SYS-DATUM, bruing the record. What should be the statement in 'Result' for keeping the record?
     (I don't know ABAP, so need help here...!)
2. Similalry, before loading the data above, I want to delete all the existing records from Planning cube where
     0NETDUEDATE is GT System date (records loaded the previosu day need to be deleted as the amounts may
     have changed). How can I achieve this selective deletion automatically in PC??
Appreciate your feedback very much.
Thanks... SMaa

Hi Shruti,
1) If i understand your requirement correctly you need to load only data which has 0NETDATE greater then sydatum.
So you can do this in infopackage .Infopackage>Data selection->0NETDATE(field)-->type(colum).here you can selecy ABAP type(6).Now you create ABAP code in this.
Here in ABAP code you need to specify high and low range ,Also the relation character like GT or BT(between).
i guess some how ur code will look like this
$$ begin of global - insert your declaration only below this line -
TABLES: ...
DATA:   ...
$$ end of global - insert your declaration only before this line   -
    InfoObject      = 0NETDUEDATE
    Fieldname       = NETDUEDATE
    data type       = NUMC
    length          = 0000010
    convexit        = PERI6
form compute_NETDUEDATE
tables l_t_range structure rssdlrange
changing p_subrc like sy-subrc.
      Insert source code to current selection field
$$ begin of routine - insert your code only below this line        -
data: l_idx like sy-tabix.
          l_idx = sy-tabix.
          l_t_range-low = sy-datum.
          l_t_range-option = 'GT'.
          modify l_t_range index l_idx.
          p_subrc = 0.
$$ end of routine - insert your code only before this line         -
endform.
i am not sure about coding but you can surely do it it is like specifying the ranges dynamically(you need to try it out)
(also as mentioned by others you can do this in traansformation routines)but infopackage routine you can directly write in PROD system.
2) And regarding the Selective deletion you donu2019t have nay process type in Process chain which does this
But there is function module u201C rsdri_infoprov_delete " which can be used to develop a program( I guess you must be having a existing ZProgram in ur system already available because this is used commonly.
Other wise you need to develop a ZProgram for this.
Thanks and Regards
Arun

How to handle date prompt in report

Hi I have date column on database as timestamp.
Now need to apply filter in report based on date prompt.
Do i nee to cast it in Date format..or how i can do it.
thanks,

Hi User,
Which versionof obiee are you in? If you are on 10.1.3.4.1 version, apply PATCH 9492821 to fix this issue.
Can you do the following checks,
In the repository physical layer, set the column to have a data type of date
In the BMM layer, where the column is mapped, edit the column mapping to contain a CAST function to cast as date
Regards,
Dpka

How to Handle Data Loads in BW

Similar Messages

Maybe you are looking for