Slow Loading data from DB, one record at a time, HOW?

I am currently doing a project....
I need to display a record of data on a "form"(JFrame)
the way i handle it is
1. Load first record
2. pass to the display object
3. display object -> on click next, call manager object to process the request
4. manager, get next record with the help of DBManager
5. manager, set the record into the display object
6. manager, command object to refresh screen...
manager contains DBManager and disply instance....
it work perfectly fine for the first 20 or so record, but after some time, every click is slowed down about 20 over times...
i.e. first record take .5 sec, 50th take 30 sec
I traced the memory usage it is ok... but what could the problem be???
Memory usage list...
Before total:2031616 Free:810728
Before total:2031616 Free:1055016
Before total:2031616 Free:784272
Before total:2031616 Free:1049648
Before total:2031616 Free:821464
Before total:2031616 Free:1091312
Before total:2031616 Free:832568
Before total:2031616 Free:1057192
Before total:2031616 Free:793864
Before total:2031616 Free:1009872
Before total:2031616 Free:736552
Before total:2031616 Free:1000048
Before total:2031616 Free:776496
Before total:2031616 Free:563400
Before total:2031616 Free:834264
Before total:2031616 Free:600392
Before total:2031616 Free:883232
Before total:2031616 Free:637144
Before total:2031616 Free:877344
Before total:2031616 Free:625808
Before total:2031616 Free:876032
Before total:2031616 Free:603856
Before total:2031616 Free:440688
Before total:2031616 Free:681816
Before total:2031616 Free:431080
Before total:2031616 Free:714832
Before total:2031616 Free:465848
Before total:2031616 Free:790152
Before total:2031616 Free:613168
Before total:2031616 Free:364024
Before total:2031616 Free:677752
Before total:2031616 Free:421064
Before total:2031616 Free:716144
Before total:2031616 Free:541320
Before total:2031616 Free:372024
Before total:2031616 Free:595424
Before total:2031616 Free:425288
Before total:2031616 Free:250016
Before total:2031616 Free:576584
Before total:2031616 Free:405744
Before total:2031616 Free:225440
Before total:2031616 Free:546728
Before total:2031616 Free:370240
Before total:2031616 Free:204136
Before total:2031616 Free:540816
Before total:2031616 Free:367384
Before total:2031616 Free:196400

Hey,
I was wondering about a compelling reason, why you want to go back to the database on every click. I would fetch "X" rows at a time from the Database and cache them as Value Objects. X could be 10 or even 100. It depends on your requirements.
As far as the performance going down there could be lot of reasons. Do you open and close the connection on every click. How many connections are open at a time? do you close the connection? or the connection is alive?
Chidda

Similar Messages

Source System Creation in BI to load data from ECC 6.0

Our ECC 6.0 and BI 7.0 is one server. I would like to load data from ECC 6.0 to BI. How to create source system? I have source system for flat file and Data Mart (Myself source system) but don't know how to create source system to laod data from ECC 6.0. Please provide step by step guide.
Thanks.

Check here:
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c0751ba5-b7a8-2b10-6d97-e91e85c0fafa
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/0087780d-81b0-2a10-eb82-86e1062f1a2d
http://www.saptechies.com/connectivity-file-system-in-sap-bi/
http://help.sap.com/bp_biv370/html/index.htm
Thanks...
Shambhu

Load Data from a table on one server's database, to the same table structure in multiple server databases

Hi,
I have a situation where i have to load data from one server/database table to multiple servers/databases.
Example:
I need to load data from dbo.TABLE_A (on Server: Server_A & Database: Database_A) to the same table on the list of server databases like
Server: Server_B , Database: Database_B
Server: Server_C , Database: Database_C
Server: Server_D , Database: Database_D
Server: Server_E , Database: Database_E
Server: Server_F , Database: Database_F
Server: Server_G , Database: Database_G
Server: Server_H , Database: Database_H
so on and so forth on 250 such server database combinations.
The table structure is the same on all the servers.
If i make the source or destination dynamic, it throws an error while mapping ?
I cannot get Linked server permissions and SQL Server Config thing doesn't work as well.
Please suggest on how to load data from one source to multiple server/databases.
Thank you.

I just need to transfer one table's data. its like i have to use a query to pick data for
the most recent data. So i use something like, select A, B, C, D from dbo.table where ETL_TIMESTAMP > (the max(etltimestamp) in the destination on different server). There are no foreign key relationships and the data should not be truncated. it just had
to append the new records.

Load data from one Application to another in BPC NW 7.5

Hi
I have a requirement to load data from One application to another application.
eg. HR Plannning to CostCenter Planning.
I am aware of a method to load data using transformation file mapping source and target dimensions and run DM package and this works fine.
However, the challenge is if i try to automate this process then i will have to hardcode the application tech name and save as variant to use that in process chain. Now, if i hardcode the tech name of the application then it might happen that during full optimization the tech name might change and the automation load will fail.
so how to handle this situation? any standard practices available?
appreciate your inputs/thoughts.
Regards.

Hi,
Exactly. The main problem is that there is no statement to transfer the data. Please take a look at the below link from help.sap:
http://help.sap.com/saphelp_bpc75_nw/helpdata/en/f4/33f0bbc46c4a53860f1410f20ffb8b/content.htm
Since the structure of both the applications is same, you dont require add_dim or rename_dim, etc. However, you definitely need the when statement.
If you want to transfer all the records, then you can have something like
*WHEN ACCOUNT
*IS *
*REC(EXPRESSION = %VALUE%)
*ENDWHEN
Hope this helps.

Loading Data from one Cube into another Cube

Hi Guys,
I am trying to load data from one cube A to another cube B. Cube A has data around 200,000 records. I generate export datasource on Cube A. Replicated the datasource and created InfoSource and activated it.
I created update rules for Cube B selecting Source as Cube A. I do have a start routine to duplicate records in Cube A. Now when I schedule load,
It stops at Processing Datapacket and says no data. Is there something wrong with the update routine or is there any other way to load form cube to cube in a simpler way?
Thanks in advance

This is the start routine to duplicate records in two currencies.
DATA: datew   TYPE /bi0/oidateto,
      datew2 TYPE rsgeneral-chavl,
      fweek   TYPE rsgeneral-chavl,
      prodhier TYPE /bi0/oiprod_hier,
      market TYPE /bic/oima_seg,
      segment TYPE /bic/oizsegment.
DATA: BEGIN OF S_DATA_PACK OCCURS 0.
        INCLUDE STRUCTURE /BIC/CS8ZSDREV.
DATA: END OF S_DATA_PACK.
S_DATA_PACK[] = DATA_PACKAGE[].
REFRESH DATA_PACKAGE.
LOOP AT S_DATA_PACK.
    move-corresponding s_data_pack to DATA_PACKAGE.
    if DATA_PACKAGE-loc_currcy = 'EUR'.
      DATA_PACKAGE-netval_inv = DATA_PACKAGE-/bic/zsdvalgrc.
      DATA_PACKAGE-CURRENCY = 'USD'.
      APPEND DATA_PACKAGE.
      DATA_PACKAGE-netval_inv = DATA_PACKAGE-/bic/zsdvalloc.
      DATA_PACKAGE-CURRENCY = 'EUR'.
      APPEND DATA_PACKAGE.
    else.
      DATA_PACKAGE-netval_inv = DATA_PACKAGE-/bic/zsdvalgrc.
      DATA_PACKAGE-CURRENCY = 'USD'.
      APPEND DATA_PACKAGE.
    endif.
ENDLOOP.
This is to load Quantity field
RESULT = COMM_STRUCTURE-BILL_QTY.
This is to load Value field
RESULT = COMM_STRUCTURE-NETVAL_INV.
UNIT = COMM_STRUCTURE-currency.

Data load failed while loading data from one DSO to another DSO..

Hi,
On SID generation data load failed while loading data from Source DSO to Target DSO.
Following are the error which is occuuring--
Value "External Ref # 2421-0625511EXP " (HEX 450078007400650072006E0061006C0020005200650066
Error when assigning SID: Action VAL_SID_CONVERT, InfoObject 0BBP
So, i'm not getting WHY in one DSO i.e Source it got successful but in another DSO i.e. Target its got failed??
While analyzing all i check that SIDs Generation upon Activation is ckecked in source DSO but not in Target DSO..so it is reason its got failed??
Please explain..
Thanks,
Sneha

Hi,
I hope your data flow has been designed in such a way where the 1st DSO as a staging Device and all transformation rules and routine are maintained in between 1st to 2nd dso and sid generation upon activation maintained in 2nd DSO. By doing so you will be getting your data 1st DSO same as your source system data since you are not doing any transformation rules and routine etc.. which helps to avoid data load failure.
Please analyze the following
Have you loaded masterdata before transaction data ... if no please do it first
go to the property of first dso and check whether there maintained sid generation up on activation (it may not be maintained I guess)
Goto the property of 2nd Dso and check whether there maintained sid generation up on activation (It may be maintained I hope)
this may be the reason.
Also check whether there is any special char involvement in your transaction data (even lower case letter)
Regards
BVR

Loading data from SAP Business One to BW

We need to load data from several SAP Business One databases to BW. I find some special objects for SAP Business One in the Business Content
And a short description in Help:
The source for the data upload is a file system. The relevant files can be generated using the views provided within the SAP Business One database.
May be someone has an experience in this area or could advise documentation about this
May be better to use DB-connect?
Thanks for your help

Hello Elena,
I am just curious if you've found answer to your question how to extract data fron BusinessOne into SAP/BI? What can we use? Only flat files or there are some standard extractors exists? Or have you used DB Connect for that?
Thank you,
-Vitaliy

Problem in loading data from one DSO to another

Hi All,
I am using Multicube 0PM_MP03, this mutli cube use DSO 0PM_DS02 & 0PM_DS01, data will flow from PSA to 0PM_DS01 & then to 0PM_DS02 using a Update rule, i have triggered my infopackage & i can see data in 0PM_DS01 & i have activated the data in 0PM_DS01, now on right click on 0PM_DS01 i click on the option Additional Functions -> Update 3.x Data to Target, now the request status is green in 0PM_DS02 but i cannot see any data in this DSO, neither in new or active, can anyone help me out in this where i am doing wrong.
Thanks,
Pritam.

Hi Pritam,
This is strange, Check the below points.
1. If the number of records added 0, in manage of DSO2?
2. Confirm if there is a transfprmation between ODS1 & ODS2. If there is, you need to use a DTP (not Additional Functions -> Update 3.x Data to Target).
3. Activate the request and then check.
4. Maintain PSA for the datasource 80PM_DS01 and when loading data from DSO1 to DSO2 choose option PSA and then into targets.
Regards,
Panksj.

Problem in loading data from one DSO to onother DSO in BI7

Hi
Iam loading data from one DSO to onother, After executing DTP Iam getting error message as "Aggregation NOP not allowed on DB (key figure 0DSCT_DAYS1) ", Here 0DSCT_DAYS1 is the keyfigure aggragation type is "NOP" , exception aggregation is " No aggregation(X if more than one value unique to 0 occurs)"
Please kindly give me solution .
Radha Krishna

We had this issue in our project. It mainly arises when you have written start routine in transformation. At start routine level it can not aggregate the fields for which NOP is set & that's why it throws an error.
Please check the load by removing the start routine. If it works then introduce a data target in between which has exactly same structure as previous data target except the fields with NOP aggregation.

How load data from One Infoobject to Another Infoobject?

Hi GuRUS
Any body has succeded in loading data from One Infoobject to Another Infoobject?
Pls Let me kNow
Regards
Durai

Hi Durai,
Create export datasource on the info-object and then load the data to the other info-object.
Check this link also:
Re: How load data from Infoobject to another Infoobject?
Bye
Dinesh

How to load data from one Infocube to another request by request using DTP

Hi All,
I have a scenario where we are maintaining backup Infocube B for Infocube A. User loads data from a flat file many times a day in different requests. We need to maintain backup of the Infocube A on weekly basis. i.e data to be refreshed with delta update from cube A to cube B. There are some situations where user deletes some of the requests in Infocube A randomly after use. Will this change effect in back up cube B when performed data refresh from cube A to cube B. i.e. this functionality is similar to reconstruct in BW 3.5. Now we are running on BI 7.0 SP 9.
Can anyone answer this ASAP.
Many Thanks,
Ravi

You cannot load request by request, " Get Data By Request " DTP loads request by request on " First In First Out " basis. You can run some Pseudo/Fake DTP's if you dont want to load data from a particular request.
If the user deletes a request from Cube A, it wont be loaded to Cube B but if it is already loaded in Cube B and later the user deletes the request from Cube A you have to delete the request frm Cube B. Inorder to monitor request by request run DTP with " Get Data By Request set.

Loading data from Cube to Planning area

Hi,
If I am loading data from a cube to a planning area using transaction TSCUBE,
does the system load data into planning area for the combinations that exist in the cube or does it load for all CVCs?
For example,
I have my CVC as Plant, Material, Customer
If there are 4 CVCs in the POS that were previously generated as
Plant--Material--Customer
01--M1--
C1
01--M2--
C3
01--M2--
C2
01--M4--
C5
If the cube has data like this:
Plant--MaterialCustomer----Qty.
01--M1C1--
10
01--M2C3--
20
01--M2C2--
5
(doesnot have the last combination), then if I use TSCUBE transaction to load data to Planning area from this cube,
is the data loaded as
Plant--MaterialCustomer----Qty.
01--M1C1--
10
01--M2C3--
20
01--M2C2--
5
Only for the 3 combinations that exist in the cube and not load anything for the last one
OR
is the data loaded as
Plant--MaterialCustomer----Qty.
01--M1C1--
10
01--M2C3--
20
01--M2C2--
5
01--M4C5--
0
Load all 4 combinations and send 0 as the cube doesnot have this combination?
Hope I am clear on this question.
Thanks.

Thanks a lot Vinod, Srinivas and Harish. The reason why I am asking you is that we have a scenario where we get this situation.
We initially get data from R/3 to BW to APO like the below:
Plant--MaterialCustomer----Qty.
01--M1C1--
10
Later when the customer is changed or bought out by somebody C1 is changed to C2. Some times when the business doesnot know who the customer is initially they just put C1 as dummy and then after sometime replace it by C2. Then the new record coming in is as follows:
Plant--MaterialCustomer----Qty.
01--M1C2--
10
BW can identify changes in transaction data but not in Master data. What I mean by this is when Qty. 10 changes from 10 to 20, the system can identify it in deltas.
If the customer (master data) changes to C2 from C1, the system thinks it's a new record all together then if I use delta loads, it gets me the following:
Plant--MaterialCustomer----Qty.
01--M1C1--
10
01--M1C2--
10
If I am looking at Plant and Material Level, my data is doubled.
So we are planning to do a full load that works like this:
1. Initial data like the below:
Plant--MaterialCustomer----Qty.
01--M1C1--
10
The CVC is created and the planning area has Qty.10
Then we delete the contents of cube and do a full load into the cube with changed customer
Plant--MaterialCustomer----Qty.
01--M1C2--
10
This time a new CVC is created. Then we have another 10 loaded into Planning area.
If the system loads all CVCs, then the it would send
Plant--MaterialCustomer----Qty.
01--M1C1--
0
01--M1C1--
10
If the system loads only combinations in cube,
then it loads
Plant--MaterialCustomer----Qty.
01--M1C2--
10
But the system already has another 10 for Customer C1 duplicating the values.
We are trouble in the second case.
We had to go fr this solution instead of realignment as our business has no way pf knowing that C1 was replaced by C2.
Hope I am clear.

Loading Data from PSA to ODS

Hi,
right now there are nearly 50 requests in my PSA. I want to load entire data from PSA to ODS at a time. Ours is 3.x flow. I m able to load one request at a time. Can anyone help me to load all requests at atime?
Please give me step by step.

Hi Narendra,
First of all all the records are unable to pick up at a time i single attempt due to. follow the reason
It is not possible to load all the PSA requests' to target at a time.Because records picks up from PSA to Data target one by one due dependencies of routine run on the data transfer up to the extent. anyway if you can do this thing the process of loading the requests' one by one, offcourse it is totally depends upon manual .
Hope you are understood the scenario.
Regards,
Vamsi Krishna Chandolu

Error while loading data from 0FI_TX_4 to ODS

Dear Friends,
One of our proccess chain getting error since a week, every day with same error while loading data from R3 data source 0FI_TX_4 to ODS.
Short dump is raised in R3 system.
"DBIF_RSQL_SQL_ERROR" CX_SY_OPEN_SQL_DBC
"SAPLBWFIR" or "LBWFIRU13"
"BWFIR_READ_BSET_CPUDT_DATA"
ORA-01555
The number of records are below 10000 only.
All other chains are OK.
we have already searched SDN and Notes...and applied below solution, but still no use
Increase the size of all the roll back segments
Increase number of REDO logs
Decrease the number of records per data packet
Any more suggessions please ??
Thanks
Tony

How are you telling that no. of records are below 10000 .Have you checked the no.of records in PSA???
Now ,I tell you something which I had done for this error .I increased the Package size such that whole data gets transferred in single data package.Please make the package size as large as possible so that all the data can be processed in single package.
I know you have read so many threads on the same but what I am suggesting is what I have done and successfully rectified this issue.
regards,
rocks

Load data from BW 7.0 DSO into a ECC 6.0 SD standard/Custom pricing table

Gurus,
We have a scenario, where we need to load data from a BW DSO to SD standard/Custom pricing table in ECC 6.0. That data will be few thousand records.
Per my knowledge, in BPS, retractors are available to update data from BW to ECC and OpenHub also can be used to handle similar scenarios.
Any one of you came across similar scenario?
If you have any third option (not BPS rectors or OpeHub) as a solution to handle this kind scenario, and share the knowledge, it will be greatly appreciated.
Thanks in advance,
Vittal

Hi Yogesh,
Thanks for your reply.
We have large data volumes for Billing datasource and hence moving flat files using PI is not an option and also as i mentioned a part of requirement is monitoring of the whole process as well. What i mean by this is if a 1000 billing document items were passed on to CRM7 to create a member activities and because of some reason a member activity was not created for one of the billing doc items, we should know the problematic record and reason why member activity was not created in CRM7 (reason code). And then be able to fix it. All this requires an end to end monitoring capability and also guaranteed delivery of data to CRM7.
Hence i was trying to explore the enterprise web service option.
What i am not sure of is how to expose BW DSO delta request to CRM7 using a web service ? or any other method that gives end to end monitoring capability and also guaranteed delivery of data to CRM7.
Any other suggestions ?
Thanks
CK

Slow Loading data from DB, one record at a time, HOW?

Similar Messages

Maybe you are looking for