How to manage millions of rows in a BADI

Hi,
I'm going to write a BADI to copy data from a category to another one. It will manage several millions of rows. I imagine that I'm going to have some problems if I execute a loop on ct_data. What is the best approach to manage such a huge number of data?
regards,
Romuald

Hi Romuald,
I would request you to please go through the below doc on Performance considerations for BADI.
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/70187b3e-d5ce-2d10-d780-bb8b1a5b0bdc?QuickLink=index&overridelayout=true
Hope this helps.
Regards,
Raghu

Similar Messages

How to write a cursor to check every row of a table which has millions of rows

Hello every one.
I need help. please... Below is the script (sample data), You can run directly on sql server management studio.
Here we need to update PPTA_Status column in Donation table. There WILL BE 3 statuses, A1, A2 and Q.
Here we need to update PPTA_status of January month donations only. We need to write a cursor. Here as this is a sample data we have only some donations (rows), but in the real table we have millions of rows. Need to check every row.
If i run the cursor for January, cursor should take every row, row by row all the rows of January.
we have donations in don_sample table, i need to check the test_results in the result_sample table for that donations and needs to update PPTA_status COLUMN.
We need to check all the donations of January month one by one. For every donation, we need to check for the 2 previous donations. For the previous donations, we need to in the following way. check
If we want to find previous donations of a donation, first look for the donor of that donation, then we can find previous donations of that donor. Like this we need to check for 2 previous donations.
If there are 2 previous donations and if they have test results, we need to update PPTA_STATUS column of this donatioh as 'Q'.
If 2 previous donation_numbers has test_code column in result_sample table as (9,10,11) values, then it means those donations has result.
BWX72 donor in the sample data I gave is example of above scenario
For the donation we are checking, if it has only 1 previous donation and it has a result in result_sample table, then set this donation Status as A2, after checking the result of this donation also.
ZBW24 donor in the sample data I gave is example of above scenario
For the donation we are checking, if it has only 1 previous donation and it DO NOT have a result in result_sample table, then set this donation Status as A1. after checking the result of this donation also.
PGH56 donor in the sample data I gave is example of above scenario
like this we need to check all the donations in don_sample table, it has millions of rows per every month.
we need to join don_sample and result_sample by donation_number. And we need to check for test_code column for result.
-- creating table
CREATE TABLE [dbo].[DON_SAMPLE](
[donation_number] [varchar](15) NOT NULL,
[donation_date] [datetime] NULL,
[donor_number] [varchar](12) NULL,
[ppta_status] [varchar](5) NULL,
[first_time_donation] [bit] NULL,
[days_since_last_donation] [int] NULL
) ON [PRIMARY]
--inserting values
Insert into [dbo].[DON_SAMPLE] ([donation_number],[donation_date],[donor_number],[ppta_status],[first_time_donation],[days_since_last_donation])
Select '27567167','2013-12-11 00:00:00.000','BWX72','A',1,0
Union ALL
Select '36543897','2014-12-26 00:00:00.000','BWX72','A',0,32
Union ALL
Select '47536542','2014-01-07 00:00:00.000','BWX72','A',0,120
Union ALL
Select '54312654','2014-12-09 00:00:00.000','JPZ41','A',1,0
Union ALL
Select '73276321','2014-12-17 00:00:00.000','JPZ41','A',0,64
Union ALL
Select '83642176','2014-01-15 00:00:00.000','JPZ41','A',0,45
Union ALL
Select '94527541','2014-12-11 00:00:00.000','ZBW24','A',0,120
Union ALL
Select '63497874','2014-01-13 00:00:00.000','ZBW24','A',1,0
Union ALL
Select '95786348','2014-12-17 00:00:00.000','PGH56','A',1,0
Union ALL
Select '87234156','2014-01-27 00:00:00.000','PGH56','A',1,0
--- creating table
CREATE TABLE [dbo].[RESULT_SAMPLE](
[test_result_id] [int] IDENTITY(1,1) NOT NULL,
[donation_number] [varchar](15) NOT NULL,
[donation_date] [datetime] NULL,
[test_code] [varchar](5) NULL,
[test_result_date] [datetime] NULL,
[test_result] [varchar](50) NULL,
[donor_number] [varchar](12) NULL
) ON [PRIMARY]
---SET IDENTITY_INSERT dbo.[RESULT_SAMPLE] ON
---- inserting values
Insert into [dbo].RESULT_SAMPLE( [test_result_id], [donation_number], [donation_date], [test_code], [test_result_date], [test_result], [donor_number])
Select 278453,'27567167','2013-12-11 00:00:00.000','0009','2014-01-20 00:00:00.000','N','BWX72'
Union ALL
Select 278454,'27567167','2013-12-11 00:00:00.000','0010','2014-01-20 00:00:00.000','NEG','BWX72'
Union ALL
Select 278455,'27567167','2013-12-11 00:00:00.000','0011','2014-01-20 00:00:00.000','N','BWX72'
Union ALL
Select 387653,'36543897','2014-12-26 00:00:00.000','0009','2014-01-24 00:00:00.000','N','BWX72'
Union ALL
Select 387654,'36543897','2014-12-26 00:00:00.000','0081','2014-01-24 00:00:00.000','NEG','BWX72'
Union ALL
Select 387655,'36543897','2014-12-26 00:00:00.000','0082','2014-01-24 00:00:00.000','N','BWX72'
UNION ALL
Select 378245,'73276321','2014-12-17 00:00:00.000','0009','2014-01-30 00:00:00.000','N','JPZ41'
Union ALL
Select 378246,'73276321','2014-12-17 00:00:00.000','0010','2014-01-30 00:00:00.000','NEG','JPZ41'
Union ALL
Select 378247,'73276321','2014-12-17 00:00:00.000','0011','2014-01-30 00:00:00.000','NEG','JPZ41'
UNION ALL
Select 561234,'83642176','2014-01-15 00:00:00.000','0081','2014-01-19 00:00:00.000','N','JPZ41'
Union ALL
Select 561235,'83642176','2014-01-15 00:00:00.000','0082','2014-01-19 00:00:00.000','NEG','JPZ41'
Union ALL
Select 561236,'83642176','2014-01-15 00:00:00.000','0083','2014-01-19 00:00:00.000','NEG','JPZ41'
Union ALL
Select 457834,'94527541','2014-12-11 00:00:00.000','0009','2014-01-30 00:00:00.000','N','ZBW24'
Union ALL
Select 457835,'94527541','2014-12-11 00:00:00.000','0010','2014-01-30 00:00:00.000','NEG','ZBW24'
Union ALL
Select 457836,'94527541','2014-12-11 00:00:00.000','0011','2014-01-30 00:00:00.000','NEG','ZBW24'
Union ALL
Select 587345,'63497874','2014-01-13 00:00:00.000','0009','2014-01-29 00:00:00.000','N','ZBW24'
Union ALL
Select 587346,'63497874','2014-01-13 00:00:00.000','0010','2014-01-29 00:00:00.000','NEG','ZBW24'
Union ALL
Select 587347,'63497874','2014-01-13 00:00:00.000','0011','2014-01-29 00:00:00.000','NEG','ZBW24'
Union ALL
Select 524876,'87234156','2014-01-27 00:00:00.000','0081','2014-02-03 00:00:00.000','N','PGH56'
Union ALL
Select 524877,'87234156','2014-01-27 00:00:00.000','0082','2014-02-03 00:00:00.000','N','PGH56'
Union ALL
Select 524878,'87234156','2014-01-27 00:00:00.000','0083','2014-02-03 00:00:00.000','N','PGH56'
select * from DON_SAMPLE
order by donor_number
select * from RESULT_SAMPLE
order by donor_number

You didn't mention the version of SQL Server. It's important, because SQL Server 2012 makes the job much easier (and will also run much faster, by dodging a self join). (As Kalman said, the OVER clause contributes to this answer).
Both approaches below avoid needing the cursor at all. (There was part of your explanation I didn't understand fully, but I think these suggestions work regardless)
Here's a SQL 2012 answer, using LAG() to lookup the previous 1 and 2 donation codes by Donor: (EDIT: I overlooked a couple things in this post: please refer to my follow-up post for the final/fixed answer. I'm leaving this post with my overlooked
items, for posterity).
With Results_Interim as
Select *
, count('x') over(partition by donor_number) as Ct_Donations
, Lag(test_code, 1) over(partition by donor_number order by donation_date ) as PrevDon1
, Lag(test_code, 2) over(partition by donor_number order by donation_date ) as PrevDon2
from RESULT_SAMPLE
Select *
, case when PrevDon1 in (9, 10, 11) and PrevDon2 in (9, 10, 11) then 'Q'
when PrevDon1 in (9, 10, 11) then 'A2'
when PrevDon1 is not null then 'A1'
End as NEWSTATUS
from Results_Interim
Where Test_result_Date >= '2014-01' and Test_result_Date < '2014-02'
Order by Donor_Number, donation_date
And a SQL 2005 or greater version, not using SQL 2012 new features
With Results_Temp as
Select *
, count('x') over(partition by donor_number) as Ct_Donations
, Row_Number() over(partition by donor_number order by donation_date ) as RN_Donor
from RESULT_SAMPLE
, Results_Interim as
Select R1.*, P1.test_code as PrevDon1, P2.Test_Code as PrevDon2
From Results_Temp R1
left join Results_Temp P1 on P1.Donor_Number = R1.Donor_Number and P1.Rn_Donor = R1.RN_Donor - 1
left join Results_Temp P2 on P2.Donor_Number = R1.Donor_Number and P2.Rn_Donor = R1.RN_Donor - 2
Select *
, case when PrevDon1 in (9, 10, 11) and PrevDon2 in (9, 10, 11) then 'Q'
when PrevDon1 in (9, 10, 11) then 'A2'
when PrevDon1 is not null then 'A1'
End as NEWSTATUS
from Results_Interim
Where Test_result_Date >= '2014-01' and Test_result_Date < '2014-02'
Order by Donor_Number, donation_date

How to create a new row for a VO based on values from another VO?

Hi, experts.
in jdev 11.1.2.3,
How to create a new row for VO1 based on values from another VO2 in the same page?
and in my use case it's preferable to do this from the UI rather than from business logic layer(EO).
Also I have read Frank Nimphius' following blog,but in his example the source VO and the destination VO are the same.
How-to declaratively create new table rows based on existing row content (20-NOV-2008)
http://www.oracle.com/technetwork/developer-tools/adf/learnmore/13-create-with-params-169140.pdf
I have tried:
1.VO1(id,amount,remark1) and VO2(id,amount,remark2) are based on different EO,but render in same page,
2.Drag and drop a Createwithparams button for VO1(id,amount,remark),
3.add: Create insertinside Createwithparams->Nameddata(amount),
4.set NDName:amount, NDValue:#{bindings.VO2.children.Amount}, NDtype:oracle.jbo.domain.Number.
On running,when press button Createwithparams, cannot create a new row for VO1, and get error msg:
<Utils> <buildFacesMessage> ADF: Adding the following JSF error message: For input string: "Amount"
java.lang.NumberFormatException: For input string: "Amount"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
Can anyone give some suggestions?
Thanks in advance.
bao
Edited by: user6715237 on 2013-4-19 下午9:29

Hi,CM,
I'm really very appreciated for your quick reply! You know, today is Saturday, it's not a day for everyone at work.
My principal requirement is as follows:
1.select/check some rows from VO2, and for each selection create a new row with some attributes from VO2 as default values for VO1's corresponding attributes, and during this process the user may be cancel/uncheck or redo some of the selections.
--so it's better to implement it in UI rather than in EO.
2.it's better to implement this function with declarative way as in Frank Nimphius' blog.
--little Jave/JS coding, the better. I only have experience in ORACLE FORMS, little experience in JAVA/JS.
In order to get full information for the requirements of my use case, can take a check at:
How to set default value for a VO query bind variable in a jspx page?
(the end half of the thread: I have a more realworld requirement similar to the above requirement is:
Manage bank transactions for clients. and give invoices to clients according to their transaction records. One invoice can contain one or many transactions records. and one transaction records can be split into many invoices.
Regards
bao
Edited by: user6715237 on 2013-4-19 下午11:18
JAVE->JAVA

How to manage large database records in enterprise application

Hi All,
I am working on a large enterprise application relating to Capital Market. I am working in Java and with its extended technology. I am facing one critical problem which needs solution from your side. I have a database table which contains approximately more than 5 millions of records, I want to display the records with proper pagination. Here I am using Hibernate for database related stuffs. I am using a query which contains a join query to load the records. After the query the filtered records come to approcimately 80,000. I am unable to make proper pagination, everytime for next or previous set of 10 records I hit the database which is a time consuming affair. I do not know what I will do , should I cache the data for pagination. Everytime I load more than 80,000 records, think that in an web based application, the no of users are 5000, then how to manage. I need core java level solution not in the JSP level. Please help me in this regard.

After the query the filtered records come to approcimately 80,000. I am unable to
make proper pagination,Just a thought. If you display 50 per page, that's 1600 pages. Say it takes the user
15 seconds to read the page: total 400min=6 2/3 hours and probably a bad case of RSI.
The proper pagination would possibly be no pagination.

How to manage scrolling in Table Control

Hi,
I have one scenario in which i have one table control in that one column contains check boxes. After i check, some check boxes in some rows and press scrolling button means the selection previously made goes off. I want the selection remains selected.
Please any one suggest me how to manage this.
Regards,
Mithun.

Your PAI flow logic should be following this type of structure:
process after input.
module at_exit_command at exit-command. "don't exit here if you want TC data changes
module d9999_before_tc.
loop with control tc_9999.
    chain.
      field:
        gs_9999_tc-sel,
        gs_9999_tc-checkbox.
      module d9999_line_tc_handler. "hold onto TC row data in here
    endchain.
endloop.
module d9999_user_command.
module d9999_pai_scroll. "scroll the table control
As noted above, you need to catch the values of your checkboxes within the loop / endloop for the table control, and do any scrolling after this.
Jonathan

How to load 1 milloin rows ???

Hi,
I want to load 1 million rows in one of my table through sql loader,is it possible to load??or I have a table named sales in which i have 1 milloin rows how could I write those rows into notepad or any other text editor or even excel??
thanks in advance ...

Hello Alok,
You can load/import from anywhere, thats not a problem.
For making one input file for SQL Loader what you can do is use a editor that can save 1 million rows.
OR
You can make small files using excel or some text-editor, and than at OS level merge these files for example in unix you can do :
cat *.txt > merge.txt
Then you can use merge as input to SQL loader
My question which still remain unanswered.. from which package you want to import?
If it is access than you can export directly to a text/csv file
If you are using oracle, then you can use export, copy command, or db-link!!
Cheers,
Sudhanshu

How to read block of rows from database tables

I have created a Database Application in Java and display all the records in tabular format of one Table. This table have Millions of Rows, If I run Select * from Table, then my Machine not responding, so Now I wants to add paging of 1000 rows at one time.
Is there are any option to read block of rows at one time and then query again for next page ?
Please let me know..
Database : SQL Server 2000/2005, Oracle and MYSQL
Thanks in Advance
Laxmilal

Refer to the SQL documentation of the database server how to query a subset of records.
This has certainly nothing to do with JDBC nor Java, but with SQL.

Loading millions of rows using SQL*loader to a table with constraints

I have a table with constraints and I need to load millions of rows in it using SQL*Loader.
What is the best way to do this, means what SQL*Loader options to use, for getting the best loading performance and how to deal with constraints?
Regards

- check if your table has check constraints (like column not null)
if you trust the data in the file you have to load you can disable this constrainst and after the loader enable this constrainst.
- Check if you can modify the table and place it in nologging mode (generate less redo but ONLY is SOME Conditions)
Hope it helps
Rui Madaleno

How to Commit before Insert Row when Press Create Insert Button ?

Hi all;
I'm Using JDev 11.1.1.2.0
How to Commit before Insert Row when Press Create Insert Button in ADF11g?
<af:commandButton actionListener="#{bindings.CreateInsert.execute}"
text="CreateInsert"
disabled="#{!bindings.CreateInsert.enabled}"
id="cb8" />
best regards;

You need to do a custom method eather in managed bean or in Application module to do that.
in managed bean it would be something like:
public void CommitAndInsert(ActionEvent actionEvent) {
OperationBinding opCommit = ADFUtils.findOperation("Commit");
opCommit.execute();
OperationBinding opCreateInsert = ADFUtils.findOperation("CreateInsert");
opCreateInsert.execute();
In page bindings Commit and CreateInsert must exist
then the button actionListener will be
<af:commandButton actionListener="#{backing.CommitAndInsert}"

Can any one tell me how to change the current row header in FB1LN tcode

Hi,
can any one tell me how to change the current row header in FB1LN tcode.
I want to show input date also in the layout.
Regards
Mave

Hi
I know two ways:
- transaction obvu: here you can insert new fields, but only if these fields are included in certain table (like BKPF, BSEG, BSIS.....
- if your field isn't included in those table, you have to modify the structure RFPOS and manage it in the BTE 1650. After updating RFPOS you have to run the program RFPOSXEXTEND to update the structure RFPOSXEXT.
Max

How to manage sequence number in PLD but not use LineNum()?

Hi,All
I want to show number in PLD.
Probem, I can't use LineNum() because in some row in matrix that no price i will not show data.
Now,if i use LineNum() when i print preview in column system will skip record that no have price.
How to manage for get number order ascending sequence ?
thank you
Message was edited by: Frank Moebius

Hi,
Unfortunately I am not an expert in this area.
I hope someone else can help you out though!
However, I would like to draw your attention to the fact that the main purpose of this forum is around the SAP Business One SDK - and so appears to be the main expertise of the people participating here...
...this might be one of the reasons why it happens that you don't get an answer top your posts sometimes...
Therefore I would like to advise you to use the "Implementation" Forum in the CPSN (Channel Partners Solution Network) to ask your questions.
I hope you get more answers there + quicker.
Sorry & Thanks for understanding,
Frank

Update million of rows

Dear all,
DB : 10.2.0.4.
OS : Solaris 5.10
I have a partitioned table with million of rows and am updating a filed like
update table test set amount=amount*10 where update='Y';
when I run this query it generates many archive logs as it doesn't commits the transaction anywhere..
Please give me an idea where I can commit this transaction after every 2000 rows.. such that archive log generation will not be much..
Please guide
Kai

There's not a lot you can do about the the amount of redo you generate (unless perhps you make the table unrecoverable, and that might not help much either).
It possible that if the column being updated is in an index dropping that during the update might help and recreating afterwards, but that could land you in more trouble.
One area of concern is the amount of undo space for the large transaction, this could even be exceeded and your statement might fail,
and that might be a reason for splitting it to smaller transactions.
Certainly there is no point in splitting down to 2000 records chunks, I'd want to aim much higher that that.
If you feel you want to divide it the record may contain a field that could be used, eg create_date, or if you are able to do partition by partition that might help.
If archive log management is the problem then speaking to the DBA should help.
Hope these thoughts help - but you are responsible for any actions you take, regards - bigdelboy

How to manage my media

Hi all
I need some advice please on how to manage my media between my original laptop which has all my current media on, and my new mac mini.
Some background....I got the mac mini mainly to use with my LCD TV and front row listening at home and viewing pics on a big screen.
So starting with music, im confused as to the best approach to take. I have an iPod and want to maintain my play counts and ratings as these drive my playlists. If i transfer my current metadata to the mac mini but continue synching to the laptop, then immediately the databases start to diverge. Im not worried about play counts on the mac mini, but changes to my ratings on my ipod (my method of rating) will not flow through to the mac mini. I dont want to synch to the mini as my wife will totally wreck my play counts, as she plays the same music over and over again!! However, I would like updated ratings to be on both systems. Im not sure there is a solution to this, but any ideas would be welcomed!
With pictures I want to maintain editing on my laptop as I have Elements 3.0 and dont fancy editing on my TV anyway. But of course I want to view my pictures through my mac mini on our tv!
I am right in that maybe my best option would be to invest in a external network hard drive for my media to live on and have both computers access it that way? Im on a 54mbs wireless laptop, would an external hard drive be fast enough over that? plus its more expense I dont really want to spend!
Apologies for the length, but I dont want to get this wrong and I know it will be harder to change if I start off on the wrong foot!
Thanks
Jared

Just to hit one topic of your post...this utility might be a solution for you...
http://www.findleydesigns.com/ipodaccess/index.html
It painlessly merged my older MS Windows nano's songs with my current Mac iPod's library.

How to create table with row type in smart forms

How to create table with row type in smart forms with out line type
please explain me the procedure

HI,
A table type describes the structure and functional attributes of an internal table in ABAP. In ABAP programs you can reference a table type TTYP defined in the ABAP Dictionary with the command DATA <inttab> TYPE TTYP. An internal table <inttab> is created in the program with the attributes defined for TTYP in the ABAP Dictionary.
A table type is defined by:
its line type, that defines the structure and data type attributes of a line of the internal table
the options for managing and accessing the data ( access mode) in the internal table
the key ( key definition and key category) of the internal table
The row type is defined by directly entering the data type, length and number of decimal places or by referencing a data element, structured type ( structure, table or view) or other table type. Or the row type can be a reference type.
<b>for more info :</b> http://help.sap.com/saphelp_nw2004s/helpdata/en/fc/eb366d358411d1829f0000e829fbfe/content.htm
Internal table
Regards
Sudheer

How to get First X rows

I need to select only the first X (say 10) rows of an ordered result. Because the result set may be millions of rows, I do not like the idea of producing the entire result, including having it ordered, and then only iterating over the first few.
How can I get oracle to only return the first 10 rows?
Ex.
select
calls.ph_number
calls.call_length
call_types.call_desc
from
calls,
call_types
where
calls.cust_num = SOME_CUST_NUM
calls.type = call_type.type
order by
calls.call_length

Since it is answered incorrectly or incompletely so frequently, I would suggest that you read Tom Kyte's article on the subject http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:127412348064
Justin
Distributed Database Consulting, Inc.
http://www.ddbcinc.com/askDDBC

How to manage millions of rows in a BADI

Similar Messages

Maybe you are looking for