Database table with potentially millions of records

Hello,
We want to keep track of user's transaction history from the performance database. The workload statistics contain the user transaction history information, however since the workload performance statistics are intended for temporary purposes and data from these tables are deleted every few months, we loose all the user's historical records.
We want to keep track of the following in a table that we can query later:
User ID - Length 12
Transaction - Length 20
Date - Length 8
With over 20,000 end users in production this can translate into thousands of records to be inserted into this table daily.
What is the best way to store this type of information? Is there a specific table type that is designed for storing massive data quantity? Also, over time (few years) this table can grow into millions or hundreds of millions of records. How can we manage that in terms of performance and storage space?
If anyone has worked with database tables with very large amounts of records, and would like to share your experiences, please let us know how we could/should structure this function in our environment.
Best Regards.

Hi SS
Alternatively, you can use a cluster table. For more help refer to F1 help on "IMPORT TO / EXPORT FROM DATABASE" statements.
Or you can store data as a file on the application server using "OPEN DATASET, TRANSFER, CLOSE DATASET" statements.
You can also select to archieve data of older than some definite date.
You can also mix your alternatives for the recent and archieve data.
*--Serdar <a href="https://www.sdn.sap.com:443http://www.sdn.sap.comhttp://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/com.sap.sdn.businesscard.sdnbusinesscard?u=qbk%2bsag%2bjiw%3d">[ BC ]</a>

Similar Messages

Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the rows? Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the records?
Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

The Oracle documentation has a good overview of the options available
Generating XML Data from the Database
Without knowing your version, I just picked 11.2, so you made need to look for that chapter in the documentation for your version to find applicable information.
You can also find some information in XML DB FAQ

How to update database table with key field?

Hello Experts,
I have a database table with following fields
MATNR   - Key
SSOUR - Key
KUNNR   - Key
MENG1
MENG2
MENG3
And this table contains records like...
MSD50001     R      1000001     5.30    2.30    5.25
MSD50002     R      1000002     5.30    2.30    5.25
MSD50003     R      1000003     5.30    2.30    5.25
MSD50005     R      1000004     5.30    2.30    5.25
MSD50004     R      1000005     5.30    2.30    5.25
I have an internal table with same fields of above database table.
MSD50001     A      1000001     5.30    2.30    5.25
MSD50002     A      1000002     5.30    2.30    5.25
MSD50003     A      1000003     5.30    2.30    5.25
MSD50005     A      1000004     5.30    2.30    5.25
MSD50004     A      1000005     5.30    2.30    5.25
MSD50006     A      1000006     5.30    2.30    5.25
I want to update the DB table with following internal table records.
If internal table records = db table records are same then Update....else insert from internal table to db table.
But here, SSOUR is key field so i am not able to use
MODIFY dbtab from itab.
It results me , entries in internal table are inserted into db table.
So i have double records.
Is there any statement which updates the key field? and if no fields in db table then insert it?
Regards
RH

hi,
u cannot update akey field.u can update only non key fields by using key field as a selection criteria.For example if u go to sm30 if u enter any table for inserting values, the key field will always be greyed out it is not succumbed to chnges.but u can insert value into ,if ur inserting and in the insert staemnet if u have key field which is already present in the table it will return sy-subrc = 4 otherwise it will insert the record and return 0.
eg for update :
UPDATE zdm_wtyprof SET upload_status = 'S'
                         WHERE spart = wa_upd-spart.
here spart is the key field and it will update the status field in the table as S.
if updated succesfully it will return 0.
SIMILARLY U CAN USE select.

Select data from database tables with high performance

hi all,
how to select data from different database tables with high performance.
im using for all entries instead of inner joins, even though burden on data base tables is going very high ( 90 % in se30)
hw to increase the performance.
kindly, reply.
thnks

Also Check you are not using open sql much like distict order by group by , use abap techniques on internal table to acive the same.
also Dont use select endselect.
if possible use up to n rows claus....
taht will limit the data base hits.
also dont run select in siode any loops.
i guess these are some of the trics oyu can use to avoid frequent DATA BASE HITS AND ABVOID THE DATA BASE LAOD.

Issue in adding not null constraint on 250 G table with 50 million rows.

Guys,
I need to add not null constraint on 2 column of a table with 50 million rows and ~250 GB in size, These 2 columns are newly added and I have also update the value for each of these columns to not null for each row.
After that I am adding not null constraint on these 2 columns this is taking 1 hour to complete, Is there any way to speed up this, I don't want to use ENABLE NOVALIDATE option or rather I can't use that option.

user445775 wrote:
Guys,
I need to add not null constraint on 2 column of a table with 50 million rows and ~250 GB in size, These 2 columns are newly added and I have also update the value for each of these columns to not null for each row.
After that I am adding not null constraint on these 2 columns this is taking 1 hour to complete, Is there any way to speed up this, I don't want to use ENABLE NOVALIDATE option or rather I can't use that option.And what's wrong with it taking an hour? Presumably, this is a one time operation, and it doesn't really interfere with anything else.

How to enter values to a database table with data type 'LCHR'?

Hello All,
I have created a database table with five fields, of which the last field is 'Value' field in which i have to enter text with a maximum of 1000characters. I have also specified a 'Length' field preceding the 'Value' field.
The datatype specified for the 'Length' field is 'INT2' and for the value field is 'LCHR' with length 1000.
When i tried inserting values to the table through a report, the report is giving an error that 'Literals that take up more than one line are not permitted', which means 255characters.
Also, when i entered a short text which is less than 250 characters to the 'Value' field,, the report is working fine and the table entry is visible.
Can anyone help me in this?
It would be really helpful if you explain how to use 'LCHR' data type in tables and how to enter values to such a field with more than 255characters.
Thanks in Advance,
Shino

Hi shino,
Alternatively,
you can use STRING instead of LCHR.
I too have tried with LCHR and am facing the same issue.. i am not sure why its coming..
i have tried STRING and its working perfectly.
wa-key_field = 'IN4'.
CONCATENATE 'Long text over 200 chars length '
              'another Long text over 200 chars length '
              'another Long text over 200 chars length '
              INTO wa-long_text.
wa-len = strlen( wa-long_text ).
INSERT zshino_table from wa.
SELECT * FROM zshino_table INTO table it.
clear wa.
LOOP AT it INTO wa.
      WRITE:/ wa-len, ' ' , wa-long_text.
ENDLOOP.
Also, remember to set the type of the database field as String.
hope this was of some help,
thanks and regards,
Sagar.

To split a table as two tables with equal no of records

Hi Everyone,
I have a table with size of 100000 records which is reterieved from different tables thats why idx is not in a sequential order. for example
idx
1
5
100 and so on
i want to create two different tables with same number of records but i dont want any duplicate records. Please anyone can help how can i get it.
Best Regards,

Please check the following script - hope that will help you --
SQL>
SQL> set serveroutput on
SQL>
SQL>
SQL> create table test_vaneeza
2 as
3 (
4 select 1 idx,'55 Mary Road' address from dual
5 union all
6 select 2, '10 Glenhuntly Road' from dual
7 union all
8 select 3, '5 Terrace' from dual
9 union all
10 select 100, '1212 Dandenong Road' from dual
11 union all
12 select 1000, '12 Liverpool Street' from dual
13 );
Table created.
SQL>
SQL>
SQL> set lin 1000
SQL>
SQL> select * from test_vaneeza;
 IDX ADDRESS
 1 55 Mary Road
 2 10 Glenhuntly Road
 3 5 Terrace
 100 1212 Dandenong Road
 1000 12 Liverpool Street
SQL>
SQL>
SQL> drop table gghh;
Table dropped.
SQL>
SQL>
SQL> create table gghh(a number(10),
2 b varchar2(100));
Table created.
SQL>
SQL>
SQL> drop table ggyy;
Table dropped.
SQL>
SQL>
SQL> create table ggyy(c number(10),
2 d varchar2(100));
Table created.
SQL>
SQL>
SQL> select count(*) from ggyy;
COUNT(*)
 0
SQL>
SQL>
SQL> select count(*) from gghh;
COUNT(*)
 0
SQL>
SQL> insert all
2 when rownum<=(select count(distinct idx) from test_vaneeza)/2
3 then
4 into gghh
5 else
6 into ggyy
7 select distinct idx,address
8 from test_vaneeza;
5 rows created.
SQL>
SQL>
SQL> commit;
Commit complete.
SQL>
SQL>
SQL> select count(*) from ggyy;
COUNT(*)
 3
SQL>
SQL>
SQL> select count(*) from gghh;
COUNT(*)
 2
SQL>
SQL>
SQL> select * from ggyy;
 C D
 3 5 Terrace
 100 1212 Dandenong Road
 1000 12 Liverpool Street
SQL>
SQL>
SQL> select * from gghh;
 A B
 1 55 Mary Road
 2 10 Glenhuntly Road
SQL> Regards.
Satyaki De.

Find - tables with largest number of records?

Hi,
I need to find tables with largest number of records. Any transaction show this details?
aRs

Go to transaction DB02, then click on the button that reads, "Space Statistics", the dialog box, click ok, leave the "*" for all tables, In the next screen put your cursor in the appropriate column labeled as Rows and click the sort button. Now you will see your biggest tables at the top of the list.
Regards,
Rich Heilman

Copying database table with data

Dear Team,
I want to copy a database table with all entries of that table. example: if i am copying MARA table as ZMARA i want all the entries of MARA should also exist in ZMARA.
Is it possible to do this without writing any piece of code.
Regards,
N.Jain

Hi,
It is not possible to do the same without coding.
Regards,
Sriram
Edited by: Srirama Murthy Maddirala on Jun 4, 2008 11:14 AM

Sample report for filling the database table with test data .

Hi ,
Can anyone provide me sample report for filling the database table with test data ?
Thanks ,
Abhi.

hi
the code
data : itab type table of Z6731_DEPTDETAIL,
wa type Z6731_DEPTDETAIL.
wa-DEPT_ID = 'z897hkjh'.
wa-DESCRIPTION = 'computer'.
append wa to itab.
wa-DEPT_ID = 'z897hkjhd'.
wa-DESCRIPTION = 'computer'.
append wa to itab.
loop at itab into wa.
insert z6731_DEPTDETAIL from wa.
endloop.
rewards if helpful

Importing a Partitioned Table with 10 Million Records.

I've been trying to import from a dump file using:
imp system/######@******** fromuser=fusr touser=tusr file=/f1/f2/expfl.dbf log=/o1/implg.log grants=N &
import done in US7ASCII character set and UTF8 NCHAR character set
import server uses UTF8 character set (possible charset conversion)
This contains a Table 'Tab_Mil_Rec', with almost 10 millions of records and has 10 partitions.
Done in 9i, on Solaris9.
Problem is the process abruptly ends at 'Tab_Mil_Rec'. This table is created but nothing is imported. I checked the log file, but it has logged events before this table, but nothing (not evens errors or termination message) after that. No errors are thrown even at os level, I don't know exactly because this was done as backgruond job.
Can anybody guess went wrong and whats the next step?

Hi,
Can you tried import partition by partition of this table ?
Cheers

Deleting records from a table with 12 million records

We need to delete some records on this table.
SQL> desc CDR_CLMS_ADMN.MDL_CLM_PMT_ENT_bak;
Name Null? Type
CLM_PMT_CHCK_NUM NOT NULL NUMBER(9)
CLM_PMT_CHCK_ACCT NOT NULL VARCHAR2(5)
CLM_PMT_PAYEE_POSTAL_EXT_CD VARCHAR2(4)
CLM_PMT_CHCK_AMT NUMBER(9,2)
CLM_PMT_CHCK_DT DATE
CLM_PMT_PAYEE_NAME VARCHAR2(30)
CLM_PMT_PAYEE_ADDR_LINE_1 VARCHAR2(30)
CLM_PMT_PAYEE_ADDR_LINE_2 VARCHAR2(30)
CLM_PMT_PAYEE_CITY VARCHAR2(19)
CLM_PMT_PAYEE_STATE_CD CHAR(2)
CLM_PMT_PAYEE_POSTAL_CD VARCHAR2(5)
CLM_PMT_SUM_CHCK_IND CHAR(1)
CLM_PMT_PAYEE_TYPE_CD CHAR(1)
CLM_PMT_CHCK_STTS_CD CHAR(2)
SYSTEM_INSERT_DT DATE
SYSTEM_UPDATE_DT
I only need to delete the records based on this condition
select * from CDR_CLMS_ADMN.MDL_CLM_PMT_ENT_bak
where CLM_PMT_CHCK_ACCT='00107' AND CLM_PMT_CHCK_NUM>=002196611 AND CLM_PMT_CHCK_NUM<=002197018;
Thsi table has 12 million records.
Please advise
Regards,
Narayan

user7202581 wrote:
We need to delete some records on this table.
SQL> desc CDR_CLMS_ADMN.MDL_CLM_PMT_ENT_bak;
Name Null? Type
CLM_PMT_CHCK_NUM NOT NULL NUMBER(9)
CLM_PMT_CHCK_ACCT NOT NULL VARCHAR2(5)
CLM_PMT_PAYEE_POSTAL_EXT_CD VARCHAR2(4)
CLM_PMT_CHCK_AMT NUMBER(9,2)
CLM_PMT_CHCK_DT DATE
CLM_PMT_PAYEE_NAME VARCHAR2(30)
CLM_PMT_PAYEE_ADDR_LINE_1 VARCHAR2(30)
CLM_PMT_PAYEE_ADDR_LINE_2 VARCHAR2(30)
CLM_PMT_PAYEE_CITY VARCHAR2(19)
CLM_PMT_PAYEE_STATE_CD CHAR(2)
CLM_PMT_PAYEE_POSTAL_CD VARCHAR2(5)
CLM_PMT_SUM_CHCK_IND CHAR(1)
CLM_PMT_PAYEE_TYPE_CD CHAR(1)
CLM_PMT_CHCK_STTS_CD CHAR(2)
SYSTEM_INSERT_DT DATE
SYSTEM_UPDATE_DT
I only need to delete the records based on this condition
select * from CDR_CLMS_ADMN.MDL_CLM_PMT_ENT_bak
where CLM_PMT_CHCK_ACCT='00107' AND CLM_PMT_CHCK_NUM>=002196611 AND CLM_PMT_CHCK_NUM<=002197018;
Thsi table has 12 million records.
Please advise
Regards,
NarayanDELETE from CDR_CLMS_ADMN.MDL_CLM_PMT_ENT_bak
where CLM_PMT_CHCK_ACCT='00107' AND CLM_PMT_CHCK_NUM>=002196611 AND CLM_PMT_CHCK_NUM<=002197018;

Update the database table with the content of the internal table

Hello!
I have the next form:
FORM erase_data.
SELECT * FROM zadrress INTO CORRESPONDING FIELDS OF TABLE itab_adrress.
DELETE TABLE itab_adrress: FROM zadrress,
 WITH TABLE KEY adrid = '456'.
WRITE 'The information after the DELETE operation'
COLOR 2.
LOOP AT itab_adrress INTO wa_adrress .
 WRITE:/
 wa_adrress-adrid COLOR 5,
 wa_adrress-name COLOR 7,
 wa_adrress-email COLOR 5,
 wa_adrress-depart COLOR 7,
 wa_adrress-display COLOR 5.
ENDLOOP.
SKIP.
LOOP AT itab_adrress INTO wa_adrress.
 MODIFY zadrress FROM wa_adrress.
 COMMIT WORK.
 IF sy-subrc = 0.
 WRITE 'OK !'.
 SKIP.
 ELSE.
 WRITE 'FAIL !'.
 SKIP.
 ENDIF.
ENDLOOP.
MODIFY zadrress FROM TABLE itab_adrress.
UPDATE zadrress FROM TABLE itab_adrress.
TRANSPORTING adrid, name, email, depart, display.
INSERT zadrress FROM TABLE itab_adrress ACCEPTING DUPLICATE KEYS.
PERFORM display_data .
ENDFORM. "erase_data
I see that my record is deleted when I display the records from my internal table, but
now I want to delete the record from database table.
 For that I want to move the content of the modified internal table in the database table. You can see the methods I tried (some of them commented), but nothing seems to work.
Any advice ?
Thank you.

FORM erase_data.
SELECT * FROM zadrress INTO CORRESPONDING FIELDS OF TABLE itab_adrress.
DELETE TABLE itab_adrress: FROM zadrress,
WITH TABLE KEY adrid = '456'.
WRITE 'The information after the DELETE operation'
COLOR 2.
LOOP AT itab_adrress INTO wa_adrress .
WRITE:/
wa_adrress-adrid COLOR 5,
wa_adrress-name COLOR 7,
wa_adrress-email COLOR 5,
wa_adrress-depart COLOR 7,
wa_adrress-display COLOR 5.
ENDLOOP.
SKIP.
LOOP AT itab_adrress INTO wa_adrress.
MODIFY zadrress FROM wa_adrress.
COMMIT WORK.
IF sy-subrc = 0.
WRITE 'OK !'.
SKIP.
ELSE.
WRITE 'FAIL !'.
SKIP.
ENDIF.
ENDLOOP.
DELETE FROM zadrress. "Make this change and try
MODIFY zadrress FROM TABLE itab_adrress.
UPDATE zadrress FROM TABLE itab_adrress.
TRANSPORTING adrid, name, email, depart, display.
INSERT zadrress FROM TABLE itab_adrress ACCEPTING DUPLICATE KEYS.
PERFORM display_data .
ENDFORM. "erase_data'.

Problems with retrieving data from tables with 240 and more records

Hi,
I've been connecting to Oracle 11g Server (not sure exact version) using Oracle 10.1.0 Client and O10 Oracle 10g driver. Everything was ok.
I installed Oracle 11.2.0 Client and I started to have problems with retrieving data from tables.
First I used the same connection string, driver and so on (O10 Oracle 10g) then I tried ORA Oracle but with no luck. The result is like this:
I'm able to connect to database. I'm able to retrieve data but from small tables (e.g. with 110 records it works perfectly using both O10 and ORA drivers). When I try to retrieve data from tables with like 240 and more records retrieval simply hangs (nothing happens at all - no error, no timeout). Application seems to hang forever.
I'm using Powerbuilder to connect to Database (either PB10.5 using O10 driver or PB12 using ORA driver). I used DBTrace, so I see that query hangs on the first FETCH.
So for the retrievals that hang I have something like:
(3260008): BIND SELECT OUTPUT BUFFER (DataWindow):(DBI_SELBIND) (0.186 MS / 18978.709 MS)
(3260008): ,len=160,type=DECIMAL,pbt=4,dbt=0,ct=0,prec=0,scale=0
(3260008): ,len=160,type=DECIMAL,pbt=4,dbt=0,ct=0,prec=0,scale=1
(3260008): ,len=160,type=DECIMAL,pbt=4,dbt=0,ct=0,prec=0,scale=0
(3260008): EXECUTE:(DBI_DW_EXECUTE) (192.982 MS / 19171.691 MS)
(3260008): FETCH NEXT:(DBI_FETCHNEXT)
and this is the last line,
while for retrievals that end, I have FETCH producing time, data in buffer and moving to the next Fetch until all data is retrieved
On the side note, I have no problems with retrieving data either by SQL Developer or DbVisualizer.
Problems started when I installed 11.2.0 Client. Even if I want to use 10.0.1 Client, the same problem occurs. So I guess something from 11.2.0 overrides 10.0.1 settings.
I will appreciate any comments/hints/help.
Thank you very much.

pgoel wrote:
I've been connecting to Oracle 11g Server (not sure exact version) using Oracle 10.1.0 Client and O10 Oracle 10g driver. Everything was ok.Earlier (before installing new stuff) did you ever try retrieving data from big tables (like 240 and more records), if yes, was it working?Yes, with Oracle 10g client (before installing 11g) I was able to retrieve any data, either it was 10k+ records or 100 records. Installing 11g client changed something that even using old 10g client (which I still have installed) fails to work. The same problem occur no matter I'm using 10g or 11g client now. Powerbuilder hangs on retrieving tables with more than like 240 records.
Thanks.

How to update database table with this new internal table

hi
i have a internal table having 8 records and it contains one record which was modified by user
not i want this new record to be updated to the database table which was modified by the user
how to do it pls suggest?
i want to use index for comarision as using index will not have any problems like promary key etc
like i want to capture the index for this table which was modified thr recoed index and use thi record indext to modify the database table
or any other suggestion
regards
nishant

Hi Nishant,
Have an additional field in a table to indicate a change with which you avoid in updating the record modified by user.
Regards,
Madhu.

Database table with potentially millions of records

Similar Messages

Maybe you are looking for