Rollbacking huge records
Hi,
we are getting a daily feed of 7 million records and need to be inserted to the oracle table.
After loading we identified that there were some issues in the data loaded and want to load new 7 milllion records.
a)first of all, database design should not allow incorrect data to be inserted.
b)After inserting due to design flaw, how we can rollback and insert new 7 million records?
Whats the best way to achieve that rather than going for a delete statement.
The table is partitioned based on month.
Thanks
Pandeesh wrote:
Hi,
we are getting a daily feed of 7 million records and need to be inserted to the oracle table.
After loading we identified that there were some issues in the data loaded and want to load new 7 milllion records.
a)first of all, database design should not allow incorrect data to be inserted.What determines "correct"?
>
b)After inserting due to design flaw, how we can rollback and insert new 7 million records?You can not ROLLBACK any data that has been COMMITTED.
>
Whats the best way to achieve that rather than going for a delete statement.How do you unring the bell?
>
The table is partitioned based on month.SO?
Similar Messages
-
Hi,
I have a requirement to fetch the huge records from prod DB.
Select query forming charter length is more than 10000, but command allows to execute only <2499 charters length.
How could resolve this.
Query eg:
Select column1, column2, column3,column4 from where (some join condition) AND (id='xxxx' OR id ='xxx' etc)As suggested already you can use external table.
Here is an example using EMP table from SCOTT schema.
Following is a sample file in which i have list of empno
SQL> !cat test.txt
7369
7499
7521
7566
7654
7698
7782
7788
7839
7844
7876The file is in the following directory
SQL> !pwd
/homeNow i create a Oracle directory object for the above directory.
SQL> create directory test_dir as '/home'
2 /
Directory created.Now i create a external table to query the file test.txt
SQL> create table test_et
2 (
3 empno integer
4 )
5 organization external
6 (
7 default directory test_dir
8 access parameters
9 (
10 records delimited by newline
11 )
12 location
13 (
14 'test.txt'
15 )
16 )
17 /
Table created.Now you can query the external table like this. It will pick up the data from test.txt
SQL> select * from test_et;
EMPNO
7369
7499
7521
7566
7654
7698
7782
7788
7839
7844
7876
11 rows selected.Now i can use it in my WHERE clause like this.
SQL> select * from emp where empno in (select empno from test_et);
EMPNO ENAME JOB MGR HIREDATE SAL COM DEPTNO
7369 SMITH CLERK 7902 02-APR-81 2975 0 20
7499 ALLEN SALESMAN 7698 20-FEB-81 1600 300 30
7521 WARD SALESMAN 7698 22-FEB-81 1250 500 30
7566 JONES MANAGER 7839 02-APR-81 2975 0 20
7654 MARTIN SALESMAN 7698 28-SEP-81 1250 1400 30
7698 BLAKE MANAGER 7839 01-MAY-81 2850 0 30
7782 CLARK MANAGER 7839 09-JUN-81 2450 0 10
7788 SCOTT ANALYST 7566 19-APR-87 3000 0 20
7839 KING PRESIDENT 17-NOV-81 5000 0 10
7844 TURNER SALESMAN 7698 08-SEP-81 1500 0 30
7876 ADAMS CLERK 7788 23-MAY-87 1100 0 20
11 rows selected.
SQL> -
Program times out while looping at internal table with huge records - Needs fine tuning suggestions
Hi,
I am trying to execute a report. It times out while looping at vbap internal table. This internal table has 140000 records and does the validation within this loop and geenrates data for the output. Due to this huge volume, the program times out when executed foreground.
There are no nested loops, so I cannot apply 'Parallel Cursor' here.
Is there any way I can fine tune the program so that it doesn't timeout? Is it possible to apply 'Parallel Processing' . If yes, how?
Thanks,
PavanHi Pavan ,
->sort your internal table by all primary key for vbap.
->Read a Record from the table (use your condition here)
->if record satisfys your where condition ,get that record index .
->loop the table from the index (without condition) .
its like parallel cursor only but for single loop .;-)
->use field symbols ,wherever possible .
if still dump is coming ,contact your basis team .
regards,
Krishna. -
How to Handle Huge record count in JDBC
Hi,
Can we process 20 lakhs records which would size near to 136 MB using JDBC sender channel in a single call and the same need to deliver to Target system which is JDBC too.
Kindly mention which way this can be achieved.
RegardsHi Venkata,
Kindly go through this below link to processing the huge amount of data from the sender JDBC.
http://scn.sap.com/people/peng.shen2/blog/2009/12/23/pi-how-to-handle-high-volume-data-per-jdbc-adapter
https://scn.sap.com/thread/2005124
http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/09/24/jdbc-receiver-scenarios-best-practices--how-to-handle-high-volume-load-stored-procedure-design-part6
Best Regards,
Monikandan. -
Hi Guys,
Can you please give me solution for the following issue.
I am deleting existing records from the database table. and adding new records to that same database table. After completion of deleting the records and add the new records to that data base table, user wants to get back the deleted records. So the deleted records will be add and new records will be deleted from the database table. Can you please give me solution for this process.
Thanks in advance,
Regards,
SaiHi sai,
Before adding the new reocrds:
use a selet query to get the already existing records into an internal table. Now delete these reocrds from the database table and add the new records.
Now as the user wants to add the old deleted records, you can make use of the above internal table records which contains the initial deleted records.
Regards,
Swarna Munukoti
Edited by: Swarna Munukoti on Nov 30, 2009 1:44 PM -
Friends,
500 Million Records,Client wants Ad-Hoc reporting.Is it possible with Essbase?Mehmet,
I will have to disagree with you slightly.
I would answer that it depends. 50 million recodrs could result in a much smaller database becaue everything is intersections. Think about a block storage DB where time and accounts are dense. If each record had one account and one time period, then it would condense down quite a bit. That said, I'll guess this is not the case, and without knowing more about the data, it is difficult if not impossible to answer you -
Can xcelsius handle huge records in excel?
I read the doc and it say no limit but I see some slowness and sometime the range won't even take affect. I created an excel pulling from sql server which has 7500 rows, I want to do a combo box on all of these, is there a way around this? since i dont see an
option to pull from database.Hi cssanangeles:
In theory, there will be no limit for the binding excel range, but the performance will decrease depends on the increases of binding excel records. You can also configure the u201CMaximum Limits Of Rowsu201D under u201CFile->Preference->Excel Optionu201D (the default suggested rows is u201C512u201D)
For you question you want to retrieve thousands of rows data from SQL server then binds these data to Combo Box. I can give you some suggestions:
1.Write a web service to retrieve data from sql server, then use Xcelsius u201Cweb service connectionu201D under u201CManage Connectionu201D section to consume the service and get the data from sql server to spread sheet.
2.If you use sql server 2005, you can use u201CReporting Serviceu201D to generate service for your data base, also then consume the data using Xcelsius u201CWeb Service Connectionu201D and do the same steps like previous option.
3.Xcelsius also provide you a tool to generate web service with easy way (without coding).
Using flynet, you can get flynet installation file under Xcelsius installation folder u201CXcelsius\Connectivity\Flynet Webservice Generatoru201D for more information about flynet please refer to
http://myxcelsius.com/2008/11/07/generating-web-services-for-xcelsius-using-flynet-web-service-generator/
Hope it helps with your work! And let me know if you need any further help:)
Thanks!
Bill -
Why I can not unlock the record when close a page?
I develop a jsp page as client,the jsp application use a business componment with statefull application module.i call a database rowset's lock function in jsp page to lock a record,now,after i close the jsp page without commit and rollback,the record is not unlocked,until i restart the oc4j application server.how can i unlock the record when close the page without commit and rollback?????
Li Ping:
If you lock a row in the course of a transaction, you cannot unlock the row until the end of transaction (either rollback or commit). DBMS doesn't allow this.
Alternatively, you can try to delay locking of the row. E.g., you can try using optimistic locking. Please see the Java Doc for oracle.jbo.Transaction if you're interested in optimistic locking.
Thanks.
Sung -
Container Managed XA transaction failing to rollback
We are running two distributed instances of WL 6.0 SP1, each with their own domain,
each accessing their own Oracle 8.1.5 database instance.
Server 1 has the following transaction configuration:
JDBC Transaction Pool Driver - weblogic.jdbc.oci.xa.XADataSource
Server 2 is using the oracle thin driver, a non-XA datasource, as it's JDBC datasource.
The JDBCTxDataSource has the attribute 'EnableTwoPhaseCommit' set to 'true'.
The deployment descriptor's <transaction-type> is set to 'container' and the <trans-attribute>
is set to 'required' in both of the server's session beans.
A test is performed where server 1 calls a method on server 2, via the remote
interface, to insert a record in server 2's database. A call is then made to
a session bean in Server 1 to insert a record in server 1's database. In the call
to Server 1's bean a system level error, 'EJBException', is thrown to test if
the XA transaction driver will rollback the record inserted during the initial
call to server 2 - it does not. However, if the method sessionContext.setRollbackOnly()
is explicity called in the catch () block of server 1's method, where the call
to server 2 was made, the rollback is successful on server 2.
Is there a problem using the oracle thin driver or can the problem be related
to the version of Oracle we are using, 8.1.5 and the fact it does not support
JDBC 2.0? An additional note - if all goes well both data sources are successfully
updated.
Thanks,
Keith Denson
We are running two distributed instances of WL 6.0 SP1, each with their own domain,
each accessing their own Oracle 8.1.5 database instance.
Server 1 has the following transaction configuration:
JDBC Transaction Pool Driver - weblogic.jdbc.oci.xa.XADataSource
Server 2 is using the oracle thin driver, a non-XA datasource, as it's JDBC datasource.
The JDBCTxDataSource has the attribute 'EnableTwoPhaseCommit' set to 'true'.
The deployment descriptor's <transaction-type> is set to 'container' and the <trans-attribute>
is set to 'required' in both of the server's session beans.
A test is performed where server 1 calls a method on server 2, via the remote
interface, to insert a record in server 2's database. A call is then made to
a session bean in Server 1 to insert a record in server 1's database. In the call
to Server 1's bean a system level error, 'EJBException', is thrown to test if
the XA transaction driver will rollback the record inserted during the initial
call to server 2 - it does not. However, if the method sessionContext.setRollbackOnly()
is explicity called in the catch () block of server 1's method, where the call
to server 2 was made, the rollback is successful on server 2.
Is there a problem using the oracle thin driver or can the problem be related
to the version of Oracle we are using, 8.1.5 and the fact it does not support
JDBC 2.0? An additional note - if all goes well both data sources are successfully
updated.
Thanks,
Keith Denson
-
Display records from database table
My database table has 495K records i want to display them all in se11 tcode and than download into excel ...but because of huge records it timeout.
Do I need to write a program for this?
Or is there any other smarter way.
Regards,
DNPHi,
Please follow the steps below:
1. Go to the selection screen of SE11 and provide your selection criteria (if required).
2. Click on Settings menu->User parameters->ALV Grid display.
3. Now click on Program menu->Execute in background->Provide output device name as LP01 or LOCAL or any valid printer parameters->Click continue->Click Immediate button->Click Save.
4. You can see the background job for this in SM37.
5. Once the background job is completed, it will appear in gleen color.
6. Click on the completed job and then click on Spool button.
7. Once the output is displayed in spool you can now easily download it to excel by clicking on System menu->List->Save->Local File.
Hope this will help you.
Regards,
Venkat -
Issue of grouping IDOC segments in huge loads
Hi All,
My Scenario is File to IDoc. No problem in posting IDoc in R3. But when i see the Idoc structure in R3, all the data records are grouped in different segements as seen below:-
This is how i see in we05.
IDoc No (00000000001234)
- Control Rec.
- Segments 000001 to 001000
- S01 Segment 000001 (Actual records)
- S02 Segment 000002
- S03 Segment 000003
- Segments 001001 to 001001
- S04 Segment 000004
- S05 Segment 000005
- S06 Segment 000006
- Status Rec.
I dont want the grouping of Idoc segments to take place. It should be under same data segment.
Please provide me your valuable solutions to resolve the same.
Note: This problem occurs only when processing huge records.
Thanks in advance,Hi Sriram,
It is the standard program flow of transaction WE02 which checks the number of segments.if the count is more than 1000,it creates a another group and displays it for the sake of readability.
Anyway it does not affect the inbound processing of the IDOC.
Thank you.
Regards,
Karun.M -
ROLLBACK SEGMENT의 MINEXTENTS를 20 이상으로 하면 좋은 이유
제품 : ORACLE SERVER
작성날짜 : 2003-06-19
ROLLBACK SEGMENT의 MINEXTENTS를 20 이상으로 하면 좋은 이유
=========================================================
PURPOSE
이 자료는 다음과 같은 주제에 대하여 소개하는 자료이다.
이 문서는 database application의 요구 사항을 충족시키기 위해 고려되어
져야 할 rollback segment tablespace 구성에 관한 내용을 담고 있다.
Creating, Optimizing, and Understanding Rollback Segments
-Rollback Segment 구성과 기록 방식
-Transaction에 Rollback Segment를 할당하는 Oracle 내부 메커니즘
-Rollback Segment 크기와 갯수
-Rollback Segment의 크기와 갯수 결정을 위한 테스트
-Rollback Segment extent의 크기와 갯수
-Rollback Segment의 minextents를 20 이상으로 하면 좋은 이유?
-Rollback Segment의 Optimal storage parameter와 Shrink
Explanation
Rollback Segment 구성과 기록 방식
Rollback segment는 extent라 불리는 연속적인 여러 개의 block으로 구성된다.
Rollback segment는 ordered circular 방식으로 extent를 쓰게 되는데,
current extent가 full이 되면 next extent로 옮겨 가며 사용하게 된다.
Transaction은 rollback segment 내의 current location에 record를 쓴 다음,
record의 size 만큼 current pointer를 옮겨 간다.
Rollback segment에 현재 record가 쓰여지고 있는 위치를 "Head"라고 한다.
또한, "Tail"이란 용어는 rollback segment에서 가장 오래된 active
transaction record의 시작 위치가 되는 부분을 말한다.
Transaction에 Rollback Segment를 할당하는 Oracle 내부 메커니즘
새로운 transaction이 rollback segment 를 요청하면, 각 rollback segment
를 이용하고 있는 active transaction 갯수를 확인하여 가장 적은 갯수의
active transaction 을 가진 rollback segment를 할당하게 된다.
Rollback segment는 transaction load를 처리하기에 충분한 크기를 가져야
하고, 필요한 만큼의 rollback segment를 사용할 수 있도록 적당한 갯수의
rollback segment를 가져야 한다.
1. 한 transaction은 단 하나의 rollback segment만을 사용할 수 있다.
2. 같은 extent에 여러 transaction이 기록할 수 있다.
3. Rollback segment의 Head는 Tail에 의해 현재 사용 중인 extent를
침범하지 않는다.
4. 링 형태로 구성되어 있는 rollback segment의 extent들은 다음 extent를
찾을 때 절대 건너 뛰는 일이 없으며, 순서를 뒤바꾸어 사용하지도 않는다.
5. Head가 next extent를 찾지 못하면, 새로운 extent를 추가로 할당하고,
그 extent를 링 안에 포함시킨다.
위와 같은 원리를 감안할 때, transaction size 뿐만 아니라 transaction
time도 상당히 중요한 고려 사항이라는 것을 알 수 있다.
Rollback Segment 크기와 갯수
Rollback segment size가 충분한지 판단하는 기준은 transaction activity에
직접적으로 영향을 받는다. 주로 일어나는 transaction activity에 근거하여
rollback segment size를 결정하여야 하고, 잘 일어나지 않는 특수한 경우의
큰 transaction이 문제라면 별도의 rollback segment로 관리되어야 한다.
Transaction 발생 중 Head가 너무 빨리 wrap around 시켜서 tail을 catch하
지 않도록 하여야 하며, 자주 변경되는 data에 대해 long-running query가
수행되었을 경우 read-consistency가 유지될 수 있도록 rollback segment
가 wrap around되지 않아야 한다.
Rollback segment 갯수를 적당히 잡아야 하는 이유는 process들 간에
contention을 방지하기 위함이고, V$WAITSTAT, V$ROLLSTAT, V$ROLLNAME
view를 통해서 contention을 확인할 수 있으며, 조회문은 다음과 같다.
sqlplus system/manager
select rn.name, (rs.waits/rs.gets) rbs_header_wait_ratio
from v$rollstat rs, v$rollname rn
where rs.usn = rn.usn
order by 1;
위의 query에 의해 조회된 rbs_header_wait_ratio 가 0.01 보다 크면,
rollback segment 갯수를 추가한다.
Rollback Segment의 크기와 갯수 결정을 위한 테스트
1. Rollback segment tablespace 생성
2. 테스트하기 위해 생성할 Rollback segment 갯수 결정
3. 같은 크기의 extent로 rollback segment 생성
extent 갯수는 최대 확장 시 10 - 30 개 정도가 되도록 extent 크기를 결정
4. Rollback segment의 minextents는 2이다.
5. 테스트할 rollback segment와 system rollback segment만 online 상태로 한다.
6. Transaction을 수행하고, 필요하면 application을 load한다.
7. Rollback segment contention을 확인한다.
8. Rollback segment가 최대 얼마까지 확장하는지 모니터링한다.
Rollback Segment extent의 크기와 갯수
Rollback segment가 자라나는 최대 사이즈를 알 수 있는데, 이 수치를
"minimum coverage size"라 한다. 만약, contention이 발생한다면 rollback
segment 갯수를 늘려 가면 테스트를 반복한다. 또한, extent 갯수가 10개
미만이나 30개 이상이 될 필요가 있다면 extent 크기를 늘리거나 줄이면서
테스트를 반복해 나가면 된다.
Rollback segment의 extent 크기를 정할 때, 각 extent는 모두 같은 크기로
생성할 것을 recommend한다.
Rollback tablespace의 크기는 extent size의 배수로 지정한다.
최적의 성능을 위한 rollback segment의 minextents는 20 이상이어야 한다.
Rollback Segment의 minextents를 20 이상으로 하면 좋은 이유?
Rollback segment는 dynamic하게 allocate되고, 더 이상 필요 없게 되었을 때
(만약, Optimal parameter가 셋팅되어 있으면) 모두 commit된 extent에
대해서는 optimal size 만큼만 남기고 release(deallocate)된다.
Rollback segment가 적은 수의 extent를 가질 수록, space 할당/해제 시
extent 수가 많을 때보다 큰 사이즈의 space가 할당되고, 해제된다.
다음과 같은 예를 들어 보자.
200M 정도의 rollback segment가 있는데, 100M 짜리 2개의 extent로 이루어져
있다고 가정해보자. 이 rollback segment에 추가로 space를 할당해야 할 일이
생겼을 때, 모든 rollback segment extent는 같은 크기를 가져야 한다는 점을
감안할 때, 100M 짜리 extent를 하나 더 할당해야 할 것이다.
이 결과 직전의 rollback segment 크기에 비하여 50% 만큼의 크기 증가분이
생겨나게 된 것인데, 실제 필요로 하는 space보다 더 많은 space가 할당되었을
것이다.
이와 반대로, 10M 짜리 extent 20개로 구성된 200M 짜리 rollback segment를
생각해보자.
여기에 추가로 space를 할당해야 할 일이 생겼을 때, 10M 짜리 extent 하나만
추가되면 되는 것이다.
Rollback segment가 20개 또는 그 이상의 extent로 구성되어 있다면 extent가
하나 더 증가할 경우가 생겼을 때, rollback segment의 전체 크기가 5% 이상은
늘어나지 않는다는 것이다.
즉, space의 할당과 해제 작업이 보다 유연하고 쉽게 일어날 수 있다.
요약하면, rollback segment의 extent 갯수를 20 이상으로 잡으면 space
할당과 해제가 "보다" 수월해진다.
실제로 extent 갯수를 20 이상으로 잡았을 때, 처리 속도가 훨씬 빨라진다는
사실이 많은 테스트 결과 밝혀졌다.
한가지 확실한 사실은, space를 할당하고 해제하는 작업은 cost가 적게 드는
작업이 아니라는 사실이다.
실제로 extent가 할당/해제되는 작업이 일어날 때, performance가 저하되는
일이 발생한다는 것이다.
Extent 하나에 대한 cost는 별 문제가 안 된다고 할지라도, rollback segment
는 끊임없이 space를 할당하고 해제하는 작업을 반복하기 때문에 작은 크기의
extent를 갖는 것이 cost 측면에서 훨씬 효율적이라는 결론이다.
Rollback Segment의 Optimal storage parameter와 Shrink
Optimal은 deallocate 시에 rollback segment 내에 optimal size 만큼의
extents를 유지하기 위해 사용하는 rollback segment storage parameter이다.
다음과 같은 명령으로 사용한다.
alter rollback segment r01 storage (optimal 1m);Optimal size는 storage 절 안에서 기술되어야 한다.
Optimal size 이상이 되면, 모두 commit된 extent에 대해서는 optimal size
만큼만 남기고 release된다.
즉, optimal에서 지정한 크기 만큼만 rollback segment를 유지하겠다는
뜻이며, 일정한 크기로 늘어났다가 다음번 tx이 해당 rbs를 취할 경우
optimal size만큼 resize하는 option이다.
rbs의 가장 최근에 사용된 extent가 다 차서 다른 extent를 요구할 때
이 optimal size와 rbs size를 비교하게 되며, 만약 rbs size가 더 크다면
active tx에 관여하지 않는 tail extent에 대하여 deallocation이 이루어진다.
특정 rollback segment가 너무 큰 space를 차지해서 다른 rollback segment가
extent를 발생할 수 있는 여유 공간을 부족하게 만들기 때문에 이를 극복하기
위해서 optimal size를 지정할 필요가 있다.
즉, optimal parameter를 지정하면 space availability 측면에서 효율적이다.
다음과 같이 shrink 명령을 수행하는데, size를 지정하지 않으면 optimal
size 만큼 shrink된다.
alter rollback segment [rbs_name] shrink to [size];Shrink 명령 수행 후, 바로 줄어들지 않는 경우가 있는데,
transaction이 있는 경우는 줄어들지 않고, transaction이 종료되면 줄어든다.
Optimal이 적용되는 시간은 session이 빠져 나가고 약 5~10 분 정도 걸린다.
적당한 OPTIMAL SIZE?
=> 20 ~ 30 extents 정도가 적당한데, batch job의 성격에 따라 size는 달라
지며 각 optimal의 합이 datafile의 size를 넘어도 전혀 상관없다.
Optimal size를 initial, next와 같게 주면 extent가 발생하는 매번 shrink가
일어나므로 좋지 않다.
RBS들의 평균 크기를 구하여 이것을 optimal 크기로 지정하여 사용하는 것을
권한다.
다음의 query를 이용하여 peak time에 rollback segment들의 평균 크기를 구한다.
select initial_extent + next_extent * (extents-1) "Rollback_size", extents
from dba_segments
where segment_type ='ROLLBACK';
이 크기의 평균값(bytes)을 rollback segment들의 optimal size로 사용할 수
있다.
주의할 사항은 너무 자주 shrink된다거나 optimal 값을 너무 작게 주면
ora-1555 : snapshot too old error가 발생할 확률이 높아지므로,
사용하지 않는 것이 좋을 수도 있고, 되도록 큰 값으로 셋팅해야 한다.
Rollback segment의 optimal size를 확인할 수 있는 view는 V$ROLLSTAT
이라는 dynamic view로서 OPTSIZE column에서 확인이 가능하다.
Example
none
Reference Documents
<Note:69464.1> -
BPC NW 7.0: Limit of records in Application
Hello Experts,
I have heard that there is a limit of records in BPC application (I mean fact table records) till 4 millions records. Is it true? What happens if this limit will be exceeded?
Thanks in advance.Hi Yulia,
I dont think there is any limitations on records in BPC. But yes, if there are huge records, then the performance to tool decreases and it takes loads of time in page opening, optimizing, sending data and also in displaying reports.
Therefore, it is suggested to keep data records minimum as per your requirement.
Hope this clarifies your question.
Rgds,
Poonam -
SQL Query to fetch records from tables which have 75+ million records
Hi,
I have the explain plan for a sql stmt.Require your suggestions to improve this.
PLAN_TABLE_OUTPUT
| Id | Operation | Name | Rows | Bytes | Cost |
| 0 | SELECT STATEMENT | | 340 | 175K| 19075 |
| 1 | TEMP TABLE TRANSFORMATION | | | | |
| 2 | LOAD AS SELECT | | | | |
| 3 | SORT GROUP BY | | 32M| 1183M| 799K|
| 4 | TABLE ACCESS FULL | CLM_DETAIL_PRESTG | 135M| 4911M| 464K|
| 5 | LOAD AS SELECT | | | | |
| 6 | TABLE ACCESS FULL | CLM_HEADER_PRESTG | 1 | 274 | 246K|
PLAN_TABLE_OUTPUT
| 7 | LOAD AS SELECT | | | | |
| 8 | SORT UNIQUE | | 744K| 85M| 8100 |
| 9 | TABLE ACCESS FULL | DAILY_PROV_PRESTG | 744K| 85M| 1007 |
| 10 | UNION-ALL | | | | |
| 11 | SORT UNIQUE | | 177 | 97350 | 9539 |
| 12 | HASH JOIN | | 177 | 97350 | 9538 |
| 13 | HASH JOIN OUTER | | 3 | 1518 | 9533 |
| 14 | HASH JOIN | | 1 | 391 | 8966 |
| 15 | TABLE ACCESS BY INDEX ROWID | CLM_DETAIL_PRESTG | 1 | 27 | 3 |
| 16 | NESTED LOOPS | | 1 | 361 | 10 |
| 17 | NESTED LOOPS OUTER | | 1 | 334 | 7 |
PLAN_TABLE_OUTPUT
| 18 | NESTED LOOPS OUTER | | 1 | 291 | 4 |
| 19 | VIEW | | 1 | 259 | 2 |
| 20 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66C9_DA2D01AD | 1 | 269 | 2 |
| 21 | INDEX RANGE SCAN | CLM_PAYMNT_CLMEXT_PRESTG_IDX | 1 | 32 | 2 |
| 22 | TABLE ACCESS BY INDEX ROWID| CLM_PAYMNT_CHKEXT_PRESTG | 1 | 43 | 3 |
| 23 | INDEX RANGE SCAN | CLM_PAYMNT_CHKEXT_PRESTG_IDX | 1 | | 2 |
| 24 | INDEX RANGE SCAN | CLM_DETAIL_PRESTG_IDX | 6 | | 2 |
| 25 | VIEW | | 32M| 934M| 8235 |
| 26 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66C8_DA2D01AD | 32M| 934M| 8235 |
| 27 | VIEW | | 744K| 81M| 550 |
| 28 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66CA_DA2D01AD | 744K| 81M| 550 |
PLAN_TABLE_OUTPUT
| 29 | TABLE ACCESS FULL | CCP_MBRSHP_XREF | 5288 | 227K| 5 |
| 30 | SORT UNIQUE | | 163 | 82804 | 9536 |
| 31 | HASH JOIN | | 163 | 82804 | 9535 |
| 32 | HASH JOIN OUTER | | 3 | 1437 | 9530 |
| 33 | HASH JOIN | | 1 | 364 | 8963 |
| 34 | NESTED LOOPS OUTER | | 1 | 334 | 7 |
| 35 | NESTED LOOPS OUTER | | 1 | 291 | 4 |
| 36 | VIEW | | 1 | 259 | 2 |
| 37 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66C9_DA2D01AD | 1 | 269 | 2 |
| 38 | INDEX RANGE SCAN | CLM_PAYMNT_CLMEXT_PRESTG_IDX | 1 | 32 | 2 |
| 39 | TABLE ACCESS BY INDEX ROWID | CLM_PAYMNT_CHKEXT_PRESTG | 1 | 43 | 3 |
PLAN_TABLE_OUTPUT
| 40 | INDEX RANGE SCAN | CLM_PAYMNT_CHKEXT_PRESTG_IDX | 1 | | 2 |
| 41 | VIEW | | 32M| 934M| 8235 |
| 42 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66C8_DA2D01AD | 32M| 934M| 8235 |
| 43 | VIEW | | 744K| 81M| 550 |
| 44 | TABLE ACCESS FULL | SYS_TEMP_0FD9D66CA_DA2D01AD | 744K| 81M| 550 |
| 45 | TABLE ACCESS FULL | CCP_MBRSHP_XREF | 5288 | 149K| 5 |
The CLM_DETAIL_PRESTG table has 100 million records and the CLM_HEADER_PRESTG table has 75 million records.
Any gussestions on how to getch huge records from tables of this size will help.
Regards,
NarayanWITH CLAIM_DTL
AS ( SELECT
ICN_NUM,
MIN (FIRST_SRVC_DT) AS FIRST_SRVC_DT,
MAX (LAST_SRVC_DT) AS LAST_SRVC_DT,
MIN (PLC_OF_SRVC_CD) AS PLC_OF_SRVC_CD
FROM CCP_STG.CLM_DETAIL_PRESTG CD WHERE ACT_CD <>'D'
GROUP BY ICN_NUM),
CLAIM_HDR
AS (SELECT
ICN_NUM,
SBCR_ID,
MBR_ID,
MBR_FIRST_NAME,
MBR_MI,
MBR_LAST_NAME,
MBR_BIRTH_DATE,
GENDER_TYPE_CD,
SBCR_RLTNSHP_TYPE_CD,
SBCR_FIRST_NAME,
SBCR_MI,
SBCR_LAST_NAME,
SBCR_ADDR_LINE_1,
SBCR_ADDR_LINE2,
SBCR_ADDR_CITY,
SBCR_ADDR_STATE,
SBCR_ZIP_CD,
PRVDR_NUM,
CLM_PRCSSD_DT,
CLM_TYPE_CLASS_CD,
AUTHO_NUM,
TOT_BILLED_AMT,
HCFA_DRG_TYPE_CD,
FCLTY_ADMIT_DT,
ADMIT_TYPE,
DSCHRG_STATUS_CD,
FILE_BILLING_NPI,
CLAIM_LOCATION_CD,
CLM_RELATED_ICN_1,
SBCR_ID||0
|| MBR_ID
|| GENDER_TYPE_CD
|| SBCR_RLTNSHP_TYPE_CD
|| MBR_BIRTH_DATE
AS MBR_ENROLL_ID,
SUBSCR_INSGRP_NM ,
CAC,
PRVDR_PTNT_ACC_ID,
BILL_TYPE,
PAYEE_ASSGN_CODE,
CREAT_RUN_CYC_EXEC_SK,
PRESTG_INSRT_DT
FROM CCP_STG.CLM_HEADER_PRESTG P WHERE ACT_CD <>'D' AND SUBSTR(CLM_PRCSS_TYPE_CD,4,1) NOT IN ('1','2','3','4','5','6') ),
PROV AS ( SELECT DISTINCT
PROV_ID,
PROV_FST_NM,
PROV_MD_NM,
PROV_LST_NM,
PROV_BILL_ADR1,
PROV_BILL_CITY,
PROV_BILL_STATE,
PROV_BILL_ZIP,
CASE WHEN PROV_SEC_ID_QL='E' THEN PROV_SEC_ID
ELSE NULL
END AS PROV_SEC_ID,
PROV_ADR1,
PROV_CITY,
PROV_STATE,
PROV_ZIP
FROM CCP_STG.DAILY_PROV_PRESTG),
MBR_XREF AS (SELECT SUBSTR(MBR_ENROLL_ID,1,17)||DECODE ((SUBSTR(MBR_ENROLL_ID,18,1)),'E','1','S','2','D','3')||SUBSTR(MBR_ENROLL_ID,19) AS MBR_ENROLLL_ID,
NEW_MBR_FLG
FROM CCP_STG.CCP_MBRSHP_XREF)
SELECT DISTINCT CLAIM_HDR.ICN_NUM AS ICN_NUM,
CLAIM_HDR.SBCR_ID AS SBCR_ID,
CLAIM_HDR.MBR_ID AS MBR_ID,
CLAIM_HDR.MBR_FIRST_NAME AS MBR_FIRST_NAME,
CLAIM_HDR.MBR_MI AS MBR_MI,
CLAIM_HDR.MBR_LAST_NAME AS MBR_LAST_NAME,
CLAIM_HDR.MBR_BIRTH_DATE AS MBR_BIRTH_DATE,
CLAIM_HDR.GENDER_TYPE_CD AS GENDER_TYPE_CD,
CLAIM_HDR.SBCR_RLTNSHP_TYPE_CD AS SBCR_RLTNSHP_TYPE_CD,
CLAIM_HDR.SBCR_FIRST_NAME AS SBCR_FIRST_NAME,
CLAIM_HDR.SBCR_MI AS SBCR_MI,
CLAIM_HDR.SBCR_LAST_NAME AS SBCR_LAST_NAME,
CLAIM_HDR.SBCR_ADDR_LINE_1 AS SBCR_ADDR_LINE_1,
CLAIM_HDR.SBCR_ADDR_LINE2 AS SBCR_ADDR_LINE2,
CLAIM_HDR.SBCR_ADDR_CITY AS SBCR_ADDR_CITY,
CLAIM_HDR.SBCR_ADDR_STATE AS SBCR_ADDR_STATE,
CLAIM_HDR.SBCR_ZIP_CD AS SBCR_ZIP_CD,
CLAIM_HDR.PRVDR_NUM AS PRVDR_NUM,
CLAIM_HDR.CLM_PRCSSD_DT AS CLM_PRCSSD_DT,
CLAIM_HDR.CLM_TYPE_CLASS_CD AS CLM_TYPE_CLASS_CD,
CLAIM_HDR.AUTHO_NUM AS AUTHO_NUM,
CLAIM_HDR.TOT_BILLED_AMT AS TOT_BILLED_AMT,
CLAIM_HDR.HCFA_DRG_TYPE_CD AS HCFA_DRG_TYPE_CD,
CLAIM_HDR.FCLTY_ADMIT_DT AS FCLTY_ADMIT_DT,
CLAIM_HDR.ADMIT_TYPE AS ADMIT_TYPE,
CLAIM_HDR.DSCHRG_STATUS_CD AS DSCHRG_STATUS_CD,
CLAIM_HDR.FILE_BILLING_NPI AS FILE_BILLING_NPI,
CLAIM_HDR.CLAIM_LOCATION_CD AS CLAIM_LOCATION_CD,
CLAIM_HDR.CLM_RELATED_ICN_1 AS CLM_RELATED_ICN_1,
CLAIM_HDR.SUBSCR_INSGRP_NM,
CLAIM_HDR.CAC,
CLAIM_HDR.PRVDR_PTNT_ACC_ID,
CLAIM_HDR.BILL_TYPE,
CLAIM_DTL.FIRST_SRVC_DT AS FIRST_SRVC_DT,
CLAIM_DTL.LAST_SRVC_DT AS LAST_SRVC_DT,
CLAIM_DTL.PLC_OF_SRVC_CD AS PLC_OF_SRVC_CD,
PROV.PROV_LST_NM AS BILL_PROV_LST_NM,
PROV.PROV_FST_NM AS BILL_PROV_FST_NM,
PROV.PROV_MD_NM AS BILL_PROV_MID_NM,
PROV.PROV_BILL_ADR1 AS BILL_PROV_ADDR1,
PROV.PROV_BILL_CITY AS BILL_PROV_CITY,
PROV.PROV_BILL_STATE AS BILL_PROV_STATE,
PROV.PROV_BILL_ZIP AS BILL_PROV_ZIP,
PROV.PROV_SEC_ID AS BILL_PROV_EIN,
PROV.PROV_ID AS SERV_FAC_ID ,
PROV.PROV_ADR1 AS SERV_FAC_ADDR1 ,
PROV.PROV_CITY AS SERV_FAC_CITY ,
PROV.PROV_STATE AS SERV_FAC_STATE ,
PROV.PROV_ZIP AS SERV_FAC_ZIP ,
CHK_PAYMNT.CLM_PMT_PAYEE_ADDR_LINE_1,
CHK_PAYMNT.CLM_PMT_PAYEE_ADDR_LINE_2,
CHK_PAYMNT.CLM_PMT_PAYEE_CITY,
CHK_PAYMNT.CLM_PMT_PAYEE_STATE_CD,
CHK_PAYMNT.CLM_PMT_PAYEE_POSTAL_CD,
CLAIM_HDR.CREAT_RUN_CYC_EXEC_SK
FROM CLAIM_DTL,(select * FROM CCP_STG.CLM_DETAIL_PRESTG WHERE ACT_CD <>'D') CLM_DETAIL_PRESTG, CLAIM_HDR,CCP_STG.MBR_XREF,PROV,CCP_STG.CLM_PAYMNT_CLMEXT_PRESTG CLM_PAYMNT,CCP_STG.CLM_PAYMNT_CHKEXT_PRESTG CHK_PAYMNT
WHERE
CLAIM_HDR.ICN_NUM = CLM_DETAIL_PRESTG.ICN_NUM
AND CLAIM_HDR.ICN_NUM = CLAIM_DTL.ICN_NUM
AND CLAIM_HDR.ICN_NUM=CLM_PAYMNT.ICN_NUM(+)
AND CLM_PAYMNT.CLM_PMT_CHCK_ACCT=CHK_PAYMNT.CLM_PMT_CHCK_ACCT
AND CLM_PAYMNT.CLM_PMT_CHCK_NUM=CHK_PAYMNT.CLM_PMT_CHCK_NUM
AND CLAIM_HDR.MBR_ENROLL_ID = MBR_XREF.MBR_ENROLLL_ID
AND CLM_DETAIL_PRESTG.FIRST_SRVC_DT >= 20110101
AND MBR_XREF.NEW_MBR_FLG = 'Y'
AND PROV.PROV_ID(+)=SUBSTR(CLAIM_HDR.PRVDR_NUM,6)
AND MOD(SUBSTR(CLAIM_HDR.ICN_NUM,14,2),2)=0
UNION ALL
SELECT DISTINCT CLAIM_HDR.ICN_NUM AS ICN_NUM,
CLAIM_HDR.SBCR_ID AS SBCR_ID,
CLAIM_HDR.MBR_ID AS MBR_ID,
CLAIM_HDR.MBR_FIRST_NAME AS MBR_FIRST_NAME,
CLAIM_HDR.MBR_MI AS MBR_MI,
CLAIM_HDR.MBR_LAST_NAME AS MBR_LAST_NAME,
CLAIM_HDR.MBR_BIRTH_DATE AS MBR_BIRTH_DATE,
CLAIM_HDR.GENDER_TYPE_CD AS GENDER_TYPE_CD,
CLAIM_HDR.SBCR_RLTNSHP_TYPE_CD AS SBCR_RLTNSHP_TYPE_CD,
CLAIM_HDR.SBCR_FIRST_NAME AS SBCR_FIRST_NAME,
CLAIM_HDR.SBCR_MI AS SBCR_MI,
CLAIM_HDR.SBCR_LAST_NAME AS SBCR_LAST_NAME,
CLAIM_HDR.SBCR_ADDR_LINE_1 AS SBCR_ADDR_LINE_1,
CLAIM_HDR.SBCR_ADDR_LINE2 AS SBCR_ADDR_LINE2,
CLAIM_HDR.SBCR_ADDR_CITY AS SBCR_ADDR_CITY,
CLAIM_HDR.SBCR_ADDR_STATE AS SBCR_ADDR_STATE,
CLAIM_HDR.SBCR_ZIP_CD AS SBCR_ZIP_CD,
CLAIM_HDR.PRVDR_NUM AS PRVDR_NUM,
CLAIM_HDR.CLM_PRCSSD_DT AS CLM_PRCSSD_DT,
CLAIM_HDR.CLM_TYPE_CLASS_CD AS CLM_TYPE_CLASS_CD,
CLAIM_HDR.AUTHO_NUM AS AUTHO_NUM,
CLAIM_HDR.TOT_BILLED_AMT AS TOT_BILLED_AMT,
CLAIM_HDR.HCFA_DRG_TYPE_CD AS HCFA_DRG_TYPE_CD,
CLAIM_HDR.FCLTY_ADMIT_DT AS FCLTY_ADMIT_DT,
CLAIM_HDR.ADMIT_TYPE AS ADMIT_TYPE,
CLAIM_HDR.DSCHRG_STATUS_CD AS DSCHRG_STATUS_CD,
CLAIM_HDR.FILE_BILLING_NPI AS FILE_BILLING_NPI,
CLAIM_HDR.CLAIM_LOCATION_CD AS CLAIM_LOCATION_CD,
CLAIM_HDR.CLM_RELATED_ICN_1 AS CLM_RELATED_ICN_1,
CLAIM_HDR.SUBSCR_INSGRP_NM,
CLAIM_HDR.CAC,
CLAIM_HDR.PRVDR_PTNT_ACC_ID,
CLAIM_HDR.BILL_TYPE,
CLAIM_DTL.FIRST_SRVC_DT AS FIRST_SRVC_DT,
CLAIM_DTL.LAST_SRVC_DT AS LAST_SRVC_DT,
CLAIM_DTL.PLC_OF_SRVC_CD AS PLC_OF_SRVC_CD,
PROV.PROV_LST_NM AS BILL_PROV_LST_NM,
PROV.PROV_FST_NM AS BILL_PROV_FST_NM,
PROV.PROV_MD_NM AS BILL_PROV_MID_NM,
PROV.PROV_BILL_ADR1 AS BILL_PROV_ADDR1,
PROV.PROV_BILL_CITY AS BILL_PROV_CITY,
PROV.PROV_BILL_STATE AS BILL_PROV_STATE,
PROV.PROV_BILL_ZIP AS BILL_PROV_ZIP,
PROV.PROV_SEC_ID AS BILL_PROV_EIN,
PROV.PROV_ID AS SERV_FAC_ID ,
PROV.PROV_ADR1 AS SERV_FAC_ADDR1 ,
PROV.PROV_CITY AS SERV_FAC_CITY ,
PROV.PROV_STATE AS SERV_FAC_STATE ,
PROV.PROV_ZIP AS SERV_FAC_ZIP ,
CHK_PAYMNT.CLM_PMT_PAYEE_ADDR_LINE_1,
CHK_PAYMNT.CLM_PMT_PAYEE_ADDR_LINE_2,
CHK_PAYMNT.CLM_PMT_PAYEE_CITY,
CHK_PAYMNT.CLM_PMT_PAYEE_STATE_CD,
CHK_PAYMNT.CLM_PMT_PAYEE_POSTAL_CD,
CLAIM_HDR.CREAT_RUN_CYC_EXEC_SK
FROM CLAIM_DTL, CLAIM_HDR,MBR_XREF,PROV,CCP_STG.CLM_PAYMNT_CLMEXT_PRESTG CLM_PAYMNT,CCP_STG.CLM_PAYMNT_CHKEXT_PRESTG CHK_PAYMNT
WHERE CLAIM_HDR.ICN_NUM = CLAIM_DTL.ICN_NUM
AND CLAIM_HDR.ICN_NUM=CLM_PAYMNT.ICN_NUM(+)
AND CLM_PAYMNT.CLM_PMT_CHCK_ACCT=CHK_PAYMNT.CLM_PMT_CHCK_ACCT
AND CLM_PAYMNT.CLM_PMT_CHCK_NUM=CHK_PAYMNT.CLM_PMT_CHCK_NUM
AND CLAIM_HDR.MBR_ENROLL_ID = MBR_XREF.MBR_ENROLLL_ID
-- AND TRUNC(CLAIM_HDR.PRESTG_INSRT_DT) = TRUNC(SYSDATE)
AND CLAIM_HDR.CREAT_RUN_CYC_EXEC_SK = 123638.000000000000000
AND MBR_XREF.NEW_MBR_FLG = 'N'
AND PROV.PROV_ID(+)=SUBSTR(CLAIM_HDR.PRVDR_NUM,6)
AND MOD(SUBSTR(CLAIM_HDR.ICN_NUM,14,2),2)=0; -
CTAS, INSERT million of records
Hi All,
I have a table which has approx 20 million record, I will delete about 10 million record. After some research, I decided to perform CTAS + TRUNCATE rather than DELETE.
I will be executing the following statements, TEMP_TABLE will retain the record I wish to keep.
1. CREATE TEMP_TABLE AS ( SELECT * FROM ACTUAL_TABLE WHERE DATE BETWEEN X AND Y );
2. TRUNCATE TABLE ACTUAL_TABLE;
3. INSERT /*+ append */ INTO ACTUAL_TABLE (SELECT * FROM TEMP_TABLE);
4. DROP TABLE TEMP_TABLE;
My production database is running on Oracle 10g, I am worried the INSERT operation might fail due to the large amount of records to be transferred (Est. approx. 10 million records)
I have read by adding append hints might help during the insert process and UNDO tablespace might not be sufficient for huge record inserts.
How can I improve the process and avoid any possible failures?
Please kindly advise.
Thank you.What do you mean by your INSERT will fail. Is your UNDO Tablespace not AUTO EXTENDABLE?
Another option is instead of doing the insert, you can
1. drop the ACTUAL_TABLE
2. Rename the TEMP_TABLE as ACTUAL_TABLE
3. Re Create the indexes, constraints, Triggers on the table back.
Edited by: Karthick_Arp on Apr 29, 2009 10:15 PM
Maybe you are looking for
-
Link to Communication Channel and Adapter Monitoring not opening
Hello All, we are on PI 7.0. In teh RWB, we are not able to see the Communication Channnel Monitoring as well the Adapter Monitoring It is giving Page cannot be Displayed. please tell us the way ahead. Regards Chakra and Somnath
-
Songs in my Itunes show up as separate folders. How do I combine them into one album?
Songs in my Itunes show up as separate folders. How do I combine them into one album?
-
Hi I am new to iWeb, having just purchased a new iMac with iLife '08 installed. I had been using a shareware program called "Better HTML Export" with iPhoto '04 to quickly create photo albums for my website,which worked well, but I have been looking
-
Can I output two streams of video from a 2 channel multi clip? I need each track to go to a diffrent monitor but they must be kept perfectly in sync during playback.
-
hi all, what is this output type?why it is needed? i have developed a print prog for exise invoice but not used output type anywhere to see the output of the form. so my question is when this output type is reqired and when not? whther i should creat