Update all rows in a table which has 8-10 million rows take for ever

Hi All,
Greetings!
I have to update 8million rows on a table. Basically have to reset the batch_id with the current batch number. it contains 8-10 million rows and i have tried with bulk update and then also it takes long time. below is the table structure
sales_stg (it has composite key of product,upc and market)
=======
product_id
upc
market_id
batch_id
process_status
I have to update batch_id,process_status to current batch_id (a number) and process_status as zero. I have to update all the rows with these values for batch_id = 0.
I tried bulk update an it takes more than 2hrs to do. (I limit the update to 1000).
Any help in this regard.
Naveen.

The fastest way will probably be to not use a select loop but a direct update like in William's example. The main downside is if you do too many rows you risk filling up your rollback/undo; to keep things as simple as possible I wouldn't do batching except for this. Also, we did some insert timings a few years ago on 9iR1 and found that the performance curve on frequent commits started to level off after 4K rows (fewer commits were still better) so you could see how performance improves by performing fewer commits if that's an issue.
The other thing you could consider if you have the license is using the parallel query option.

Similar Messages

How to write a cursor to check every row of a table which has millions of rows

Hello every one.
I need help. please... Below is the script (sample data), You can run directly on sql server management studio.
Here we need to update PPTA_Status column in Donation table. There WILL BE 3 statuses, A1, A2 and Q.
Here we need to update PPTA_status of January month donations only. We need to write a cursor. Here as this is a sample data we have only some donations (rows), but in the real table we have millions of rows. Need to check every row.
If i run the cursor for January, cursor should take every row, row by row all the rows of January.
we have donations in don_sample table, i need to check the test_results in the result_sample table for that donations and needs to update PPTA_status COLUMN.
We need to check all the donations of January month one by one. For every donation, we need to check for the 2 previous donations. For the previous donations, we need to in the following way. check
If we want to find previous donations of a donation, first look for the donor of that donation, then we can find previous donations of that donor. Like this we need to check for 2 previous donations.
If there are 2 previous donations and if they have test results, we need to update PPTA_STATUS column of this donatioh as 'Q'.
If 2 previous donation_numbers has test_code column in result_sample table as (9,10,11) values, then it means those donations has result.
BWX72 donor in the sample data I gave is example of above scenario
For the donation we are checking, if it has only 1 previous donation and it has a result in result_sample table, then set this donation Status as A2, after checking the result of this donation also.
ZBW24 donor in the sample data I gave is example of above scenario
For the donation we are checking, if it has only 1 previous donation and it DO NOT have a result in result_sample table, then set this donation Status as A1. after checking the result of this donation also.
PGH56 donor in the sample data I gave is example of above scenario
like this we need to check all the donations in don_sample table, it has millions of rows per every month.
we need to join don_sample and result_sample by donation_number. And we need to check for test_code column for result.
-- creating table
CREATE TABLE [dbo].[DON_SAMPLE](
[donation_number] [varchar](15) NOT NULL,
[donation_date] [datetime] NULL,
[donor_number] [varchar](12) NULL,
[ppta_status] [varchar](5) NULL,
[first_time_donation] [bit] NULL,
[days_since_last_donation] [int] NULL
) ON [PRIMARY]
--inserting values
Insert into [dbo].[DON_SAMPLE] ([donation_number],[donation_date],[donor_number],[ppta_status],[first_time_donation],[days_since_last_donation])
Select '27567167','2013-12-11 00:00:00.000','BWX72','A',1,0
Union ALL
Select '36543897','2014-12-26 00:00:00.000','BWX72','A',0,32
Union ALL
Select '47536542','2014-01-07 00:00:00.000','BWX72','A',0,120
Union ALL
Select '54312654','2014-12-09 00:00:00.000','JPZ41','A',1,0
Union ALL
Select '73276321','2014-12-17 00:00:00.000','JPZ41','A',0,64
Union ALL
Select '83642176','2014-01-15 00:00:00.000','JPZ41','A',0,45
Union ALL
Select '94527541','2014-12-11 00:00:00.000','ZBW24','A',0,120
Union ALL
Select '63497874','2014-01-13 00:00:00.000','ZBW24','A',1,0
Union ALL
Select '95786348','2014-12-17 00:00:00.000','PGH56','A',1,0
Union ALL
Select '87234156','2014-01-27 00:00:00.000','PGH56','A',1,0
--- creating table
CREATE TABLE [dbo].[RESULT_SAMPLE](
[test_result_id] [int] IDENTITY(1,1) NOT NULL,
[donation_number] [varchar](15) NOT NULL,
[donation_date] [datetime] NULL,
[test_code] [varchar](5) NULL,
[test_result_date] [datetime] NULL,
[test_result] [varchar](50) NULL,
[donor_number] [varchar](12) NULL
) ON [PRIMARY]
---SET IDENTITY_INSERT dbo.[RESULT_SAMPLE] ON
---- inserting values
Insert into [dbo].RESULT_SAMPLE( [test_result_id], [donation_number], [donation_date], [test_code], [test_result_date], [test_result], [donor_number])
Select 278453,'27567167','2013-12-11 00:00:00.000','0009','2014-01-20 00:00:00.000','N','BWX72'
Union ALL
Select 278454,'27567167','2013-12-11 00:00:00.000','0010','2014-01-20 00:00:00.000','NEG','BWX72'
Union ALL
Select 278455,'27567167','2013-12-11 00:00:00.000','0011','2014-01-20 00:00:00.000','N','BWX72'
Union ALL
Select 387653,'36543897','2014-12-26 00:00:00.000','0009','2014-01-24 00:00:00.000','N','BWX72'
Union ALL
Select 387654,'36543897','2014-12-26 00:00:00.000','0081','2014-01-24 00:00:00.000','NEG','BWX72'
Union ALL
Select 387655,'36543897','2014-12-26 00:00:00.000','0082','2014-01-24 00:00:00.000','N','BWX72'
UNION ALL
Select 378245,'73276321','2014-12-17 00:00:00.000','0009','2014-01-30 00:00:00.000','N','JPZ41'
Union ALL
Select 378246,'73276321','2014-12-17 00:00:00.000','0010','2014-01-30 00:00:00.000','NEG','JPZ41'
Union ALL
Select 378247,'73276321','2014-12-17 00:00:00.000','0011','2014-01-30 00:00:00.000','NEG','JPZ41'
UNION ALL
Select 561234,'83642176','2014-01-15 00:00:00.000','0081','2014-01-19 00:00:00.000','N','JPZ41'
Union ALL
Select 561235,'83642176','2014-01-15 00:00:00.000','0082','2014-01-19 00:00:00.000','NEG','JPZ41'
Union ALL
Select 561236,'83642176','2014-01-15 00:00:00.000','0083','2014-01-19 00:00:00.000','NEG','JPZ41'
Union ALL
Select 457834,'94527541','2014-12-11 00:00:00.000','0009','2014-01-30 00:00:00.000','N','ZBW24'
Union ALL
Select 457835,'94527541','2014-12-11 00:00:00.000','0010','2014-01-30 00:00:00.000','NEG','ZBW24'
Union ALL
Select 457836,'94527541','2014-12-11 00:00:00.000','0011','2014-01-30 00:00:00.000','NEG','ZBW24'
Union ALL
Select 587345,'63497874','2014-01-13 00:00:00.000','0009','2014-01-29 00:00:00.000','N','ZBW24'
Union ALL
Select 587346,'63497874','2014-01-13 00:00:00.000','0010','2014-01-29 00:00:00.000','NEG','ZBW24'
Union ALL
Select 587347,'63497874','2014-01-13 00:00:00.000','0011','2014-01-29 00:00:00.000','NEG','ZBW24'
Union ALL
Select 524876,'87234156','2014-01-27 00:00:00.000','0081','2014-02-03 00:00:00.000','N','PGH56'
Union ALL
Select 524877,'87234156','2014-01-27 00:00:00.000','0082','2014-02-03 00:00:00.000','N','PGH56'
Union ALL
Select 524878,'87234156','2014-01-27 00:00:00.000','0083','2014-02-03 00:00:00.000','N','PGH56'
select * from DON_SAMPLE
order by donor_number
select * from RESULT_SAMPLE
order by donor_number

You didn't mention the version of SQL Server. It's important, because SQL Server 2012 makes the job much easier (and will also run much faster, by dodging a self join). (As Kalman said, the OVER clause contributes to this answer).
Both approaches below avoid needing the cursor at all. (There was part of your explanation I didn't understand fully, but I think these suggestions work regardless)
Here's a SQL 2012 answer, using LAG() to lookup the previous 1 and 2 donation codes by Donor: (EDIT: I overlooked a couple things in this post: please refer to my follow-up post for the final/fixed answer. I'm leaving this post with my overlooked
items, for posterity).
With Results_Interim as
Select *
, count('x') over(partition by donor_number) as Ct_Donations
, Lag(test_code, 1) over(partition by donor_number order by donation_date ) as PrevDon1
, Lag(test_code, 2) over(partition by donor_number order by donation_date ) as PrevDon2
from RESULT_SAMPLE
Select *
, case when PrevDon1 in (9, 10, 11) and PrevDon2 in (9, 10, 11) then 'Q'
when PrevDon1 in (9, 10, 11) then 'A2'
when PrevDon1 is not null then 'A1'
End as NEWSTATUS
from Results_Interim
Where Test_result_Date >= '2014-01' and Test_result_Date < '2014-02'
Order by Donor_Number, donation_date
And a SQL 2005 or greater version, not using SQL 2012 new features
With Results_Temp as
Select *
, count('x') over(partition by donor_number) as Ct_Donations
, Row_Number() over(partition by donor_number order by donation_date ) as RN_Donor
from RESULT_SAMPLE
, Results_Interim as
Select R1.*, P1.test_code as PrevDon1, P2.Test_Code as PrevDon2
From Results_Temp R1
left join Results_Temp P1 on P1.Donor_Number = R1.Donor_Number and P1.Rn_Donor = R1.RN_Donor - 1
left join Results_Temp P2 on P2.Donor_Number = R1.Donor_Number and P2.Rn_Donor = R1.RN_Donor - 2
Select *
, case when PrevDon1 in (9, 10, 11) and PrevDon2 in (9, 10, 11) then 'Q'
when PrevDon1 in (9, 10, 11) then 'A2'
when PrevDon1 is not null then 'A1'
End as NEWSTATUS
from Results_Interim
Where Test_result_Date >= '2014-01' and Test_result_Date < '2014-02'
Order by Donor_Number, donation_date

To select table which has TAB as data column with in

Hi All
I have a table which has column type defined as TAB type. When I try selecting the table in PL/SQL developer, I got below error.
What should I do? Even SQL* Plus shows the same error!
SQL> select *
2 from my_tab_cap
3 /
from my_tab_cap
ERROR at line 2:
ORA-00904: : invalid identifierThanks in advance.
Message was edited by:
guru paran
Changed the table name.

You probably want to help us out with all your ddl:
create table tab2 (my_tab_col my_type ) nested table my_tab_col store as my_tab;
create type my_type as table of varchar2(100);
declare
v_type my_type := my_type();
begin
v_type.extend(4);
v_type(1) := 'I like table types1';
v_type(2) := 'I like table types2';
v_type(3) := 'I like table types3';
v_type(4) := 'I like table types4';
insert into tab2(my_tab_col) values (v_type);
end;
select * from tab2;
MY_TAB_COL
VARCHAR(I like table types1,I like table types2,I like table types3,I like table types4)

How to update table which has column name as Oracle keyword?

Hi All,
Somebody has created one table which has column name as "OPTION", now i am trying update this table column value and its throwing an error. "Invalid Identifier".
Could you please help me in this?
Thanks in advance.
Regards,
Ganesh Patil

check this :
batch@DOLN1> create table "from" ("select" varchar2(10));
Table created.
batch@DOLN1> desc "from";
Name Null? Type
select VARCHAR2(10)
batch@DOLN1> update "from" set "select" = 'hello World';
0 rows updated.
batch@DOLN1> insert into "from" values('hello World');
insert into "from" values('hello World')
ERROR at line 1:
ORA-12899: value too large for column "TCM_BATCH"."from"."select" (actual: 11, maximum: 10)
batch@DOLN1> update "from" set "select" = 'hello';
0 rows updated.
batch@DOLN1> insert into "from" values('hello');
1 row created.
batch@DOLN1> commit;
Commit complete.
batch@DOLN1> update "from" set "select" = 'World' where "select" = 'hello';
1 row updated.
batch@DOLN1> select * from "from";
select
World

Update row in a table based on join on multiple rows in another table

I am using SQL Server 2005. I have the following update query which is not working as desired.
UPDATE DocPlant
SET DocHistory = DocHistory + CONVERT(VARCHAR(20), PA.ActionDate, 100) + ' - ' + PA.ActionLog + '. '
FROM PlantDoc PD INNER JOIN PlantAction PA on PD.DocID = PA.DocID AND PD.PlantID = PA.PlantID
For each DocID and PlantID in PlantDoc table there are multiple rows in PlantAction table. I would like to concatenate ActionDate and ActionLog information into DocHistory column of DocPlant table. But the above update query is considering only one row from
PlantAction table even though there are multiple rows that match with DocID and PlantID.
DocHistory column is of type NVARCHAR(MAX).
How do I fix my query to achieve what I want ? Thanks for the help.

UPDATE DocPlant
SET DocHistory = DocHistory + CONVERT(VARCHAR(20), PA.ActionDate, 100) + ' - ' + PA.ActionLog + '. '
FROM PlantDoc PD INNER JOIN PlantAction PA on PD.DocID = PA.DocID AND PD.PlantID = PA.PlantID
We do not use the old Sybase UPDATE..FROM.. syntax. Google it and learn how it does not work. We do not use the old Sybase CONVERT() string function. You are still writing 1950's COBOL with string dates instead of temeproal data types.
You also did not post DDL, so we have to guess about everything. Does your boss make you work without DDL? How do you do it?
>> For each DocID and PlantID in PlantDoc table there are multiple rows in PlantAction [singular name?] table. I would like to concatenate ActionDate and ActionLog information into DocHistory column of DocPlant table. <<
Why? What does this new data element mean? This is like dividing Thursday by Red and expecting a reasonable answer. Now, non-SQL programmers who are still writing COBOL will violate the tiered architecture rule about doing display formatting in the database.
If you will follow forum rules, we can help you.
--CELKO-- Books in Celko Series for Morgan-Kaufmann Publishing: Analytics and OLAP in SQL / Data and Databases: Concepts in Practice Data / Measurements and Standards in SQL SQL for Smarties / SQL Programming Style / SQL Puzzles and Answers / Thinking
in Sets / Trees and Hierarchies in SQL

INSERTING DATA INTO A SQL SERVER 2005 TABLE, WHICH HAS A IDENTITY COLUMN

Hi All,
I have to insert the data into a SQL SERVER 2005 Database table.
I am able to insert the data into a normal SQL Server table.
When I am trying to insert the data into a SQL Server table, which has a identity column (i.e. auto increment column in Oracle) I am getting error saying that can't insert value explicitly when IDENTITY_INSERT is set to OFF.
Had anybody tried this??
There are some SRs on this issue, Oracle agreed that it is a bug. I am wondering if there is any workaround from any one of you (refer Insert in MS-SQL database table with IDENTITY COLUMN
Thanks
V Kumar

Even I had raised a SR on this in October 2008. But didn't get any solution for a long time, finally I had removed the identity column from the table. I can't to that now :).
I am using 10.1.3.3.0 and MS SQL SERVER 2005, They said it is working for MS SQL SERVER TABLE, if the identity column is not a primary key and asked me to refer to note 744735.1.
I had followed that note, but still it is not working for me.
But my requirement is it should work for a MS SQL SERVER 2005 table, which has identity column as primary key.
Thanks
V Kumar

Need to make a copy/clone of a table which has sdo_geometry data type

Folks,
We are running Oracle 11gr1 on Suse Linux platform.
We have a requirement, where we need to make a copy of a table, in the same database but in a difference schema, the source table has about 130 millions rows and also has the sdo_getometry data type.
I tried materialized view, but MV does not support sdo_geometry data type.
According to one of my DBAs, doing it via data pump , will take few days, which is not acceptable.
Is there a way, we can make a copy of this table in the same db ?
Thanks
Ashish

Siva,
This is not for backup purpose.
Here is the background information on this requirement.
I need to have a static copy of a table, which has spatial column, sdo_geometry in it.
So we can run another application against this table, just read-only, since we need static data, data in the source table changes quite often.
One of my DBA suggested to have another instance/database, to keep the copy of the source table, I thought idea of having another database just to keep a copy of a table is an over kill.
That is why I had asked this question and you had provided a viable solution.
Then we need to refresh this table , with the data (changes) from the source table, maybe once in 2- 3 weeks.
I tried creating a Materialized view, but it does not support Spatial columns.
So now I am trying to figure out, the most efficient way to refresh the clone table from the source table.
Regards,
Ashish

How to find tables which has more than 1000 Partitions

Hi All,
Is there any other way to find out the tables which has more than 1000 Partitions ?
Apart from SAP report RSDD_MSSQL_CUBEANALYZE. Because this report is not working for me as job getting cancel again and again with ABAP dump DBIF_DSQL2_SQL.I already check SAP Note 1309838, but itu2019s not applicable for us because we are on highest support package level SAP_BW 701 SP06.
Thanks,
Harshal

If you are running SQL Server as your database platform run this query:
select o.object_id,o.name,p.Partition_count from sys.objects o
inner join
(select object_id,count(distinct partition_number) as Partition_count
from sys.partitions
group by object_id)
p
on o.object_id = p.object_id
where o.type = 'U'
order by p.Partition_count desc

Always fill out the very last row of a table (which can expand)

Hi, I'm having problems with a button in table A that adds a row to another table (table B) and then automatically fills out some of the table row data (table A) into the newly created row of the other table (table B)
Here's my code (on click of button in a row of Table A) - I should also mention that Table A has a button to add more rows to itself:
This almost works - however, it only works if the amount of rows in each table match - which they don't necessarily as the user may or may not opt to add the row from table A to table B. eg. Table A could be 4 rows deep, but table B may only contain 2 Rows.
tableB.row.addInstance(0);
var vCol1 = this.resolveNodes("tableA.row.cellA");
var vCol2 = this.resolveNodes("tableB.row.cellA");
var vCol3 = this.resolveNodes("tableA.row.cellB");
var vCol4 = this.resolveNodes("tableB.row.cellB");
for (var i = 0; i < vCol1.length; i ++) {
             vCol2.item(i).rawValue = vCol1.item(i).rawValue;
             vCol4.item(i).rawValue = vCol3.item(i).rawValue;
I'm new to scripting (learning on the fly) so my question is, seeing that the first command is to add a new row to table B, can I alter the code so that var vCol2 and vCol4 always relate to the very latest row that has been created? In that way, any previous rows that have been added will shunt down the list and will not be affected but the new button click? Or is there a better way of doing this.

Hi,
I think the key here is that the addInstance method returns the row it has just added. So your add can be something like;
var button = xfa.host.messageBox("You are about to ADD this entry to Table B. Do you wish to proceed?", "Add to Table B Section", 1, 2);
if (button == 4)
   var newRow = tableB._tableBrow.addInstance(0);
   newRow.cellA.rawValue = cellA.rawValue;
   newRow.cellB.rawValue = cellB.rawValue;
I'm not sure how you want the table A remove button to work? Should it remove the first table B row with a matching cellA and cellB?
Regards
Bruce

Trying to access row values in a table which does not have any rows yet

try{
                         MappedRecord importParams = recordFactory.createMappedRecord("CONTAINER_OF_IMPORT_PARAMS");
                         IFunction function1 = client.getFunctionsMetaData().getFunction(funModGetDet);
                         IStructureFactory strucFact = interaction.retrieveStructureFactory();
                         response.write("try2 :"+pnumber);
                        IRecord structure = (IRecord) strucFact.getStructure(function1.getParameter("PERNR_TAB").getStructure());
                         response.write("try111 :"+pnumber);
                         structure.setString("PERNR",pnumber);
I am getting the following error "Trying to access row values in a table which does not have any rows yet " where PERNR_TAB is a table containing field "PERNR".
Can anybody help me out?

Please re-post this question in the appropriate forum. It seems to have nothing to do with Web Dynpro.

How to find table which has no index?

Hi,
How to find a table which has no index?
Please provide the query to find the above.
Thanks
Jafar

select owner,
       table_name
from dba_tables dt
where not exists(select 'x'
                    from dba_indexes di
                   where dt.owner=di.table_owner
                     and dt.table_name = di.table_name)

Just purchased a new pc. Downloaded Itunes. Downloaded some new music. Went to add the new music to my iphone off my new computer but it wants to wipe all of my existing music which has been purchased from itunes. HELP! What should i do?

Just purchased a new pc. Downloaded Itunes. Downloaded some new music. Went to add the new music to my iphone off my new computer but it wants to wipe all of my existing music which has been purchased from itunes. HELP! What should i do?

Kford_red wrote:
Feel a bit gutted as spent £££'s downloading music tonight and not been able to get the songs onto my phone so far!
The music you just downloaded tonight to your new computer is safe. The music you purchased on your phone is safe. You shouldn't feel gutted. It's your other music that you should have taken better care of. That would include any music you ripped yourself or purchased outside of iTunes. That you can't get back.
It has always been the user's responsibility to back up and protect downloaded music. If you didn't have a backup of your computer before it started acting funny, that's your fault. It would be no different than if you lost a music CD. It would not be a record store's responsibility to replace if you lost it. Digital music is no different. Similarly, a phone is not a backup medium. Phone's break, get misplaced or stolen all the time and are not a substitute for a backup copy of your music and media.
Computers WILL break. Hard drives WILL fail. That's why you back up your data.

Analyse a partitioned table with more than 50 million rows

Hi,
I have a partitioned table with more than 50 million rows. The last analyse is on 1/25/2007. Do I need to analyse him? (query runs on this table is very slow).
If I need to analyse him, what is the best way? Use DBMS_STATS and schedule a job?
Thanks

A partitioned table has global statistics as well as partition (and subpartition if the table is subpartitioned) statistics. My guess is that you mean to say that the last time that global statistics were gathered was in 2007. Is that guess accurate? Are the partition-level statistics more recent?
Do any of your queries actually use global statistics? Or would you expect that every query involving this table would specify one or more values for the partitioning key and thus force partition pruning to take place? If all your queries are doing partition pruning, global statistics are irrelevant, so it doesn't matter how old and out of date they are.
Are you seeing any performance problems that are potentially attributable to stale statistics on this table? If you're not seeing any performance problems, leaving the statistics well enough alone may be the most prudent course of action. Gathering statistics would only have the potential to change query plans. And since the cost of a query plan regressing is orders of magnitude greater than the benefit of a different query performing faster (at least for most queries in most systems), the balance of risks would argue for leaving the stats alone if there is no problem you're trying to solve.
If your system does actually use global statistics and there are performance problems that you believe are potentially attributable to stale global statistics and your partition level statistics are accurate, you can gather just global statistics on the table probably with a reasonably small sample size. Make sure, though, that you back up your existing statistics just in case a query plan goes south. Ideally, you'd also have a test environment with identical (or nearly identical) data volumes that you could use to verify that gathering statistics doesn't cause any problems.
Justin

General Scenario- Adding columns into a table with more than 100 million rows

I was asked/given a scenario, what issues do you encounter when you try to add new columns to a table with more than 200 million rows? How do you overcome those?
Thanks in advance.
svk

For such a large table, it is better to add the new column to the end of the table to avoid any performance impact, as RSingh suggested.
Also avoid to use any default on the newly created statement, or SQL Server will have to fill up 200 million fields with this default value. If you need one, add an empty column and update the column by using small batches (otherwise you lock up the whole
table). Add the default after all the rows have a value for the new column.

I have a Maxtor 3200 which has stopped communicating with my iMac for no reason can you help i'am running the latest system

I have a Maxtor 3200 which has stopped communicating with my iMac for no reason can you help i'am running the latest system

I have never seen this before. Try using these installers
Flash Player for ActiveX (Internet Explorer)
Flash Player Plug-in (All other browsers)

Update all rows in a table which has 8-10 million rows take for ever

Similar Messages

Maybe you are looking for