Select count(*) where exists (takes 5 hours).

Hello Gurus,
I have two databases on two servers, I am counting how many rows are similiar on two tables that are identical, and the rows should be identical as well. I am running a select count(*) where exists query and it takes 5 hours to complete.
Each table only has two million rows.
What can I do to speed it up?

5 hours to process 2M rows does sound a bit long :(
I didn't see this mentioned explicitly, but I thought the idea of comparing data on 2 servers implied a database link. Tuning distributed queries can be nasty.
Start by getting an execution plan of the query to figure out what it is doing. Compare that to the plan generated by the already suggested MINUS operator. You'll need to do MINUS twice with each query in the other's position the second time. Alternately, check the indexing on the subqueries; EXISTS tends to work best with fast indexed lookups. FTS on an EXISTS subquery is not good :(
Think about copying the data locally to one system or the other first, maybe in a materialized view or even global temporary table.
Finally, think about tuning the transfer. There are articles on Metalink on tuning the transfer packet sizes (SDU/TDU) which might help IF YOU ARE ON UNIX; I haven't had any luck changing these values on Windows. You can also look into setting tcp.nodelay which can affect when packets get sent (another Metalink article should cover this).

Similar Messages

SELECT COUNT(*) WHERE Receiver JDBC?

How can I do a SELECT COUNT(*) WHERE SQL statement with a Receiver JDBC?

Hi,
Refer this document - but if you have select query
"All return values are returned in an XML structure."
http://help.sap.com/saphelp_nw04/helpdata/en/2e/96fd3f2d14e869e10000000a155106/content.htm
Hope this helps,
Regards,
Moorthy

"select count(*) from dba_jobs_running" takes 5 minutes to return

Hi,
I login as sys or system from sql*plus to run query "select count(*) from dba_jobs_running". This query takes about 5 minutes to run.
Querying other DBA_* views returns instantly, only this dba_jobs_running view is troubling.
I have 11 records in dba_jobs, and 18 records in v$lock as usual.
No error or warning in logfile, no trace file.
What's possibly wrong with the server?
Thanks,
Harry

Hi,
The following are the exact statements I used to create the trace file:
alter session set timed_statistics=true;
alter session set max_dump_file_size=unlimited;
alter session set events '10046 trace name context forever, level 12';
select count(*) from dba_jobs_running;
exit;
The content of the trace file is attached at the end.
If it is CPU bound, I have no activity on the db host. I am solely using it to analyze this problem.
load averages: 0.02, 0.02, 0.07 22:16:38
48 processes: 47 sleeping, 1 on cpu
CPU states: 98.4% idle, 1.6% user, 0.0% kernel, 0.0% iowait, 0.0% swap
Memory: 1792M real, 983M free, 517M swap in use, 2937M swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
655 oracle 1 59 0 0K 0K sleep 1:14 0.32% oracle
865 oracle 1 59 0 2072K 1184K cpu 0:00 0.14% top
667 oracle 1 59 0 0K 0K sleep 0:13 0.08% oracle
663 oracle 1 59 0 0K 0K sleep 0:27 0.07% oracle
665 oracle 1 59 0 0K 0K sleep 0:15 0.02% oracle
671 oracle 1 59 0 0K 0K sleep 0:00 0.02% oracle
645 oracle 18 59 0 0K 0K sleep 0:23 0.01% oracle
657 oracle 1 59 0 0K 0K sleep 0:14 0.00% oracle
649 oracle 1 59 0 0K 0K sleep 0:02 0.00% oracle
659 oracle 1 59 0 0K 0K sleep 0:01 0.00% oracle
251 root 20 59 0 3256K 2512K sleep 0:00 0.00% nscd
70 root 5 59 0 2864K 2096K sleep 0:00 0.00% picld
224 root 3 59 0 3912K 2008K sleep 0:00 0.00% automountd
271 root 1 59 0 4408K 1896K sleep 0:00 0.00% sendmail
588 oracle 1 59 0 2600K 1864K sleep 0:00 0.00% bash
Any help is greatly appreciated.
<-- TRACE FILE CONTENT -->
*** 2006-11-21 21:34:54.413
*** SESSION ID:(21.832) 2006-11-21 21:34:54.412
APPNAME mod='[email protected] (TNS V1-V3)' mh=0 act='' ah=0
=====================
PARSING IN CURSOR #1 len=69 dep=0 uid=0 oct=42 lid=0 tim=107123626140 hv=2004533713 ad='97219d80'
alter session set events '10046 trace name context forever, level 12'
END OF STMT
EXEC #1:c=0,e=346,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=4,tim=107123625473
WAIT #1: nam='SQL*Net message to client' ela= 11 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message from client' ela= 6476024 p1=1650815232 p2=1 p3=0
=====================
PARSING IN CURSOR #1 len=37 dep=0 uid=0 oct=3 lid=0 tim=107130104260 hv=2246554324 ad='97c21958'
select count(*) from dba_jobs_running
END OF STMT
PARSE #1:c=0,e=548,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4,tim=107130104237
BINDS #1:
EXEC #1:c=0,e=1412,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4,tim=107130106034
WAIT #1: nam='SQL*Net message to client' ela= 12 p1=1650815232 p2=1 p3=0
*** 2006-11-21 21:40:53.931
FETCH #1:c=350760000,e=344612113,p=0,cr=2,cu=16652,mis=0,r=1,dep=0,og=4,tim=107474718456
WAIT #1: nam='SQL*Net message from client' ela= 1936 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=6,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=0,tim=107474721309
WAIT #1: nam='SQL*Net message to client' ela= 8 p1=1650815232 p2=1 p3=0
*** 2006-11-21 21:56:30.972
WAIT #1: nam='SQL*Net message from client' ela= 915075702 p1=1650815232 p2=1 p3=0
XCTEND rlbk=0, rd_only=1
STAT #1 id=1 cnt=1 pid=0 pos=1 obj=0 op='SORT AGGREGATE (cr=2 r=0 w=0 time=100535896 us)'
STAT #1 id=2 cnt=0 pid=1 pos=1 obj=0 op='NESTED LOOPS (cr=2 r=0 w=0 time=100535876 us)'
STAT #1 id=3 cnt=5550 pid=2 pos=1 obj=0 op='MERGE JOIN CARTESIAN (cr=2 r=0 w=0 time=128639 us)'
STAT #1 id=4 cnt=10 pid=3 pos=1 obj=0 op='NESTED LOOPS OUTER (cr=2 r=0 w=0 time=5839 us)'
STAT #1 id=5 cnt=10 pid=4 pos=1 obj=69 op='FIXED TABLE FULL X$KSQRS (cr=0 r=0 w=0 time=4790 us)'
STAT #1 id=6 cnt=10 pid=4 pos=2 obj=202 op='INDEX UNIQUE SCAN OBJ#(202) (cr=2 r=0 w=0 time=561 us)'
STAT #1 id=7 cnt=5550 pid=3 pos=2 obj=0 op='BUFFER SORT (cr=0 r=0 w=0 time=90039 us)'
STAT #1 id=8 cnt=555 pid=7 pos=1 obj=16 op='FIXED TABLE FULL X$KSUSE (cr=0 r=0 w=0 time=1277 us)'
STAT #1 id=9 cnt=0 pid=2 pos=2 obj=0 op='VIEW (cr=0 r=0 w=0 time=344148954 us)'
STAT #1 id=10 cnt=105477 pid=9 pos=1 obj=0 op='UNION-ALL (cr=0 r=0 w=0 time=343814331 us)'
STAT #1 id=11 cnt=105476 pid=10 pos=1 obj=0 op='VIEW (cr=0 r=0 w=0 time=203487737 us)'
STAT #1 id=12 cnt=105476 pid=11 pos=1 obj=0 op='UNION-ALL (cr=0 r=0 w=0 time=202961626 us)'
STAT #1 id=13 cnt=0 pid=12 pos=1 obj=272 op='FIXED TABLE FULL X$KDNSSF (cr=0 r=0 w=0 time=13633786 us)'
STAT #1 id=14 cnt=105476 pid=12 pos=2 obj=72 op='FIXED TABLE FULL X$KSQEQ (cr=0 r=0 w=0 time=188317145 us)'
STAT #1 id=15 cnt=0 pid=10 pos=2 obj=253 op='FIXED TABLE FULL X$KTADM (cr=0 r=0 w=0 time=76301426 us)'
STAT #1 id=16 cnt=1 pid=10 pos=3 obj=254 op='FIXED TABLE FULL X$KTCXB (cr=0 r=0 w=0 time=62693462 us)'
STAT #1 id=16 cnt=1 pid=10 pos=3 obj=254 op='FIXED TABLE FULL X$KTCXB (cr=0 r=0 w=0 time=62693462 us)'
<---- END of TRACE FILE -->

Select count(*) from table in oracle 11g with direct path read takes time

select count(*) from table takes long time, even more than couple of hours..
direct path read is the wait event which is almost is at 99%..
can u someone provide some info on this.. on solution.. thankx

knowledgespring wrote:
table has millions of records... 130 millions..
select count(*) from BIG_SIZE_TABLE; --- executed in sql plus command prompt.
Rows     Execution Plan
0 SELECT STATEMENT   MODE: ALL_ROWS
0   SORT (AGGREGATE)
0    TABLE ACCESS   MODE: ANALYZED (FULL) OF 'BIG_SIZE_TABLE' (TABLE)
Elapsed times include waiting on following events:
Event waited on                             Times   Max. Wait Total Waited
----------------------------------------   Waited ---------- ------------
SQL*Net message to client                       1        0.00          0.00
enq: KO - fast object checkpoint                1        0.01          0.01
Disk file operations I/O                       18        0.00          0.00
direct path read                            58921        0.34        418.54direct path read time waited is : 58921 total time waited: 418.54
That 418 seconds - not the hours you reported earlier. Is it possible that your connection to the database broke ?
On a typical system, by the way, you can usually turn one direct read for tablescan into 1MB, so your scan seems to have covered about 59 GB, which seems to be in the right sort of ballpark for 130M rows.
we have another query and when we test the query execution using v$sql, is_bind_sensitive =N, how to make is_bind_sensitive=Y all the time.. There is a hint /*+ bind_aware */ - I'd have to check whether or not it's documented at present. It might help.
I would be interested in hearing why you think the hint should be bind sensitive when the optimizer doesn't.
Regards
Jonathan Lewis

Select count(*) on sql statement returning zero when a matching row exists.

Our account has an ANSI C application that checks for the existence a row on an Oracle table(s) by using the following SQL:
int iCount = 0;
EXEC SQL
SELECT count(rownum) INTO :iCount
FROM sys.all_tab_columns
WHERE table_name IN
(SELECT table_name FROM
sys.all_synonyms
WHERE upper(synonym_name) = upper(:szDestTable))
AND upper(column_name) = upper(:szColumnName)
AND owner = 'DBAUSER';
The bind variables szDestTable and szColumnName are populated with values parsed from columns on another database table. This application is executed through out the day. Occasionally, the application will report a zero in the iCount when there should be a match. I have verified the szDestTable and szColumnName are populated with the correct values which would find a match and they are correct. To make matters even stranger, the application will parse the same input values and find a match (as it should). At some point during the day, and it can be at any time, the application will NOT find a match on the same file, same values. Every subsequent execution of this application will not find a match on the same values. Once the database is brought down and started up in the evening for its normal backups, the application will find a match again on the same values. This problem does not occur every day. I could be a week or a week and a half between incidents.
I printed the contents of the sqlca.sqqlerrm.sqlerrmc field to a log file. The sqlca.sqlerrm.sqlerrmc field reported an ORA-1405 bind variable was null when the iCount was reporting a zero. When I compiled this application, I set the Proc*C flag to UNSAFE_NULLS=yes since there are other bind variable in the application that can be NULL and that is ok.
The above SQL is compiled into the C application using the Proc*C compiler. It is compiled using the Oracle 11.2.0.2 libraries. The application is executed against an Oracle 11.2.0.2 database. The database and application are executed on an HP/Unix 11.31 platform.
This problem did not start occurring until our account went from Oracle 10.2 to Oracle 11.2. Recently, I have changed the SQL to perform a “SELECT COUNT(rownum)” instead of the “SELECT COUNT(*)”. I compiled the application and executed the new application with the SELECT COUNT(rownum) against the same database where the same application with the SELECT COUNT(*) was failing to find a row that actually existed. The application with the SELECT COUNT(rownum) found the matching row as it should have. The new application has been executing in production for about 10 days now without any problems against ten various Oracle 11.2 databases.
Why would SELECT COUNT(*) and SELECT COUNT(rownum) be any different?

This forum is about C programming in general, and about using Studio C in particular.
Your question is about Oracle database programming. You are more likely to find a helpful answer in a forum about database programming. Start here:
https://forums.oracle.com/forums/category.jspa?categoryID=18

Count(*) does not take into consideration the where condition

the following sql function is supposed to calculate the number of rows that respect the conditions in the where clause. After I call this function I get the the total number of rows in the table. Any help would be much appreciated.
FUNCTION NBRTRIMCOMPLETES (ETU_NO NUMBER, PRG_NO NUMBER) RETURN VARCHAR2 IS
nbrTrimCompletes number(6);
begin
     SELECT COUNT(*) into nbrTrimCompletes from (select * from INSCRIPTION I WHERE I.ETU_NO = ETU_NO AND I.PRG_NO = PRG_NO);
     RETURN nbrTrimCompletes;
end NBRTRIMCOMPLETES;

The compile can't distinguish between the column name ETU_NO an the variable ETU_NO, that's why your where condition always evaluates to true.
Change the name of the parameters.
FUNCTION NBRTRIMCOMPLETES (P_ETU_NO INSCRIPTION.ETU_NO%TYPE,
                           P_PRG_NO INSCRIPTION.PRG_NO%TYPE
                          ) RETURN NUMBER
IS
nbrTrimCompletes PLS_INTEGER;
begin
SELECT COUNT(*)
into   nbrTrimCompletes
FROM   INSCRIPTION I
WHERE I.ETU_NO = P_ETU_NO
AND    I.PRG_NO = P_PRG_NO;
RETURN nbrTrimCompletes;
end NBRTRIMCOMPLETES;

Some advise on 'WHERE EXISTS' in OWB.

Hello :-) me again.
I have read a number of topics regarding 'WHERE EXISTS' statment and OWB... got confused so i thought i ask your option.
OWB 10 r2.0.1.31 paris/windows
I need to tune one of my maps - nothing fancy, just a simple delete:
I need to delete values from tables A if they are equeals to the values in table B.
Both of my tables have index on matching fields, however it is non unique index and there is no primary or unique key on the tables that i could use. Both tables are not that big either - table A just under 1mil records and table B about 300K records/
So i created simple map -
Mapped table B to table A, set loading type for table A to delete, set 'Match column when deleting' on the matching column .The map was validated without any problems, the tests on a small data set were fine and bob's my uncle!
However, when i submitted the map with 'real' tables - it took a very long time to run (over an hour). I have other more complex maps (and loads of them) and everything is ticking nicely! so what is the problem with this map?
The code generated by OWB for the delete statment as follows:
DELETE FROM TABLE_A
WHERE TABLE_A.MATCH_COL = TABLE_B_MATCH_COL
and it does take ages to run from TOAD.
So i tried running
DELETE FROM TABLE_A a
WHERE exists (select n.MATCH_COL from TABLE_B n
where a.MATCH_COL = n.MATCH_COL)
it took no time at all.
Now how do i have to construct my map in order for OWB to generate WHERE EXISTS code? Is it better to use lookup or set operator? outer join? which one of them would be WHERE EXISTS?
I have another map set up identically but with larger numbers (over 70 mil records in table A). it has been running sinse yesterday lunch time! So any help/thoughts/suggestions on this matter would be greatly appreciated.
Thank you very much
Kind Regards
Vix
null

Hi Ed.
Thank you very much for your reply - greately appreciated!
I did look into TOAD when trying to 'tune' it before askling for help. The query does gazzilion tables scans (from Long Ops in TOAD), the main reason why it takes forever is 'db file scattered read' (from waits in TOAD). I checked this job first thing this morning and it's still running!
I have been using OWB Paris for the past year or so and i don't know everything there is not know about it. So i don't know how to make it to produce the code with 'WHERE EXISTS' instead of
DELETE FROM "TABLE A"
WHERE ( "TABLE A.MATCH_COL IS NULL AND :b1 IS NULL
OR "TABLE_A.MATCH_COL" = :b2)
I tried putting non-unique index on MATCH_COL of table A, and put unique index on matching column in table B (ensured that there are no duplicates in table B etc) but oracle pessimiser decided not to use it and i don't really know if the hints are working in OWB and how to make them work with OWB code (i pressume just putting them in the code as normal or is there a 'special' way of doing it?!).
I think i am just going to wrap up my statment in a procedure and just execute the procedure within OWB. I wanted to avoid doing so as i like the auditing granularity with OWB (and if it is a procedure it just calls it, executes it and that's it).
Once again thank you very much for all your help with this issue. May be someone else would have some other ideas how to make it work in OWB? If i manage to improve the performance with this particular map - i will post it here, just in case someone else is having the same problem.
Kind Regards
Vix
Message was edited by:
Vix
- I am takling about my second job that is still running - table A has over 70 mil records and table B has over 600K records.

Elgato Game Capture HD Footage takes Hours to Import

First off, I use a late 2009 21.5 inch iMac with an Intel 3.06 Ghz core 2 Duo, 12 Gigs Ram, and an Nvidia Geforce, 9400 265 mb with Mountain Lion . And all drivers up to date.
I've been having this issue for a while and with the increased work load I have been taking on, its now becoming unbearable. Every time I import Elgato Hd Footage (720p 60 FPS at high to even lowest quality) The import can take hours, Almost as long and often longer than the video even is. I thought it was a codec problem, but I have a friends footage I'm also editing that I have to convert and it gets converted to mp4 with a h264 codec as well. (Image being added below) And his imports just in a few short minutes. I have tried converting my footage as well but this does not seem to help and the only other file format I can do is MKV which adobe doesn't seem to take.
My friend footage is in the properties to the left, and Mine are on thee right. I understand there is a difference in file size and duration, but it doesn't seem like it should be making this big of a difference, where his imports in about 2 minutes when mine is in hours. This even happens if I open up the project after saving, it takes hours for it to simply load up the video. I have also tried this with videos in smaller sizes and the results seem scaled almost the same.
Please i could really use some advice on what to try out here, cause I'm out of ideas.

You're going to notice a huge difference going from what you have to 12gb, I don't know how big a difference you'll notice from 12 to 16. The reason Apple doesn't "support" 32gb in iMacs is that they write the specs at the time the machine is released taking into account the available technology at that time. Apple doesn't update it's specs for existing machines when new tecnology becomes available. If Crucial and OWC say 32 will work you'll have no trouble by using it. You won't go wrong with Crucial, I've used them foerver. Enjoy your new ram whatever you end up getting. Run Activity Monitor now and check out the page outs you're getting and run it again after you upgrade and you'll really get a visual of the difference.

Sub-Select Count query breaking TOAD

Oracle 10.2.0.4.0
Running TOAD 9.1
I am running some SQL on our eBusiness Suite:
SELECT pha.segment1
     , pha.type_lookup_code
     , (SELECT COUNT(DISTINCT pha2.po_header_id)
          FROM po.po_headers_all pha2
             , po.po_lines_all pla
         WHERE pha2.po_header_id = pla.po_header_id
           AND pla.contract_id = pha.po_header_id) po_count
     , (SELECT MAX(pha2.creation_date)
          FROM po.po_headers_all pha2
             , po.po_lines_all pla
         WHERE pha2.po_header_id = pla.po_header_id
           AND pla.contract_id = pha.po_header_id) latest_cpa_po
FROM po.po_headers_all pha
     , po.po_vendors pv
     , po.po_vendor_sites_all pvsa
WHERE pha.vendor_id = pv.vendor_id
   AND pha.vendor_site_id = pvsa.vendor_site_id
--   AND pv.VENDOR_NAME LIKE 'H%'
   AND pha.vendor_id = 98
   AND pha.type_lookup_code = 'CONTRACT'
   AND pha.org_id IN(7041, 7042);The above query runs quicky (approx. 1 second). If I take out the AND pha.vendor_id = 98 then the query takes a few minutes to run.
When I try to export it, or scroll down to view > 500 rows, TOAD crashes.
I know this isn't a TOAD forum, but I think that this is probably an issue with my no doubt rubbish SQL.
If I take out this sub-select, then the problem doesn't happen:
     , (SELECT COUNT(DISTINCT pha2.po_header_id)
          FROM po.po_headers_all pha2
             , po.po_lines_all pla
         WHERE pha2.po_header_id = pla.po_header_id
           AND pla.contract_id = pha.po_header_id) po_countHowever, I can't work out a better way of getting the data I need.
The sub-select counts POs which have been raised where the contractID on the PO line is the same as the PO Header ID from the main query.
Any advice please, on what I could do to sort this out would be much appreciated.
Thanks!

Hi,
It looks like you can replace both scalar sub-queries with a join, like this:
WITH     header_lines_summary     AS
     SELECT    pla.contract_id
          ,       COUNT (DISTINCT pha2.po_header_id)     AS po_count
          ,       MAX (pha2.creation_date)          AS latest_cpa_po
          FROM        po.po_headers_all pha2
             ,        po.po_lines_all   pla
         WHERE        pha2.po_header_id = pla.po_header_id
      GROUP BY       pla.contract_id
)                                        -- Everything up to this line is new
SELECT pha.segment1
     , pha.type_lookup_code
     , hls.po_count                              -- Changed
     , hls.latest_cpa_po                         -- Changed
FROM po.po_headers_all     pha
     , po.po_vendors           pv
     , po.po_vendor_sites_all      pvsa
     , header_lines_summary     hls                    -- New
WHERE pha.vendor_id          = pv.vendor_id
   AND pha.vendor_site_id     = pvsa.vendor_site_id
   AND pha.po_header_id          = hls.contract_id (+)          -- New
--   AND pv.VENDOR_NAME      LIKE 'H%'
   AND pha.vendor_id           = 98
   AND pha.type_lookup_code      = 'CONTRACT'
   AND pha.org_id           IN (7041, 7042);Aside from the sub-query (which is entirely new), the query above is just what you posted, with 2 lines changed and 2 lines added, as marked.
This should be more efficient, but I don't know for certain that it will solve the Toad problem.
I hope this answers your question.
If not, post a little sample data (CREATE TABLE and INSERT statements, relevant columns only) for all tables, and also post the results you want from that data.
It never hurts to say what version of Oracle you're using.

Avoid repeating same logic in 'select' and 'where' clauses?

I'll preface by saying I'm self-taught and have only been fiddling with SQL for a couple of months, so forgive me if this is a dumb question. I have a query written to pull out customers who are configured to have their products stored at the wrong warehouse, according to the first 3 digits of the zip code. Here is an extremely simplified version of a query I'm trying to run:
select custno, custbuy_zip_cd, custbuy_prim_ship_loc_cd as Warehouse,
case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
end as StdWhse
from customers
where case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
end <> custbuy_prim_ship_loc_cd
or (case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
end is not null and custbuy_prim_ship_loc_cd is null)
Now, the query works, but it seems overly convoluted and feels like there must be a way to make it simpler and faster. I'm using the same 'case when' 3 times. Originally, I had thought I could use the aliases from the 'select' clause in the 'where' clause, which would simplify things:
select custno, custbuy_prim_ship_loc_cd as Warehouse,
case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
end as StdWhse
from customers
where StdWhse <> custbuy_prim_ship_loc_cd
or (StdWhse is not null and custbuy_prim_ship_loc_cd is null)
I then found out that that caused 'invalid identifier' errors. My first attempt at a solution was to use a subquery in the 'from' clause, but that ran the 'case when' on every single customer instead of the small subset, so it wound up taking much longer even though it looked neater. Any tips on how to clean up that first query to make it run faster?
this is Oracle 11i, I believe. As a side note, I don't have write access to the database.

Thanks for all the tips so far - still going through them. You all respond fast! Sorry about using double angle brackets for != and not using code tags, I'll make sure to format my posts properly going forward. I think the double angle brackets messed up the appearance of my original queries a little. Here's how I probably should have pasted my first query in my first post:
select custno, custbuy_zip_cd, custbuy_prim_ship_loc_cd as Warehouse,
    case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
        when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
    end as StdWhse
from customers
where case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
        when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
    end != custbuy_prim_ship_loc_cd
    or (case when substr(custbuy_zip_cd,1,3) in ('839','848') then '20'
        when substr(custbuy_zip_cd,1,3) in ('590','591') then '33'
    end is not null and custbuy_prim_ship_loc_cd is null)The almost unanimous opinion seems to be that I should use a subquery in one way or another, but the problem remains that the only significant logic to narrow down the results is the logic that matches the 'case when' results (which are what the warehouse number should be, based on the zip code) to the current warehouse number. Therefore, it seems like any subquery is going to return my entire list of 600k customers, and take a much longer time than my original (messy) query. At least it has in the test runs I created based on
Satyaki's and Peter's examples. The query based on my original example takes about 2.5 minutes, and the subquery examples take about 5+ even though they look cleaner.
to clarify what the query is trying to accomplish, I want it to pull any records where the warehouse number does not equal the correct warehouse number based on zip code (or if the warehouse number is null when it shouldn't be).
I'll try to create some sample data and sample results. Customers table:
custno   custbuy_zip_cd custbuy_prim_ship_loc_cd
1        59024           20
2        59024           33desired results:
custno   custbuy_zip_cd   warehouse   StdWhse
1        59024            20          33If I could create a table to hold the standard warehouses to join on, the whole thing would be much easier. The full version of the query really has hundreds of zip code prefixes and 5 different warehouses and each account has 4 alternate warehouses as well. However, I'm stuck with read only access so everything has to go right in the query. It wouldn't be the end of the world to just stick with my original query since it's not like it takes hours, and I'll only be running it weekly. I just wanted to make sure there wasn't some other solution that wasn't just cleaner but was also faster.

Function results in SELECT and WHERE

Is there a way to avoid calling a function twice in order to have its results appear in the SELECT clause and used in the WHERE clause?
This SELECT statement will not work (as promised by documents):
select name, getLocation(name) as location
where location is not null
The only way to avoid it that I can see is to wrap this into another select with the additional criteria:
select * from ( select name, getLocation(name) as location )
where location is not null
Any alternatives?
Thanks.
--Peter

The nested select statements are much quicker than even doing a count(*) in the select and using the function call in the where clause. It appears that a function call in the where clause is expensive. The function involves selects from the same table and others, and joins but no updates, deletes, etc.
select count(*) from ... where location(...) is null
takes > 6 min. !
select count(*) from (select location(...) ) where location is null takes < 10 sec. !

Select with Where not in (subSelect)

Hi,
Is there any special about that 'not' in this ORacle sql (10g) that it doesn't return me any rows, unlike SQLServer with same code:
select count(*) from t1_ups where id_ups not in (select b.id_ups from t1_dups b)
COUNT(*)
0
vs.
select count(*) from t1_ups where id_ups not in ('abc', 'def','xyz') --select b.id_ups from t1_dups b)
COUNT(*)
16
DDL:
    "id_ups"            VARCHAR2(20 BYTE),   is the same for both tables.I see that if remove <not> it works fine. is there any OR --> AND converssion happaned with negation?
My global task is to make UNION "with replacement" when if any records exist in id_dups they will replace ones from id_ups, so I try to do it in separate parts.
Thanks
Tr
Edited by: trento on Nov 16, 2010 3:00 PM

trento wrote:
Hi,
Is there any special about that 'not' in Yes, you need to be aware how the syntax reacts to NULL values.
create table in_or_not
col1 number
TUBBY_TUBBZ?insert into in_or_not values (2);
1 row created.
Elapsed: 00:00:00.05
TUBBY_TUBBZ?select 1 from dual where 1 not in (select col1 from in_or_not);
                 1
                 1
1 row selected.
Elapsed: 00:00:00.00
TUBBY_TUBBZ?insert into in_or_not values (NULL);
1 row created.
Elapsed: 00:00:00.00
TUBBY_TUBBZ?select 1 from dual where 1 not in (select col1 from in_or_not);
no rows selected
Elapsed: 00:00:00.01http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:442029737684

Count vs exists

I have a requirement as below :
Would relate to the emp and the dept table of the SCOTT schema.
Suppose there is a third table dept_lookup .
create table dept_lookup
deptno number,-----deptno references dept's deptno
emp_count
insert into dept_lookup values (10,100);
insert into dept_lookup values (20,200);
*note : not all department's count exist in this table
in the join of emp and the dept table , i have to derive an extra column in which i have to check whether a particular deptno exists in the dept_lookup table .If exists populate the emp_count is there then populate the count else populate a default count of 0 .I may not have pictured my actual scenario clearly but this is the extract . I want to use a scalar subquery for this .
something like
case when (select count(1) from dept_lookup dl where d.depno=dl.deptno)>=1
      then dl.emp_count else 0 end
or
case when (select 1 from dept_lookup dl where exists (select 1 from dept d where d.deptno=dl.deptno)=1
then dl.emp_count else 0 end
This has to be implemented in very large tables , considering the fact that indexes are built on the deptno on each of these tables . What would be the performance impact if i compare exists with count(1). Which one is better ?
Thanks
Rahul

Hi,
You shoold not make a table with calculated data in it. This should be a view. See below example:
select
from
emp
EMPNO ENAME      JOB        MGR HIREDATE      SAL    COMM DEPTNO
7369 SMITH      CLERK     7902 17-DEC-80     800             20
7499 ALLEN      SALESMAN 7698 20-FEB-81    1600     300     30
7521 WARD       SALESMAN 7698 22-FEB-81    1250     500     30
7566 JONES      MANAGER   7839 02-APR-81    2975             20
7654 MARTIN     SALESMAN 7698 28-SEP-81    1250    1400     30
7698 BLAKE      MANAGER   7839 01-MAY-81    2850             30
7782 CLARK      MANAGER   7839 09-JUN-81    2450             10
7788 SCOTT      ANALYST   7566 19-APR-87    3000             20
7839 KING       PRESIDENT      17-NOV-81    5000             10
7844 TURNER     SALESMAN 7698 08-SEP-81    1500       0     30
7876 ADAMS      CLERK     7788 23-MAY-87    1100             20
7900 JAMES      CLERK     7698 03-DEC-81     950             30
7902 FORD       ANALYST   7566 03-DEC-81    3000             20
7934 MILLER     CLERK     7782 23-JAN-82    1300             10
select
from
dept
DEPTNO DNAME          LOC
    10 ACCOUNTING     NEW YORK
    20 RESEARCH       DALLAS
    30 SALES          CHICAGO
    40 OPERATIONS     BOSTON
create view dept_lookup as
select
d.deptno
,count(e.empno) emp_count
from
dept d
left join emp e on d.deptno = e.deptno
group by
d.deptno
select
from
dept_lookup
DEPTNO EMP_COUNT
    10         3
    20         5
    30         6
    40         0
If you are concerned bout preformance, look into Materialized views.
Regards,
Peter

Getting Long time to execute select count(*) statement.

Hi all,
My table have 40 columns and it doesn't have the primary key column. it contain more than 5M records. it's taking long time to execute simple sql statement.
Such as select (*) take 1min and 30 sec. If i use select count(index_colunm) then it finished with in 3s. i did the following workarounds.
Analyzed the table.
created required indexes.
yet getting the same performance issues. please help me to solve this issue
Thanks

BlueDiamond wrote:
COUNT(*) counts the number of rows produced by the query, whereas COUNT(1) counts the number of 1 values.Would you care to show details that prove that?
In fact, if you use count(1) then the optimizer actually re-writes that internally as count(*).
Count(*) and Count(1) are have identical executions.
Re: Count(*)/Count(1)
http://asktom.oracle.com/pls/asktom/f?p=100:11:6346014113972638::::P11_QUESTION_ID:1156159920245

Select count(*) cause time out error

I invoke the following statement:
select count(*) as total from table1
where table1 is large(30000 rows), total size of the database is more than 20GB.
The above statement will cause time out error. How to solve the problem?

Hallo chcw,
a timeout in a table with 30.000 records is very unusual (for a simple SELECT COUNT). From my point of view Dan is on the right way and you will have a typical blocking scenario.
1. if you have LOBs they won't be part of a SELECT COUNT (either its a heap of a clustered index)
2. Microosft SQL Server will ALWAYS use the smallest index for such a simple SELECT
3. if it is a heap you will run into a blocking scenario if someone is updating records and transaction has not committed.
4. Check the isolation level of the transactions - but I expect the standard which is "read committed".
Just a demonstration of "SELECT COUNT" will work whether it is a clustered index or a heap. The next script will create a heap with a LOB and additional attributes:
CREATE TABLE dbo.foo
Id INT NOT NULL IDENTITY(1,1),
n1 INT NOT NULL,
n2 SMALLINT NULL,
c1 CHAR(250) NOT NULL,
c2 CHAR(250) NULL,
c3 VARCHAR(MAX) NOT NULL DEFAULT (REPLICATE('A', 15000))
GO
Now we enter 30,000 records into the table
SET NOCOUNT ON;
DECLARE @i INT = 1;
WHILE @i <= 30000
BEGIN
INSERT INTO dbo.foo (n1, c1) VALUES (@i, 'Just a filler');
SET @i += 1;
END
GO
Keep in mind that we have a HEAP and NO indexes! To check the IO i use the following command befor any of the following examples:
SET STATISTICS IO ON;
GO
Let's start with a first try and you will check as a result the produced IO depending on the affected table partitions:
-- USE SELECT in a HEAP
SELECT COUNT(*) FROM dbo.foo;
GO
Output:
Table 'foo'. Scan count 1, logical reads 2508, ..., lob logical reads 0,
As you can see from the result the LOB-data won't be attached only the "in-row-data" are affected (which are stored in 2508 pages. The reason is a quite simple one: Microsoft SQL Server uses different allocation units for in-row-data and LOB
SELECT p.object_id, p.index_id, p.rows, au.total_pages
FROM sys.partitions AS P INNER JOIN sys.allocation_units AS AU
ON(p.partition_id = au.container_id)
WHERE object_id = OBJECT_ID('dbo.foo');
Now I create a simple index on the column n1 which is an INT-datatype
SELECT p.object_id, p.index_id, p.rows, au.total_pages
FROM sys.partitions AS P INNER JOIN sys.allocation_units AS AU
ON(p.partition_id = au.container_id)
WHERE object_id = OBJECT_ID('dbo.foo');
and run the SELECT COUNT again with the following IO-output:
Table 'foo'. Scan count 1, logical reads 69, ...
WOW - only 69 pages have to be read. The explanation is a quite simple one - now we have an index which is quite small. Microsoft SQL Server WILL use the smallest index of a table to scan the number of records. Let's try it again with n2 which is a smallint
(2 bytes)
CREATE NONCLUSTERED INDEX ix_foo_n2 ON dbo.foo (n2);
GO
The IO will be 62 (or less) because MORE index records a fitting to one data page!
Conclusion is that either you have small record size or long record size - The query optimizer will always use the smallest index for the execution (that's maybe why Visahk has asked for the execution plan).
I don't believe it is because of the "huge amount of data" but - as Dan has pointed out - it HAS TO BE a blocking scenario which can have multiple reasons. For demonstration of a blocking scenario open SSMS and start in the query windows with the following
command(s):
BEGIN TRANSACTION
UPDATE dbo.foo
SET c1 = 'This is my name'
WHERE Id = 10000;
-- What locks do we have now in the database
SELECT resource_description,
resource_associated_entity_id,
request_mode,
request_type
FROM sys.dm_tran_locks AS DTL
WHERE resource_database_id = DB_ID() AND
resource_type != 'DATABASE';
Your result should look like that (with differences in the entity_id and resource descriptions :)
The above pic demonstrates the "locking chain". As you can see the slot 7 on PAGE 82400 has an exclusive lock. This means that NO OTHER request can currently obtain another lock to it!
Now open a second query window and run a SELECT COUNT... - it will not finish!
The reason is that the second requests has to wait for the release of the locked resources by the first transaction! Leave the statement and change back to the first transaction and finish the transaction by cancellation of the query. In the moment the locks
have been released the second query will finish immediate.
So - from my point of view - Dan has given the correct answer. Against wide spreaded statements a heap won't block the whole table exclusivly but - as in a cluster, too - only the row itself (see the 7 which is the slot where the record exists).
MCM - SQL Server 2008
MCSE - SQL Server 2012
db Berater GmbH
SQL Server Blog (german only)

Select count(*) where exists (takes 5 hours).

Similar Messages

Maybe you are looking for