Join 25 M with 200 M - Query Issue

Question

I have 2 tables, CUSTOMER and CUSTOMER Transaction. CUSTOMER has 25 M records and CUSTOMER TRANSACTION has 200 M records. I need to get the count from CUSTOMER TRANSACTION divided by transaction type. below is my tables.
CUSTOMER
========
CUST_ID          LOC_ID          (OTHER COLUMNS)
12345          001
23456          002
67890          910
54298          789
16754          909CUSTOMER TRANSACTION
====================
CUST_ID          LOC_ID          TRANSACTION_TYPE     DATE          (OTHER COLUMNS)
12345          001          CREDIT               01-jan-01
12345          001          DEBIT               02-jan-02
12345          001          CHEQUE               03-jan-03
12345          001          CASH               04-jan-04
12345          001          CASH               05-jan-06
12345          001          CASH               11-feb-11
54298          789          CREDIT               01-jan-01
54298          789          CREDIT               02-jan-01I need to have below output
CUST_ID          LOC_ID          CREDIT     DEBIT     CASH     TOTAL
12345          001          1     1     3     5
23456          002          0     0     0     0
67890          910          0     0     0     0
54298          789          2     0     0     2
16754          909          0     0     0     0
54298          789          0     0     0     0
SELECT C.CUST_ID, C.LOC_ID, CT.TOTAL, CT.CREDIT, CT.DEBIT, CT.CASH
FROM
CUSTOMER C LEFT OUTER JOIN
select      CUST_ID,
     LOC_ID,
     COUNT(*) TOTAL,
     SUM(CASE WHEN TRANSACTION_TYPE = 'CREDIT',1,0) CREDIT,
     SUM(CASE WHEN TRANSACTION_TYPE = 'DEBIT',1,0 END) DEBIT,
     SUM(CASE WHEN TRANSACTION_TYPE = 'CASH',1,0 END) CASH
FROM      CUSTOMER_TRANSACTION
GROUP BY CUST_ID, LOC_ID
) CT
ON C.CUST_ID = CT.CUST_ID and C.LOC_ID = CT.LOC_IDNow my CUSTOMER table itself is joined with other 5 tables, with left outer join to get other information.I am joining the count as above. But i am facing severe performance issue, as it does group by on 200 M records in CUSTOMER_TRANSACTION and then joining this result set with CUSTIMER table using left outer join.
Can some one help me on how to write efficient query?

Answer 1

ace_friends22 wrote:
The problem with this query is, it takes lots of time to reqturn the result set.Of course. You cannot expect it to be fast - not with the amount of I/O that needs to be done.
The simple and harsh fact is that the more data there is for the SQL to crunch, the slower it will be, as I/O is the slowest and most expensive operation on a database.
I just wanted to know if there is better way of writing this qyery?There's 2 basic ways to address this. Do less I/O. Do smarter I/O.
Doing less I/O means using optimal I/O paths to get to the relevant data. Like indexes. Or partitions. Or both. And ensuring the CBO comes up with a sane execution plan and not one based on none or skewed statistics. Etc.
Doing smarter I/O means trying to eliminate some of the I/O latency using parallel processing. Oracle supports both parallel DML and DDL. And (for example), instead of having a single process crunching 20 million rows, you could use 20 parallel processing each doing around a million rows - assuming of course you have the CPU capacity and 20 I/O intensive processes will not overload the I/O subsystem.
More than that... get something like Oracle's Exadata storage cells that provide a 40Gb I/O fabric layer and a very intelligent storage server...
And keep a firm grip on the realities of computing with regards your performance expectations.