Oracle 11 Pivot Query
Hi,
I have 2 questions on the pivot query below:
1 - I want to have total by Country and Year
2 - I want to allow the use a dynamic year range (like year between 1990 and 2000)
Thanks,
Ribhi
select * from (
select CNTRY_NAME, MYYEAR,AM_AMOUNT
from yearly_lending t
pivot
sum(AM_AMOUNT)
for MYYEAR in (1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008,2009)
order by CNTRY_NAME
Hi all,
Thank you for your help.
The qury working as expected, but i wont to add total by country and year
Country 1974 1975 1976 1977 1977 1978...2009 Total
Jordan 10 5 0 3 5 1 24
Egypt 5 0 0 0 10 0 15
Syria 8 2 10 20
Total 23 7 0 3 15 11 59
also would like to select year range rather than entering year by year example year from 2000 to year 2009
I created a view from the qury below and then my pivot query
select
rtrim(cntry.short_name_e) cntry_name,
to_char(e.date0,'yyyy') myYear,
am.value / 1000000 am_amount,
a.fk_countrycode a_cntry
from event e,
agreement_amount am,
agreement a, in_country cntry
where
a.number0 = e.fk_agreementnumber
AND A.SUB_NUMBER <>'P'
AND a.fk_countrycode = cntry.intl_code
and a.sub_number = e.fk_agreementsub_nu
and a.type = e.fk_agreementtype
/*and rtrim(a.status) <> 'CANCELLED' */
and rtrim(e.type) = 'SIGNING'
and rtrim(e.fk_agreementtype) <> 'GRANT'
and e.fk_agreementnumber = am.fk_agreementnumber
and e.fk_agreementsub_nu = am.fk_agreementsub_nu
and e.fk_agreementtype = am.fk_agreementtype
and am.serial_number = 1
order by rtrim(cntry.short_name_e) ,to_char(e.date0,'yyyy')
Best regards,
Ribhi
Edited by: Ribhi on Apr 29, 2009 7:20 PM
Similar Messages
-
APEX 3.2 -ORACLE 10G - PIVOT QUERY
Hello, i searched around the forum and i cound't find an answer to this specific matter, although i saw some replies that were close...
i need to creat a form based on a pivot query. but oracle 10g doesn't support that feature so i hope someone can help me.
my problem is that the number of columns will be variable. here's an example:
ORIGINAL TABLE
NAME KMS VEHICLE
Joe 100 AUDI
Tom 300 VW
Mark 150 FORD
Ann 250 FORD
Joe 200 VW
Tom 123 AUDI
Mark 345 AUDI
Ann 45 VW
Joe 6 FORD
Tom 67 FORD
Mark 46 VW
Ann 99 AUDI
DESIRED RESULT
Joe Tom Mark Ann Vehicle
100 123 345 99 AUDI
6 67 150 250 FORD
200 300 46 45 VW
the new columns will be the values in the old NAME column. BUT these values are variable. today its joe,tom,mark and ann tomorrow it could be silvia, tony,richard,harry , william and jane. this means the usuall replies i saw, using MAX and DECODE will not apply because i never know what values or how many values are in this column. with pivot i can get this done.... how can i do this in oracle 10g? is there a way to creat a ser function Pivot somehow? ideas?
thanks!
Mark Pereira
Edited by: 899716 on Jul 18, 2012 12:02 PMThis is the Oracle Forms forum. Post your question in the SQL forum.
Tip: check the latest Oracle Magazine (July/August 2012). There is an article by Tom Kyte about the same question.
http://www.oracle.com/technetwork/oramag/magazine/home/index.html -
Hi,
I want to create a view. Sales amounts, countries and products will be showed in this view.
Report result will be like below:
Products
Countries Sales_Amount
TV Ipad Iphone Netbook Notebook EbookReader
England 1000 1200 1400 3000 5000 200
Germany 800 1000 1300 2800 6000 400
France 1100 1100 1500 2400 3000 500
Number of products are not limited. New prodcuts can be added by the time.
Customer is using Oracle 10g. So I can not use pivot function which cames with Oracle 11g.
If the number of products is limited, I can write sql by decode operators.
But I can not find how to implement pivot function in Oracle 10g with unlimited column values.
Do you have any comment?
Thanks,RegardsFrom the SQL and PL/SQL FAQ:
"This is not easily possible as the number of columns returned by an SQL must be known before any data is fetched, it would have to be done dynamically.
See these threads:
Franks pivoting, static and dynamic
Dynamic Columns Pipelined"
SQL and PL/SQL FAQ -
11G Pivot Query with Oracle EBS
Hello all,
We are trying to use the 11G pivot query function with data from Oracle E-Business Suite. We have an 11G database installed with our Oracle APEX. We cannot seem to get the pivot function to work. At a glance, would anyone be able to see any glaring errors in our syntax. I am not certain it is possible to provide test data so...
We are trying to have column headings with the Period Names SEP-08 OCT-08 NOV-08, with rows of segment2 007751 and accounted_dr as the dataset.
When we run the sql we get an error ORA-00904: "PERIOD_NAME": invalid identifier.
Any help or insight would be greatly appreciated.
select * from (
select segment2, accounted_dr, period_name
from gl_je_lines a, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 = '007751')
pivot
sum(accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08')
group by segment2, period_namelilhelp wrote:
Hello all,
We are trying to use the 11G pivot query function with data from Oracle E-Business Suite. We have an 11G database installed with our Oracle APEX. We cannot seem to get the pivot function to work. At a glance, would anyone be able to see any glaring errors in our syntax. I am not certain it is possible to provide test data Why not?
>
We are trying to have column headings with the Period Names SEP-08 OCT-08 NOV-08, with rows of segment2 007751 and accounted_dr as the dataset.
When we run the sql we get an error ORA-00904: "PERIOD_NAME": invalid identifier.
Any help or insight would be greatly appreciated.
select * from (
select segment2, accounted_dr, period_name
from gl_je_lines a, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 = '007751')
pivot
sum(accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08')
group by segment2, period_nameDon't use GROUP BY. When you use PIVOT, the grouping is implied by what is in the PIVOT clause and what is not.
Try this:
select *
from (
select segment2
, accounted_dr
, period_name
from gl_je_lines a
, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 = '007751'
pivot (
sum (accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08')
;which is just your posted query without the GROUP BY clause. -
Setting Column Names in Dynamic Pivot Query
Hi all,
I'm having trouble setting column names in a dynamic pivot query and was wondering if someone could please help me figure out what I need to do.
To help you help me, I've setup an example scenario in my hosted account. Here's the login info for my hosted site at [http://apex.oracle.com]
Workspace: MYHOSTACCT
Username : DEVUSER1
Password : MYDEVACCTAnd, here is my test application info:
ID : 42804
Name : dynamic query test
Page : 1
Table 1: PROJECT_LIST (Alias = PL... Listing of Projects)
Table 2: FISCAL_YEAR (Alias = FY... Lookup table for Fiscal Years)
Table 3: PROJECT_FY (Alias = PF... Intersection table containing project fiscal years)
Table 4: PROJECT_FY_HEADCOUNT (Alias = PFH... Intersection table containing headcount per project and fiscal year)Please forgive the excessive normalization for this example, as I wanted to keep the table structure similar to my real application, which has much more going on.
In my sample, I have the "Select Criteria" region, where the user specifies the project and fiscal year range that he or she would like to report. Click the Search button, and the report returns the project headcount in a pivoted fashion for the fiscal year range specified.
I've got it working using a hard-coded query, which is displayed in the "Hardcoded Query" region. In this query, I basically return all years, and set conditions on each column which determines whether that column should be displayed or not based on the range selected by the user. While this works, it is not ideal, as there could be many more fiscal years to account for, and this is not very dynamic at all. Anytime a fiscal year is added to the FISCAL_YEAR table, I'd have to update this page.
So, after reading all of the OTN SQL pivot forums and "Ask Tom" pivot thread, I've been able to create a second region labeled "Dynamic Query" in which I've created a dynamic query to return the same results. This is a much more savvy solution and works great; however, the column names are generic in the report.
I had to set the query to parse at runtime since the column selection list is dynamic, which violates SQL rules. Can anyone please help me figure out how I can specify my column names in the dynamic query region to get the same column values I'm getting in the hardcoded region?
Please let me know if you need anymore information, and many thanks in advance!
MarkHi Tony,
Thanks so much for your response. I've had to study up on the dbms_sql package to understand your function... first time I've used it. I've fed my dynamic query to your function and see that it returns a colon delimited list of the column names; however, I think I need a little more schooling on how and where exactly to apply the function to actually set the column names in APEX.
From my test app, here is the code for my dynamic query. I've got it in a "PL/SQL function body returning sql query" region:
DECLARE
v_query VARCHAR2(4000);
v_as VARCHAR2(4);
v_range_from NUMBER;
v_range_to NUMBER;
BEGIN
v_range_from := :P1_FY_FROM;
v_range_to := :P1_FY_TO;
v_query := 'SELECT ';
-- build the dynamic column selections by looping through the fiscal year range.
-- v_as is meant to specify the column name as (FY10, FY11, etc.), but it's not working.
FOR i IN v_range_from.. v_range_to LOOP
v_as := 'FY' || SUBSTR(i, 3, 4);
v_query := v_query || 'MAX(DECODE(FY_NB,' || i || ',PFH_HEADCOUNT,0)) '
|| v_as || ',';
END LOOP;
-- add the rest of the query to the dynamic column selection
v_query := rtrim(v_query,',') || ' FROM ('
|| 'SELECT FY_NB, PFH_HEADCOUNT FROM ('
|| 'SELECT FY_ID, FY_NB FROM FISCAL_YEAR) A '
|| 'LEFT OUTER JOIN ('
|| 'SELECT FY_ID, PFH_HEADCOUNT '
|| 'FROM PROJECT_FY_HEADCOUNT '
|| 'JOIN PROJECT_FY USING (PF_ID) '
|| 'WHERE PL_ID = ' || :P1_PROJECT || ') B '
|| 'ON A.FY_ID = B.FY_ID)';
RETURN v_query;
END;I need to invoke GET_QUERY_COLS(v_query) somewhere to get the column names, but I'm not sure where I need to call it and how to actually set the column names after getting the returned colon-delimited list.
Can you (or anyone else) please help me get a little further? Once again, feel free to login to my host account to see it first hand.
Thanks again!
Mark -
11G Pivot Query with parameters
Hello all,
I would like to find some way, any way to pass parameters to a pivot query. The following pivot query works, but I would like segment2 to be a variable as well as the period names so....
select * from
select segment2, accounted_dr, period_name
from gl_je_lines a, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 >='007611' and segment2 <='007751' AND period_name in ('SEP-08','OCT-08','NOV-08'))
pivot
sum(accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08') )
....would be something like....
select * from
select segment2, accounted_dr, period_name
from gl_je_lines a, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 >= :P4_OBJECT_FROM AND and segment2 <=:P4_OBJECT_TO AND period_name in &P4_EPSB_PERIOD_HOLD.)
pivot
sum(accounted_dr)
for period_name in (&P4_EPSB_PERIOD_HOLD.) )
It is our understanding that we have to hardcode period names and objects, but we would like to get around that. Does anyone have any ideas or tricks?
Thankslilhelp wrote:
Hello all,
We are trying to use the 11G pivot query function with data from Oracle E-Business Suite. We have an 11G database installed with our Oracle APEX. We cannot seem to get the pivot function to work. At a glance, would anyone be able to see any glaring errors in our syntax. I am not certain it is possible to provide test data Why not?
>
We are trying to have column headings with the Period Names SEP-08 OCT-08 NOV-08, with rows of segment2 007751 and accounted_dr as the dataset.
When we run the sql we get an error ORA-00904: "PERIOD_NAME": invalid identifier.
Any help or insight would be greatly appreciated.
select * from (
select segment2, accounted_dr, period_name
from gl_je_lines a, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 = '007751')
pivot
sum(accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08')
group by segment2, period_nameDon't use GROUP BY. When you use PIVOT, the grouping is implied by what is in the PIVOT clause and what is not.
Try this:
select *
from (
select segment2
, accounted_dr
, period_name
from gl_je_lines a
, gl_code_combinations b
where b.code_combination_id = a.code_combination_id
and segment2 = '007751'
pivot (
sum (accounted_dr)
for period_name in ('SEP-08','OCT-08','NOV-08')
;which is just your posted query without the GROUP BY clause. -
Hi, I am trying to do a pivot report, first I did a simple pivot hard-coding the columns that I was about to use but, it is not convenient so I was asked to do a dynamic report where the columns were selected by a sub-query so I followed the solution from this post
Pivot query using XML option in APEX , but I get this error ORA-00932: inconsistent datatypes: expected - got CLOB, now I know that by default that query is set to return CLOB, but the thing is that when I specify the return value to varchar2 my browser freezes and I cannot go any further than that I need to reboot my browser, I'm using Oracle 11g and APEX 3.2.1 here is my query
SELECT
xmlserialize(CONTENT DEPARTMENT_XML as varchar2(4000)) XML
FROM
SELECT
DEPARTMENT,
S_GROUP AS "S GROUP",
S_GROUP
FROM MYTABLE where PER = 'BNA' and department not like 'SI%'
PIVOT XML(
COUNT(S_GROUP)
FOR DEPARTMENT
IN (select distinct department from MYTABLE ) )
ORDER BY 1
Thank you.Because execuitng the query without the
ROW_NUMBER gives the results in milliseconds
but the row_number takes it 7+ min.I think you mean that executing the query without ROW_NUMBER gives the first rows in milliseconds. However, if you wait until the last row, you may find that it takes several minutes as well. If this is not the case, please post some more details.
And, as BluShadow already said, an additional ordering takes some time, so it will be a little slower, but probably not this much. If you really want to know where time is being spent, then I suggest to take a look at [url http://forums.oracle.com/forums/thread.jspa?threadID=501834&tstart=0]this thread, and start measuring.
Regards,
Rob. -
Pivot query, phantom behavior, possible bug?
Hi experts,
Oracle 11g
We are trying to construct a binary type table using a pivot query. (I know there will be no 0's, I will fill that in later.)
The query to create this table is as follows:
create table TEMP_T1 as
select * from (
select master_id, mbr_id
sub_id,
NVL(max(den_flag),0) DF,
NVL(max(num_flag),0) NF,
NVL(sum(den_flag),0) DXF,
NVL(sum(num_flag),0) NXF
from MEM_SUM
group by master_id, mbr_id,
sub_id
pivot ( max(DF) D_FLAG,
max(NF) N_FLAG,
sum(DXF) D_COUNT,
sum(NXF) N_COUNT
FOR sub_id in
( 'MEAS-1' AS "MEAS1",'MEAS-2' AS "MEAS2",'MEAS-3' AS "MEAS3",'MEAS-4' AS "MEAS4"))I am seeing unusual results when I run this query, and am unsure why. Wanted to get some thoughts.
First issue:
Although the query selects master_id and mbr_id, I only get master_id with the pivoted results (when it should be master_id, mbr_id & pivot results). Not sure why.
Second issue:
And this is where it gets even more strange, if I
1) do something to the code to make it have a syntax problem, re-run, it will fail, naturally.
then
2) fix the syntax back to the original query above, it will run and return master_id, mbr_id & the pivoted results.
Has anyone encountered such a strange issue before? Any suggestions on a solution welcome.
Edited by: chris001 on Feb 22, 2013 8:09 AMI've experienced a similar problem with my FaderPort controller but only under odd circumstances.
If I accidentally unplugged my FaderPort and then plugged it back in, the Control Surfaces preferences that I had assigned it would get deleted and the MIDI messages would come through as note data and other really weird stuff depending on what track I had selected in the arrange window...
What I noticed was that when I just went in and re-did my Control Surfaces stuff for that controller it would reset and work perfectly.
Eventually I had to contact PreSonus and let them know what was happening. Turns out it was a problem with the driver, and the driver had to be updated to accomodate 8.0.2.
I can only imagine that this is still the case for many companies (including the ones who made your volume pedal) since Logic 9 came so suddenly, without even a 8.1 ever being released....Try contacting that company and seeing if there are any known issues with the driver for the device and Logic 9... might help, can't hurt. -
Problem saving data model when using Oracle DB Pivot feature
I have problem saving a data model - SQL Query that uses the Oracle DB PIVOT feature?
When I hit save, nothing happends.... have also tried "save as" - nothing is saved
No error message is shown.
Is Oracle DB PIVOT not supported by OBIEE?
The SQL that is causing the problem is this one:
SELECT * FROM
SELECT field_name as field,
CASE
WHEN col_1 = 'A' THEN 'Name 1'
WHEN col_1 like 'B_%' THEN 'Name 2'
END
as gastype,
energy
FROM table_1
WHERE code = 'K'
AND trunc(DAYTIME,'YYYY') = trunc(to_date('2012-01-01 00:00:00','YYYY-MM-DD HH24:MI:SS'), 'YYYY')
AND (col_1 in ('A') or col_1 like 'B_%')
PIVOT (sum(energy) for (gastype) in ('Name 1' as Name1, 'Name 2' as Name2))
order by field
We are using OBIEE 11.1.1.6.7 (Win 2008R2) against Oracle DB 11.2.0.3Just a wild guess .. Have you done dbsetup after installing IAS and said you are going to said databases ?
-
Please Help me on pivot Query .
I have table Emp with eno,emp_date , salary
EMPNO EMP_DATE salary
1 2/10/2009 1000
1 2/19/2009 2000
1 2/11/2009 2500
I want to pivot the query like
Empno date1 salary1 date2 salar2 date3 salary3 ..
1 2/10/2009 1000 2/19/2009 2000 2/11/2009 2500 ..
Regards
ManiHi, Mani,
Do you want separate columns for each date?
If so, search for pivot or rows to columns and you'll get lots of examples, such as [this thread|http://forums.oracle.com/forums/thread.jspa?messageID=3227388?].
Many of the examples you'll find will involve combining several rows into one pivoted cell, using an aggregate function like AVG or COUNT.
Your case is slightly different: it looks like there's a one-to-one correspondence between your original data and the pivoted data. In that case, you'll use the aggregate function MAX (or MIN: if there's only one item, it doesn't matter which).
Is there a limit to the number of pivoted columns you want?
Will you have multiple rows in your output, one row per employee?
If so, what happens if different employees have different dates?
Is it sufficient for you to have one big column that has a list of dates and salaries for each employee?
If so, search for+string aggregation+, which is usually easier than pivoting.
Edited by: Frank Kulash on Feb 3, 2009 12:09 PM -
Getting counts and divide by sum in a single Pivot query
Hi,
I have a pivot query which gives the counts of all codes. I also have to divide the count with the total in the same query.
e.g
col1 col2 col3
error_1 05-Jun-2012 parts
error_1 05_june_2012 parts
error_1 04_june_2012 consumables
error_2 04_june_2012 consumables
error_3 03_june_2012 parts
.Now the output should have the counts/divided by sum multiplied by 100. Basically, the percentage.
output will be something like
error June-2012 May-2012 Feb-2012 .... headers
error_1 60% 0 % 0%
error_2 20% 0 0%
error_3 20% 0% 0%Any suggestions please?
Thanks,
Sunwith
t1 as
(select 'error_1' col1,trunc(sysdate) - 1 col2 from dual union all
select 'error_1',trunc(sysdate) - 1 from dual union all
select 'error_1',trunc(sysdate) + 30 from dual union all
select 'error_2',trunc(sysdate) + 31 from dual union all
select 'error_3',trunc(sysdate) - 2 from dual union all
select 'error_3',trunc(sysdate) + 30 from dual union all
select 'error_1',trunc(sysdate) - 3 from dual union all
select 'error_2',trunc(sysdate) - 2 from dual union all
select 'error_3',trunc(sysdate) + 30 from dual union all
select 'error_4',trunc(sysdate) - 6 from dual
select col1,
count(case to_char(col2,'mon') when 'jul' then 1 end) one,
100 * ratio_to_report(count(case to_char(col2,'mon') when 'jul' then 1 end)) over () x,
count(case to_char(col2,'mon') when 'jun' then 1 end) two,
100 * ratio_to_report(count(case to_char(col2,'mon') when 'jun' then 1 end)) over () y
from t1
group by col1
order by col1Regards
Etbin
Edited by: Etbin on 8.6.2012 14:15
used next month to keep english and slovenian 'mon' the same ;) -
Dynamic SQL and Pivot Query in 11G
Hello all,
I am using APEX and 11G I am trying to create a report based on the results of a pivot query. Below is the code to build the query string. The :P4_EPSB_PERIOD_HOLD holds data like (SEP-08') for example.
declare
q varchar2(4000);
begin
q:=q ||' select * FROM';
q:=q ||' ( ';
q:=q ||' select segment2, ';
q:=q ||' accounted_dr, ';
q:=q ||' period_name ';
q:=q ||' from gl_je_lines a, ';
q:=q ||' gl_code_combinations b';
q:=q ||' where b.code_combination_id = a.code_combination_id';
q:=q ||' and segment2 >= :P4_EPSB_OBJECT_FROM';
q:=q ||' and segment2 <= :P4_EPSB_OBJECT_TO';
q:=q ||' and period_name IN :P4_EPSB_PERIOD_HOLD';
q:=q ||' and segment4 >= :P4_EPSB_LOCATION_FROM';
q:=q ||' and segment4 <= :P4_EPSB_LOCATION_TO';
q:=q ||' )';
q:=q ||' PIVOT';
q:=q ||' (';
q:=q ||' sum(accounted_dr)';
q:=q ||' for period_name IN :P4_EPSB_PERIOD_HOLD';
q:=q ||' )';
return q;
end;
I get the missingfailed to parse SQL query:
ORA-00906: missing left parenthesis
If I print the sql statement that the query generates, I get the following code, which, if the varaibles are hard-coded, works fine.
select * FROM ( select segment2, accounted_dr, period_name from gl_je_lines a, gl_code_combinations b where b.code_combination_id = a.code_combination_id and segment2 >= :P4_EPSB_OBJECT_FROM and segment2 <= :P4_EPSB_OBJECT_TO and period_name IN :P4_EPSB_PERIOD_HOLD and segment4 >= :P4_EPSB_LOCATION_FROM and segment4 <= :P4_EPSB_LOCATION_TO ) PIVOT ( sum(accounted_dr) for period_name IN :P4_EPSB_PERIOD_HOLD )
Any advice as to how to tackle this would be most welecome and appreciated.
ThanksP4_EPSB_PERIOD_HOLDcome with single quotes? like 'SEP-08' or SEP-08
-
How to find bottleneck in pivot query
I have the following table and query I am running against that table.
CREATE TABLE StagingTable(
DateKey int NOT NULL,
VersionNumber smallint NOT NULL,
SetID varchar(10) NULL,
ClassID char(5) NULL,
VariableName varchar(50) NULL,
VariableDescription varchar(255) NULL,
PeriodNumber int NULL,
PeriodData decimal(18, 6) NULL,
Column1 varchar(50) NULL,
Column2 varchar(50) NULL,
Column3 varchar(50) NULL,
Column4 varchar(50) NULL,
Column5 int NULL,
Column6 varchar(25) NULL,
Column7 int NULL,
Column8 int NULL,
Column9 varchar(50) NULL,
Column10 varchar(50) NULL,
Column11 varchar(50) NULL,
Column12 int NULL,
RowNumber int NULL
GO
WITH cte as (
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName ,
PeriodNumber ,
PeriodData
from StagingTable
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName,
[0], [1], [2], [3], ,,,, [360]
from cte
pivot (SUM(PeriodData)
FOR PeriodNumber IN (
[0], [1], [2], [3], ..., [360]
) as pvt;
CREATE UNIQUE NONCLUSTERED INDEX IX_StagingTable ON dbo.StagingTable
DateKey,
VersionNumber,
SetId,
ClassID,
VariableName,
PeriodNumber
INCLUDE ([PeriodData])
I have about 5 million rows in StageTable and the pivot query returns 32,000 rows. It takes about 3 and a half minutes to run this query, whether I am outputting the results to the results window in SSMS or selecting into a new table.
I checked the execution plan for this query (outputting to results window) and the only operation that takes any time is the non-clustered index scan (84% with Stream aggregate showing 14%).
Here are the things I compared this to. I ran a select from this table using just the columns in the index. The results returned in less than one minute and as expected the most expensive operation (100%) was the non-clustered index scan.
By the way, a select into returned almost immediately.
If I take the pivot results (after a select into) and create a new table by select into, it takes almost no time for the query to finish.
I did an estimated execution plan on these three queries (where PivotTable is a dump of the pivot query into a heap):
WITH cte as (
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName ,
PeriodNumber ,
PeriodData
from StagingTable
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName,
[0], [1], [2], [3], ,,,, [360]
from cte
pivot (SUM(PeriodData)
FOR PeriodNumber IN (
[0], [1], [2], [3], ..., [360]
) as pvt;
SELECT * from PivotTable;
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName ,
PeriodNumber ,
PeriodData
from StagingTable;
The first query (pivot query) had a cost of 41%, the second 25% (select pivottable) and the third 34% (select base table). These are nowhere close to the ratios of the amount of time that it takes to run these queries (pivot query 3.5 minutes,
select PivotTable 5 seconds and select base table almost one minute).
For the following:
WITH cte as (
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName ,
PeriodNumber ,
PeriodData
from StagingTable
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName,
[0], [1], [2], [3], ,,,, [360]
into newTable1
from cte
pivot (SUM(PeriodData)
FOR PeriodNumber IN (
[0], [1], [2], [3], ..., [360]
) as pvt;
SELECT *
into newTable2
from PivotTable;
SELECT
DateKey ,
VersionNumber ,
SetID ,
ClassID ,
VariableName ,
PeriodNumber ,
PeriodData
into newTable3
from StagingTable;
The distribution of query cost is strikingly different. The pivot query has a cost of 98% and the other two queries have costs of 1%. The big difference between these queries and the versions that just output the results (in the execution plan)
is in the table insert. The pivot query shows a cost of 26,000 while the select from pivottable shows cost of 160 and select from StagingTable shows cost of 280.
What else can I look at? Also, what else can I try to optimize the pivot query.
For now, I am going to see if I get better performance letting SSRS do the pivoting.
Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.comI have finally nailed down what is causing the bottleneck.
The Compute Scalar step for some reason shows almost no cost to the execution plan. But I noticed that the actual number of rows is 5 million (the same as the scan). It appears that SQL Server expands every row to 360+ columns, all 5 million
of them.
So I did an experiment. I took the table that I was populating and did a cross join to a number table. This table has 30 K rows and 360 columns. I figured that if I select 150 rows from the number table it will generate around 5 million
rows. The results were almost identical to using the MAX(CASE WHEN ...) construct: 1 minute and some change.
I then cross joined with only 75 rows. It took 30 seconds. 37 rows took about 15 seconds.
So the Compute Scalar costs a lot more then the Execution plan analyzer says.
SSIS appears to gather one row at a time. Once it finishes one row, one group by set, it sends the row on its way. So, in my case, the 5 million rows is read in with 7 columns and 30 K rows are output with 365 columns or so.
SQL Server creates 5 million rows with 365 columns, then aggregates them down to 30 K rows.
Thank you, Tom for you discussion with me on this. I think that I can present my thoughts to my lead.
Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.com -
Can't edit and re-Save Power Pivot query with Teradata connection (using LDAP security mechanism) unless "Save my password" option was checked. Without this option I receive this error upon attempt to Save changes made to the query ...
The following exception occurred while the managed IDbConnection interface was being used: [TeraGSS Security Library] [115022] Exception occurred in TERAGSS layer. See inner exception for details.;TdgssAuthenticationTokenExchange delegate threw an exception.
See the inner exception for details.
ErrorCode: -452984642 Severity: Error Facility: DotNet
[Teradata Database] [8017] The UserId, Password or Account is invalid..
A connection could not be made to the data source with the DataSourceID of '6a06124c-496f-4887-bf39-d2582173d499', Name of 'Teradata fsltdprd'.
An error occurred while processing table 'Query'.
The current operation was cancelled because another operation in the transaction failed.Sorry you're right, the Office category isn't currently accepting bug reports in which case Olaf's suggestion to use the smiley face is the way to go. In Excel please go to File > Options > Trust Centre > 'Trust Centre Settings...' and check
that the Feeeback Tool is enabled.
If the option is greyed out like this...
... then you should be able to enable it by changing the value of the 'Enabled' registry key from a 0 to a 1 which you will find under: HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Common\Feedback. You will then need to close all Office applications
and re-launch Excel 2013.
Once the Feeback Tool has been enabled, you should see the smile face(s) in the top right hand corner of the Excel window and be able to send your feedback.
Regards,
Michael
Please remember to mark a post that answers your question as an answer...If a post doesn't answer your question but you've found it helpful, please remember to vote it as helpful :)
Website: nimblelearn.com, Blog:
nimblelearn.com/blog, Twitter:
@nimblelearn -
Help with oracle 11g pivot operator
i need some help with oracle 11g pivot operator. is it possible to use multiple columns in the FOR clause and then compare it against multiple set of values.
here is the sql to create some sample data
create table pivot_data ( country_code number , dept number, job varchar2(20), sal number );
insert into pivot_data values (1,30 , 'SALESMAN', 5000);
insert into pivot_data values (1,301, 'SALESMAN', 5500);
insert into pivot_data values (1,30 , 'MANAGER', 10000);
insert into pivot_data values (1,301, 'MANAGER', 10500);
insert into pivot_data values (1,30 , 'CLERK', 4000);
insert into pivot_data values (1,302, 'CLERK',4500);
insert into pivot_data values (2,30 , 'SALESMAN', 6000);
insert into pivot_data values (2,301, 'SALESMAN', 6500);
insert into pivot_data values (2,30 , 'MANAGER', 11000);
insert into pivot_data values (2,301, 'MANAGER', 11500);
insert into pivot_data values (2,30 , 'CLERK', 3000);
insert into pivot_data values (2,302, 'CLERK',3500);
using case when I can write something like this and get the output i want
select country_code
,avg(case when (( dept = 30 and job = 'SALESMAN' ) or ( dept = 301 and job = 'SALESMAN' ) ) then sal end ) as d30_sls
,avg(case when (( dept = 30 and job = 'MANAGER' ) or ( dept = 301 and job = 'MANAGER' ) ) then sal end ) as d30_mgr
,avg(case when (( dept = 30 and job = 'CLERK' ) or ( dept = 302 and job = 'CLERK' ) ) then sal end ) as d30_clrk
from pivot_data group by country_code;
output
country_code D30_SLS D30_MGR D30_CLRK
1 5250 10250 4250
2 6250 11250 3250
what I tried with pivot is like this I get what I want if I have only one ( dept,job) for one alias name. I want to call (30 , 'SALESMAN') or (301 , 'SALESMAN') AS d30_sls. any help how can I do this
SELECT *
FROM pivot_data
PIVOT (SUM(sal) AS sum
FOR (dept,job) IN ( (30 , 'SALESMAN') AS d30_sls,
(30 , 'MANAGER') AS d30_mgr,
(30 , 'CLERK') AS d30_clk
this is a simple example .... my real life scenario is compliated with more fields and more combinations .... So something like using substr(dept,1,2) won't work in my real case .
any suggestions get the result similar to what i get in the case when example is really appreciated.Hi,
Sorry, I don't think there's any way to get exactly what you requested. The values you give in the PIVOT ... IN clause are exact values, not alternatives.
You could do something like this to map all alternatives to a common value:
WITH got_dept_grp AS
SELECT country_code, job, sal
, CASE
WHEN job IN ('SALESMAN', 'MANAGER') AND dept = 301 THEN 30
WHEN job IN ('CLERK') AND dept = 302 THEN 30
ELSE dept
END AS dept_grp
FROM pivot_data
SELECT *
FROM got_dept_grp
PIVOT ( AVG (sal)
FOR (job, dept_grp)
IN ( ('SALESMAN', 30)
, ('MANAGER' , 30)
, ('CLERK' , 30)
;In your sample data (and perhaps in your real data), it's about as easy to explicitly define the pivoted groups individually, like this:
WITH got_pivot_key AS
SELECT country_code, sal
, CASE
WHEN job = 'SALESMAN' AND dept IN (30, 301) THEN 'd30_sls'
WHEN job = 'MANAGER' AND dept IN (30, 301) THEN 'd30_mgr'
WHEN job = 'CLERK' AND dept IN (30, 302) THEN 'd30_clrk'
END AS pivot_key
FROM pivot_data
SELECT *
FROM got_pivot_key
PIVOT ( AVG (sal)
FOR pivot_key
IN ( 'd30_sls'
, 'd30_mgr'
, 'd30_clrk'
;Thanks for posting the CREATE TABLE and INSERT statements; that really helps!
Maybe you are looking for
-
ITunes crashes my internet connection almost every time I launch it
Hello, since my last upgrading to iTunes 10.1.2, (I did not do it for a long while), I am experiencing severe disconnections from the internet. These disconnections do not usually occur at the moment of the lauching, but a little bit later, when I am
-
Camera raw not working after recent update
I updated my creative cloud apps last week. Now when I try to open a raw image, photoshop opens then crashes before camera raw can open? I have also tried to open images from bridge and it does the same thing? I have uninstalled and reinstalled bot
-
Paid subscription, but opens in trial mode (expired)
My Premiere Pro CC 2014 has finished its trial and I have already payed for the creative Cloud in order to get the Premiere Pro CC 2014 and Creative Cloud only let me download Premiere Pro CS6. Why??? Can the newest CS6 version open CC' files?? What
-
Non-admin users and Time Machine
We have a 1-1 laptop program with 7th and 8th grade students. The students are not admins of their own laptops. This year we gave each student an external hard drive and had them use Time Machine to back-up their accounts. The problem was when a stud
-
How to make Google custom search results appear on a new page?
Anybody got any idea how to make the search results appear on a separate page? I'm using the Google custom search to get generated code to place on my pages, after providing some 'guidance' information on the look and feel of it. Normally, Google wil