Pivot Function in Power Query

Hi all,
One thing i'm struggling with is the use of the pivot function to pivot a column within PQ.  I'm confused on why this happens and why I can't clear it up...but here goes:
I unpivot a set of columns which then gives me two columns (Attribute/Value).  I do this because I have repeated columns of data, but it should all fall under one column (think Hours, Hours1, Hours2, etc) - basically the input is coming from a form
that has repeating fields but everything is the same data type.
I make some adjustments to the Attribute columns (Replace items within the Attribute column) then Pivot the two columns again...theoretically I would be able to do this since i just unpivoted the same data, but:
I get tons of errors (usually "There were too many elements in the enumeration to complete the operation".).  I'm not entirely sure why this happens and it doesn't seem to make sense to me that I can't UnPivot and Pivot and get the same results
as I originally had.  
Any recommendations would be greatly appreciated!
EDIT: I'm choosing "don't aggregate" during the Pivot operation...I get values if I keep some sort of aggregate function, but not the right values (I get a count of the fields instead of the data within the fields)

Try this solution:
Once you are to step 3, add an index column that starts at 0 and another index column that starts at 1. 
Transform by Pivot Column using the names in Attribute to create new columns and the Values Column set to Value.  Under advanced options select Don't Aggregate. 
Fill Down the Date column
Filter rows to remove null from Work Time. 
Here's the M code from the advanced editor: 
let
    Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
    #"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
    #"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1),
    #"Pivoted Column" = Table.Pivot(#"Added Index1", List.Distinct(#"Added Index1"[Attribute]), "Attribute", "Value"),
    #"Filled Down" = Table.FillDown(#"Pivoted Column",{"Date"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Work Time] <> null)),
    #"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows",{{"Work Time", type time}})
in
    #"Changed Type"
If that doesn't work, this example might do the trick:
http://www.excelguru.ca/blog/2015/01/07/transpose-stacked-tables/

Similar Messages

  • Add actions to an existing query or function in Power Query

    Hello
    I have just started using Power Query and can see this will be an amazing tool to add to my tool set. However I am rather slow on the uptake and am having trouble with some of the basics of using Power Query
    I have successful created some queries and by following along on some articles at excelguru.ca/blog i have created a usable function or two
    The issue i have and can see this being something i will do a lot in the near future (especially while still learning), is that I need to go back to the query and function and edit it. In particular i need to either rename some columns to make them compatible
    with other data sources, or delete them.
    I can see that i can manually edit the query / function in the advanced editor, but i was wondering if it is possible to open the report up again and edit the columns in the same manner as when creating them in the first place?
    Thanks in advance for any help given.
    I am using Excel 2010, although do have access to Excel 2013 if needed, but as i am using PowerPivot in 2010 i would prefer to use PQ in 2010 also.
    Proportal

    OK - What had completely stumped me, and i had to set it aside to do other things, has just been solved!
    To explain - obviously editing Queries is simple - right click the query and edit... and then make the changes you want.
    For functions - the thing i was really struggling with is also fairly simple, but takes a few more steps.
    1 - copy the query from the advanced editor
    2 - create a new blank query from Get External Data on the Home Tab / From Other Sources / Blank Query
    3 - Past your function in and remove the function statement(s) and reinstate and file path information
    4 - Edit the columns etc as you want.
    5 - when completed paste the relevent sections back into the original function.
    Done.
    Hope this helps any others out there that were struggling with something so simple... 
    Moral of this - copy and save all your queries and functions in a text editor such as notepad++ so you can cut out a lot of the steps above...

  • Dcount function in power query

    Hi,
    I have a query generated in power query
    that gives me the following results:
    ID          VALUES             
    1              A001
    2              A002
    3              A001
    4              B001
    5              B002
    6              B001
    I need to insert a new calculated field
    in the query I of the order of
    repetition of each record. The result we
    want to achieve is the following:
    ID          VALUES       RESULT 
    1              A001             1
    2              A002             1
    3              A001             2
    4              B001             1
    5              B002             1
    6              B001             2
    Using access get it using the following
    calculated field in a query: " DCont("*";"tabla1";"Valor='" & [valor] & "'and Id_prueba<=" & [id_prueba])".
    How I can do the same with
    power query?
    Thank you very much in advance and Merry Christmas
    to all,
    José Luis

    Hi,
    You can can use Table.Group and  Table.AddIndexColumn function to solve this.
    let
        Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
        Group = Table.Group(Source, {"VALUES"}, {{"Group", each Table.AddIndexColumn( Table.Sort(_, {"ID", 0 } ) , "RESULT" ,1,1 ), type table}}),
        Final = Table.Sort( Table.Combine( Group[Group] ) , { "ID", 0 } )
    in
        Final
    Regards,
    Hi,
    First thank you very much
    for your answer. Could you please tell me
    where I have to enter the code
    you indicate in your response ?, the
    query´s code where I need to create the
    RESULT field is as follows:
    let
        Origen = Xml.Tables(Web.Contents("http://localhost:xxxx/planning_units/1001/schedule/2014-09-29.xml?end_date=2015-12-31&level=schedule&auth=xxxxxxxxxxxxxxxx=")),
        #"Tipo cambiado" = Table.TransformColumnTypes(Origen,{{"Attribute:type", type text}}),
        #"Expandir schedule" = Table.ExpandTableColumn(#"Tipo cambiado", "schedule", {"activity-id", "booking-date", "employee-id"}, {"schedule.activity-id",
    "schedule.booking-date", "schedule.employee-id"}),
        #"Expandir schedule.activity-id" = Table.ExpandTableColumn(#"Expandir schedule", "schedule.activity-id", {"Element:Text"}, {"schedule.activity-id.Element:Text"}),
        #"Expandir schedule.booking-date" = Table.ExpandTableColumn(#"Expandir schedule.activity-id", "schedule.booking-date", {"Element:Text"}, {"schedule.booking-date.Element:Text"}),
        #"Expandir schedule.employee-id" = Table.ExpandTableColumn(#"Expandir schedule.booking-date", "schedule.employee-id", {"Element:Text"}, {"schedule.employee-id.Element:Text"}),
        #"Índice agregado" = Table.AddIndexColumn(#"Expandir schedule.employee-id", "Índice", 1, 1),
        #"Columnas reordenadas" = Table.ReorderColumns(#"Índice agregado",{"Índice", "schedule.activity-id.Element:Text", "schedule.booking-date.Element:Text",
    "schedule.employee-id.Element:Text", "Attribute:type"}),
        #"Columnas con nombre cambiado" = Table.RenameColumns(#"Columnas reordenadas",{{"Índice", "ID"}}),
        #"Columnas quitadas" = Table.RemoveColumns(#"Columnas con nombre cambiado",{"Attribute:type"}),
        #"Columnas reordenadas1" = Table.ReorderColumns(#"Columnas quitadas",{"ID", "schedule.booking-date.Element:Text", "schedule.activity-id.Element:Text",
    "schedule.employee-id.Element:Text"}),
        #"Personalizada agregada" = Table.AddColumn(#"Columnas reordenadas1", "VALUE", each [#"schedule.booking-date.Element:Text"]&[#"schedule.activity-id.Element:Text"]&[#"schedule.employee-id.Element:Text"])
    in
        #"Personalizada agregada
    Thank you very much in advance,
    José Luis

  • Refresh Pivot Bug in Power Query version 2.16.3822.242

    I have an Excel file with Power Query and several Pivot Tables built from the Power Query data. The Pivots are built from the table loaded to a worksheet, and there is a timeline date slicer connected to all those various Pivot Tables.
    In build 2.16.3822.242, refreshing the Pivot Table (either by right-click/Refresh, or programmatically via macro) causes the date timeline slicer to disappear (literally be deleted from the sheet).  The Pivot Refresh succeeds fine.
    This does not occur when using Power Query Version: 2.16.3785.242. So for the time being, my only workaround is to avoid installing the newer build.
    Any ideas what's causing it?  I told my coworker (the primary user of the file) that she could still use the file and go filter every pivot's date range separately, rather than the slicer, but that is laborious and for now she is just going to remote-desktop
    into a computer that doesn't have build 3822 installed yet.
    Shawn Keene

    Hey Ed, I apologize for un-proposing your answer so quick, I was on my Windows Phone but the forum loading spinner wouldn't leave the text entry field, so I couldn't input a reply until I got home.
    I actually found a solution just today. This may not be the 'only' solution, but it worked well.  When I had previously thought I had a solution, it only 'worked' because my subsequent refresh after making the table change didn't add any new rows of
    data (all the rows it would have returned that day were already there).  Today I made the change and then refreshed, and the new rows were formatted as the Power Query intended them to be.
    The settings I used were to set the table to "do not preserve formatting" and to "overwrite existing cells for new data, clear unused cells". Now my inserted Power Query table is plain white (no banded lines or colors), but I don't care
    because it's on a hidden sheet that no one will ever see.
    Here's the table settings that seemed to work well for me.
    And to be honest (although I haven't extensively tested), I think this was only a problem if the table was created using an old version of Power Query, then refreshed with the new version.  Anyway, now I can easily refresh my Query and then the Pivot
    from it without trouble.
    And since I suck at brevity:  I also think it'd be beneficial for me to re-create these workbooks and have the Power Query only add the data to the data model, and source the Pivot directly from that, rather than indirectly source the pivot from a populated
    table. Basically save a step, and then the Pivot will refresh together with the connection, rather than having to be refreshed separately after the query is done.
    In any case, I'm loving this tool and I'm practically a Power Query evangelist to every other department, because it empowers the knowledge worker to investigate data without waiting weeks for a DBA to have time to write a query and send results.  Plus
    we power this awesome real-time performance dashboard showing live KPIs of 15 metrics, and ditched our Domo.com account, saving upwards of 100k per year, just by using Power Query. Using nothing but plain Excel and Power Query, it publishes the graphics to
    our intranet and TVs around the builds live all day. It's amazing.
    Shawn Keene

  • 'Merge' function in Power Query

    A bit of background-
    I have 3 .txt files that I would like to combine into one table in Excel using Power Query. Two of the files contain the same kind of data, and the third contains geographic data I'd like to add on. The first two files I mentioned have an 'ID number' column
    that is also contained in the third file (with geographic data). My goal is to append the first two files, and then use a merge to add geographic fields to the data, matching up based on common ID numbers.
    The first file is about 10000 rows, the second is about 6000. The files are not very big, however when I try to expand the columns in the merge, it says that Excel has loaded several Gbs from the file I'm trying to merge with, when the file is only about
    16 mb.
    Anyone have any ideas how this merge could cause all this 'data' to be loaded, never allowing it to be completed?

    Hi,
    Thanks for your feedback. One way that you could approach this scenario is by importing the three files using "From File -> From Text" in Power Query, then use "Append" to get a new query that appends the first two queries, and finally do a Merge
    operation between the result of Append and the third file.
    If you are indeed doing this and still hitting the issue, you can send us more details via "send a frown" and selecting "include formulas", which will give us more information to help you debug the issue.
    Thanks,
    M.

  • With out using pivot function need a Query

    Hi
    I am having table which has 7 columns
    data in table:
    ID,Region,area, year-month,  sales_target, actual_sales,
    1, abc,    xyz,   200907,       1000,          500
    2, abc,    pqr,   200908,       2000,         1500
    3, mnr,   xyz,   200907,       3000,          2000
    I need the data in year and  month with out using pivot funtion
    intial
    region, area,    jul,   aug, sep, oct .......jun
    abc,     xyz,    1000,0,     0,    0...         0
    actual
    region, area,    jul,   aug, sep, oct .......jun
    abc,     xyz,    500,   0,     0,    0...         0Thanks

    Here it is
    with d as ( select 1 ID, 'abc' Region, 'xyz' area, 200907 yearmonth,  1000 sales_target, 500 actual_sales from dual
    union all   select 2, 'abc',    'pqr',   200908,       2000,         1500 from dual
    union all   select 3, 'mnr',   'xyz',   200907,       3000,          2000 from dual
    select  region, area,
    max(case extract(month from to_date(yearmonth,'yyyymm')) when 7 then sales_target
    else 0 end ) TGT_JUL
    max(case extract(month from to_date(yearmonth,'yyyymm')) WHEN 8 then sales_target
    else 0 end ) TGT_AUG
    max(case extract(month from to_date(yearmonth,'yyyymm')) WHEN 9 then sales_target
    else 0 end ) TGT_SEP
    from d
    group by  region, area
    REG ARE    TGT_JUL    TGT_AUG    TGT_SEP
    abc pqr          0       2000          0
    mnr xyz       3000          0          0
    abc xyz       1000          0          0You can copy and replicate the results for another one - actual_sales.

  • Power Query filters based on UserID

    I have a sales dashboard built with Power Query, Power Pivot, Power View and Power Maps that I need to filter based on the user that opens the dashboard. Users in ABCD territory, should see sales data for ABCD sales, but should not see sales from ZYXW territory.
    We do this all the time in SSRS using User!UserID as a parameter passed into a stored procedure, but I am not familiar with what function within Power Query would pass the user ID as a parameter into the stored procedure.
    Any help would be greatly appreciated.

    I have solved this limitation by automatically producing one file per user on the dashbord. This also reduce the amount of data per user so the response is fast.
    In the database MS SQL there are a number of "temporary" tables which are sources to the Power Pivot
    database. Those tables are loaded with subsets of data per user and the database is refresh and published
    on the dashbord (Power View). The tables are loaded by stored procedures with the users ID as parameter. Programmed with VBA. No SharePoint in this case.
    Gosta Munktell

  • Data loaded to Power Pivot via Power Query is not yet supported in SSAS Tabular Cube

    Hello, I'm trying to create a SSAS Tabular cube from a data loaded to Power Pivot via Power Query (SAP BOBJ connector) but looks like is not yet supported.
    Any one tried this before? any workaround that make sense?
    The final goal is pull data from SAP BW, BO Universe (using PowerQuery) and be able to create a SSAS Tabular cube.
    Thanks in advance
    Sebastian

    Sebastian, 
    Depending on the size of the data from Analysis Services, one work around could be to import the data into into Excel and then make an Excel table and then use the Excel table as a data source. 
    Reeves
    Denver, CO

  • Power Query; How do I reference a Power Pivot table from a Power Query query

    Hi,
    It's pretty awesome how you can define Extract Transform and Load processes within Power Query without having to type in a single line of code. However how do I reference a Power Pivot table from a Power Query query to avoid me repeatedly accessing
    the same data source (CSV) file with a view to increasing performance?
    We are aware of the reference sub menu option with Power Query. However the new query created by the "reference" option still seems to refresh data from the data source (CSV) rather than just referencing the base query. Is this understanding
    correct? There does seem to be a lot of hard disk activity when re-running the new query which is based on a base query rather than a data source.  So we were hoping the new query would just need to reference the base query in memory rather than rescanning
    the hard disk. Is there any way to ensure that the reference query just rescans the base query in memory?
    Kind Regards,
    Kieran.
    Kieran Patrick Wood http://www.innovativebusinessintelligence.com http://uk.linkedin.com/in/kieranpatrickwood http://kieranwood.wordpress.com/

    Hi Kieran,
    This sounds like something to be suggested for a future release. At the present time, Power Query will always re-run the entire Power Query query when refreshed. The Reference feature is analogous to a SQL view whereby the underlying query is always re-executed
    when it's queried, or in this case refreshed. Even something like using the Power Query cache to minimise the amount of data re-read from the disk would be helpful for performance but the cache is only used for the preview data and stored locally.
    It would be a good idea to suggest this feature to the Power BI team via the feedback smiley face.
    Regards,
    Michael Amadi
    Please use the 'Mark as answer' link to mark a post that answers your question. If you find a reply helpful, please remember to vote it as helpful :)
    Website: http://www.nimblelearn.com, Twitter:
    @nimblelearn
    Hi Michael, 
    Glad to hear from you about this.  And thanks to Kieran for bringing a very good valid point to debate. Will be glad to see this in future release. 
    - please mark correct answers

  • Blank Record Appearing in Power Pivot, How do I ensure that my Power Query script is filtering out all the Blank Records

    Hi,
    I noticed once of my Fact tables in my Power Pivot model contains a blank record. I can't understand why.
    So I added a Filter expression to the underlying Power Query query, something like; -
    Text.Length("MyField") > 0
    However after refreshing the Power Query query which contains the above expression the blank record still appears within my Power Pivot table.
    Kind Regards,
    Kieran.
    Kieran Patrick Wood http://www.innovativebusinessintelligence.com http://uk.linkedin.com/in/kieranpatrickwood http://kieranwood.wordpress.com/

    hi..
    As my understand, the blank record in Power Query will be return null value . So that, You try to use Text.Length("MyField" ) <> null.
    Regards,

  • Does SharePoint 2013 have Power Query services similar to Power Pivot and Power View?

    Does SharePoint 2013 include services to host/support Power Query content similar to the way it hosts Power Pivot content in the Power Pivot Gallery? 

    Miguel.  Is there an update to this?  New information or new timeline?
    We have SP 2013 Enterprise, on-prem.  I'm finding that Power Query 's ability to get CSV or XML or other Web Services content, and then load it up to PowerPivot, is a way to address the lack of ODATA connectivity options on the web.  :-(
    I would very much like to be able to create an XLS doc, with a Power Query to a CSV file, Loaded to PowerPivot, and have SP's Excel Services/PowerPivot engine continue to update the content for me.  It fails today, as you stated above.
    Do you even think this will be updated for the on-prem version of SharePoint 2013 or will it only live in Power BI for Office 365?

  • The Power Query ribbon and the Power Pivot ribbon are gone !

    Hello,
    I created in Excel 2013, using Power Query, two large tables (more than 4 000 000 rows) according this principle: << However,
    the query pane that shows up on the side while data is loading lets you disable "load to worksheet". The data will continue to download. Once the download is done, you can click on "Load to Data Model" which will then get the data into
    the underlying xVelocity/data model.>>
    I built two calculated columns in order to create PRimary and Foreign keys for relations.
    I saved the xlsx file several times
    I Closed the xlsx file
    I reopened the xlsx file but, bad news, the
    Power Query ribbon and the Power Pivot ribbon are gone !
    I can check the the data connections are still there.
    What happened? What did I miss ?
    Thanks for your help
    Error message when I refresh connections :

    When I restart the computer, Power
    Query and Power
    Pivot ribbons come back. But if I enter a
    formula, they go away again.
    What could be the solution?

  • Power Pivot & Power Query

    I have very recently purchased yet another copy of Office, I now have every version from 97-2013, all Pro versions. I downloaded powerpivot for 2010 and that works fine. When I purchased (directly from MS) 2013 I thought powerpivot came with it as standard.
    I am agahst that it doesn't, I tried downloading it only to be told "I do not have the correct version"!!!
    What is going on MS?? nowhere does it say "CAUTION this Pro version does not have PowerPivot or PowerQuery, oh and what's more you can't download it either!"
    I am a developer who constantly hears from clients "I'm not having 2013..." maybe they know more than I do eh?
    Come on please play fair!

    Hi,
    Which Office 2013 version are you purchased? Office 2013 Professional or Office 2013 Professional Plus? As far as I know, Power Pivot & Power Query
    are available in the Office 2013 Professional Plus and Office 365 Professional Plus editions, and in the standalone edition of Excel 2013, not Pro version( If you only purchased Pro, you can't use them)
    Now Power Pivot is built-in add-ins with Excel 2013, we do not need to download it, just enable it.
    Go to File > Options > Add-Ins.
    In the Manage box, click COM Add-ins>
    Go.
    Check the Microsoft Office Power Pivot in Microsoft Excel 2013 box, and then click
    OK.
    https://support.office.com/en-us/article/Start-Power-Pivot-in-Microsoft-Excel-2013-add-in-a891a66d-36e3-43fc-81e8-fc4798f39ea8?ui=en-US&rs=en-US&ad=US
    http://office.microsoft.com/en-us/excel/download-microsoft-power-query-for-excel-FX104018616.aspx
    Hope it's helpful.Regards,
    George Zhao
    TechNet Community Support
    It's recommended to download and install
    Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
    programs.

  • Power BI - Office 365 data storage for Excel (Power Pivot & Power Query) architecture

    Hi, I trying to find a comprehensive view of the Power BI architecture, specifically where the data for Power Pivot & Power Query files are held. I've seen reference to the Vertipaq Engine and the data being stored in the Excel File, however this is
    in reference to 2010.
    Other references I've seen (SharePoint on premise with Power Pivot) have shown me that the data held in the Excel files on the SharePoint sites are actually held in SQL Server instances.
    Could anyone provide clarity on the architecture for Power BI please...
    Tim
    Thank you for you time folks!

    Hi Tim,
    let my try to explain this:
    When you load data into Power Pivot it is actually loaded into Vertipaq (in-memory column-store) and it resides there until you close Excel. Technically speaking Power Pivot is a single-user SQL Server Analysis Services instance hosted within Excel. When you
    close Excel a backup of the Power Pivot "database" is created and stored within the resulting .xlsx-file .
    Any user that has access to the Excel file also has access to the Power Pivot model, so there is actually no security
    This is very similar to any other excel file
    Once the Excel is stored in SharePoint / SharePointOnline and is access the first time the backup that is stored within the Excel is restored into SQL Server Analysis Services instance running in SharePoint Integrate mode
    all connections to the Excel-File are redirected to that SSAS instance. After some time of inactivity the model is unloaded from that SSAS instance again to free up the memory
    the data itself so basically always resides inside Power Pivot or respectively the Excel file
    Power Query only stores the actual query (= M-script) but no data
    hth,
    gerhard
    Gerhard Brueckl
    blogging @ http://blog.gbrueckl.at
    working @ http://www.pmOne.com

  • Can I update datas of Power Query after making relation tables in Power Pivot ?

    Can I update datas of Power Query after making relation tables in Power Pivot ?
    I want to update datas at day by day to viewing today's graph by Power View on Excel.
    Power Query use to transform web site to Power Pivot data model table.
    Power Pivot use to making relasing tables, and making measuers.
    Regards,
    Yoshihiro Kawabata.

    Hi Yoshihiro,
    Can you share an example of the things you are doing in Power Pivot? There are some operations that prevent the query from being refreshed in Power Query; this is not ideal so we are currently working on fixing it.
    Thanks for your feedback.

Maybe you are looking for