Pushing more calcs to Power Query - replacing DAX RANKX

I have another Power Query calculated column challenge.
Customer introduced a new requirement. Staff can work multiple shifts in a single business date. They would like a flag if the timesheet entries overlap. Many of their establishments close after midnight, which makes calculations a bit trickier, but there
is a field for the "business date" of the shift which helps.
You can imagine source data like this:
Full Name Date Start Date End Date
Joe Smith 8/1/2014 8/1/2014 10:00 8/1/2014 18:00
Joe Smith 8/1/2014 8/1/2014 22:00 8/2/2014 1:00
Jane Doe 8/1/2014 8/1/2014 15:00 8/1/2014 18:00
Jane Doe 8/1/2014 8/1/2014 17:00 8/1/2014 17:45
John Brown 8/1/2014 8/1/2014 19:00 8/2/2014 1:00
John Brown 8/1/2014 8/1/2014 14:00 8/1/2014 16:00
John Brown 8/2/2014 8/2/2014 10:00 8/2/2014 18:00
Mary Black 8/1/2014 8/1/2014 18:00 8/2/2014 1:00
Mary Black 8/1/2014 8/1/2014 15:00 8/1/2014 21:00
Bob Walker 8/1/2014 8/1/2014 11:00 8/1/2014 14:00
Bob Walker 8/1/2014 8/1/2014 17:00 8/1/2014 22:00
Bob Walker 8/1/2014 8/1/2014 19:00 8/2/2014 1:00
In this case, Jane Doe, Mary Black, and Bob Walker have overlapping shifts. Joe Smith and John Brown do not.
I did the calculation in calculated columns in Power Pivot by ranking the start time on the date, ranking the end time on the date, then brute force checking to make sure the start and end dates are in the right order using some intermediate calc columns.
To do the ranking, I used the DAX RANKX function.
Now if I want to push this sort of logic back to Power Query. How do I do this? I do not see a ranking function in the formula reference as a starting point for my calculation. 
Thanks

Cathy, the following are a couple of rank functions I created as part of a custom stat function "library." Hopefully, they are self-explanatory.
//RankEqual Function
(inputValue as any, inputSeries as list, optional orderDescending as nullable logical) as number =>
let
    SortedSeries = if orderDescending or orderDescending = null then
                                List.Sort(inputSeries,Order.Descending)
                           else
                                List.Sort(inputSeries),
    RankEqual = List.PositionOf(SortedSeries,inputValue)+1
in
    RankEqual
//RankDense Function
(inputValue as any, inputSeries as list, optional orderDescending as nullable logical) as number =>
let
    SortedSeries = if orderDescending or orderDescending = null then
                                 List.Sort(inputSeries,Order.Descending)
                           else
                                 List.Sort(inputSeries),
    DistinctSeries = List.Distinct(SortedSeries),
    RankDense = List.PositionOf(DistinctSeries,inputValue)+1
in
    RankDense

Similar Messages

  • Power Query DAX sortedrows

    Hi, I'm trying to filter a query using the current year - 5 (to get everything that's only 5 years old).  I've tried a number of functions but continue to get an error. 
    The UI produces the following statement if I type in a date:  = Table.SelectRows(ChangedType, each [IncorporationDate] > #date(2008, 8, 31))
    What I'm trying to achieve is : = Table.SelectRows(ChangedType, each [IncorporationDate] > #date(YEAR(TODAY()), 8, 31))
    Can anyone help point me to the right solution.
    Thanks, Tim.
    Thank you for you time folks!

    = Table.SelectRows(ChangedType, each [IncorporationDate] > #datetime(Date.Year(DateTime.LocalNow()), 1, 1, 0, 0, 0))
    This uses DateTime.LocalNow() to get the current date and which is then parsed through Date.Year to convert it to just the year.   The following numbers you could amend to the month, day and time needed.
    Here is a library of formulas if you were not aware
    Power Query Formula Library
    Regards

  • Getting the same day of the week from a previous year in a Power Query calc column

    Hi Power Query users,
    Would like to get your suggestions on this problem. I have a customer that wants to see the previous year's sales for the same day of the week. So today is Friday July 25, 2014. Customer would like to see sales for the closest Friday on the previous year,
    which was Friday July 26, 2013. Next year on Friday July 26, 2015, customer would see the sales for Friday July 25, 2014 and so on. Customer operates 7 days a week, all days of the year including holidays.
    What is the most elegant way to do this? I am assuming the best thing to do is to add this date as a calculated column in the date table, but I am not quite sure the best way to write the formula (including handling of edge cases - leap years, providing
    null when the previous year is not part of the date table, etc).
    I am starting with the
    date dimension as presented by Matt Masson.
    Please help if you have any formula suggestions for this.
    Thanks

    Okay, I'm making a couple of assumptions, so please let me know if these assumptions are incorrect:
    1) Instead of all sales data being in a single column, current year and last year sales are provided in different columns,
    2) The previous year sales column shows the sales occurring on the same
    date as the current year sales (so in your example, 43214 is the sales figure for 6/1/2013.
    If the above assumptions are true, we need to modify how the lookup is done. Let's assume that you created a custom column named SameDayOfWeekLastYear. Create another column called Previous Year Date, using the following function (Date_PreviousYear)
    (date) =>
    let   
        PreviousYearDate = Date.AddYears(date,-1),
        PreviousYearDateAdjusted = if (Date.Day(date)=29 and Date.Month(date) = 2) then
                                       #date(9999,1,1)
                                   else PreviousYearDate
    in
        PreviousYearDateAdjusted
    Incidentally, I created this function the week before your query, for a different purpose. In that case, it was to get the previous year sales occurring on the same
    date as the current year sales. It actually gets to your starting point, but with all the sales data in a single column to start with. However, when the current year is a leap year and you apply the Date.AddYears function, February 29th is calculated
    as February 28th of the previous year. If you then do a lookup for previous year sales, you get February 28th sales looked-up twice (the real Feb 28th and the bogus one resulting from taking February 29th back one year). The PreviousYearDateAdjusted identifier
    puts a truly bogus date instead of a duplicated Feb 28th - one that will always return null when looking up the sales value.
    This new date column will then be used to lookup the row of the previous year sales column that corresponds to the dates in the SameDayOfWeekLastYear column.
    Assuming that the last step in your query is InsertedCustom1, create another custom column (Previous Year Net_Same Day?), which returns the sales on the same day as the current year (the result that you're after). The formula would be:
    LookupValue(InsertedCustom1[Previous Year Net], InsertedCustom1[Previous Year Date], [SameDayOfWeekLastYear])
    where LookupValue is the custom function mentioned previously. After creating this new custom column, you can then remove all of the columns you don't need.
    Hope that the preceding is somewhat clear.

  • Aggregating across many-to-many relationship in Power Query?

    Hi, another M question.
    I am trying to push all my DAX calculated column logic back to Power Query and what I couldn't figure out is how to aggregate across my many to many relationship.
    Customer has many companies. He has sales for each company on a bunch of dates. Also, he has timesheets for each company for each date. Customer would like to calculate the labour cost for each company and show them as a ratio of sales in the sales report.
    (eg Kitchen labour cost should be x% of food sales)
    Now I am fine writing all the measures in DAX because I can put them in a table isolated from the Power Query tables. But calculated columns I would ideally push back to Power Query. 
    This is the DAX I constructed to put a calc column in the SalesReport table for the Kitchen labour costs
    =CALCULATE(SUM(Timesheet[Labour Cost]), FILTER(Bridge_CompanyDate, Bridge_CompanyDate[CompanyDateKey]=SalesReports[CompanyDateKey]), FILTER(Timesheet, Timesheet[Category]="Kitchen"))
    Is there a way to do similar types of aggregation in Power Query? I am tired of rewriting these calc columns when I have to reload the thing in the data model :(
    Thanks

    Hi Cathy. You can do this kind of calculation in Power Query, but it's not quite the same as in Power Pivot/DAX.
    If I understand your DAX expression correctly, you're wanting the sum of the [Labour Cost] column from the Timesheet table, for all rows with a [Category] of "Kitchen", where the timesheet date matches the date of the current row in the SalesReport table.
    In M, this would look something like this:
    = Table.AddColumn(Source, "Kitchen Labour Cost", each let expectedDate = [Date] in List.Sum(Table.SelectRows(Timesheet, each [Category] = "Kitchen" and [Date] = expectedDate)[Labour Cost]))
    (Instead of going through the Bridge_CompanyDate table, I'm matching the SalesReport and Timesheet dates directly just to make the example simpler.)
    Let me know if that helps.
    Ehren

  • How can I create dynamic file references in Power Query?

    Hi all,
    I'm new at using PowerQuery, and so far I like it. There's one thing I am struggling with though... Once I have set up my PoweQuery connections, I don't find an easy way to change the file to which the query is connecting. I'm using it for a monthyl recurring
    process, and every month the source data to query on woudl be different. The same in format/structure, but just a different dataset.
    Is there a way to make the source setup more dynamic? Can I for example in a parameters sheet enter the name and path of the new source file and update the queries?
    Currently the Advanced editor shows me following file reference:
    let
        Source = Excel.Workbook(File.Contents("Z:\Templates\EMEA\Source Data Tables\EMEA_EW_Source_Data_for_Power_Queries v1.xlsm")),
    Thanks in advance for suggestions

    Yes, this is something that you can do with Power Query. Here's how you can do it:
    Create a table in Excel containing your parameter value. Let's say that it has one column, called ParameterValue, and one row.
    Create a new Power Query query that gets the data from this table. Call the query something like ParameterQuery.
    In your original query you will now be able to reference values from your parameter query by saying something like this:
    Source = Excel.Workbook(File.Contents(ParameterQuery[ParameterValue]{0})),
    HTH,
    Chris
    Check out my MS BI blog I also do
    SSAS, PowerPivot, MDX and DAX consultancy
    and run public SQL Server and BI training courses in the UK

  • Visual studio online - Excel power query usage and samples needed

    Background
    Visual Studio Online does not offer much in the way of reporting, when compared to (onsite) TFS... unless you utilize the REST API functionality.
    The Visual Studio Online REST API is fairly extensive, but not very 'reporting' friendly to applications like Microsoft Excel.
    Question
    Are there any good examples of Microsoft Excel Power Query usage - to acquire and create reports via the Visual Studio Online REST API?  I've searched - but surprisingly haven't found anything of substance.  It is fairly simple to call a VSO REST
    query using Excel Power Query, but not so simple to iterate through the lists within lists, etc... unless you are familiar with 'M language'.

    Hi,
    Thank you for reaching out to us. I am currently researching to gather more information with regards to your request. I shall revert back to you with an update at the earliest. Sincerely appreciate your patience.
    Regards,
    Nithin Rathnakar

  • Power Query/Data Management Gateway data refresh not working

    Pretty new to o365/Power BI, but here's what I've got going on (hopefully someone can help me out):
    I created a data management gateway and data source.  The data source says it's online in the PowerBI admin center, and it seems to be working correctly.  I can open Excel 2013 on my desktop (logging in as my trial o365 account which has Power
    BI associated with it) and connect to the data source via Power Query using the oData option.  I make sure to import the data into the model, and then open up the PowerPivot window and create a simple pivot table using the data in the model.
    that all works just great.  The problem comes when I upload the workbook and try to update it.  I've tried a few different ways.
    1. When I try to manually refresh the workbook by opening it in my o365 site and going to data-->refresh I get the following error:
          An error occurred while working on the Data Model in the workbook. Please try again.
          We were unable to refresh one or more data connections in this workbook.
          The following connections failed to refresh:
          Connection: Power Query - dbo_DimProductCategory
          Error: Out of line object 'DataSource', referring to ID(s) 'a75593f3-c34d-4f83-9458-49aa2cece164', has been specified but has not been used.
          The following system error occurred: Class not registered
          The provider 'Microsoft.Mashup.OleDb.1' is not registered.
          Power Query - dbo_DimProductCategory
    2. When I go into Power BI and go to "Schedule Data Refresh" for the workbook, I get the following error:
          Sorry, the data connections in this report aren’t supported for Scheduled Refresh.
          Technical Details â–¼
          Correlation ID: B3CE4B10-2137-E593-6FCF-189B73465190
          Date and Time: 03/31/2014 06:20:39 PM (UTC)
    Any help would be greatly appreciated.  If you need additional information, I'd be happy to provide it.

    Hey Guy,
    Thanks for the reply.
    The "data source type" of the data source in Power BI is "SQL Server" (there was only that and Oracle available to select from)
    The "data source type" that Power Query is using in Excel is "From oData Feed"....which is using the Power BI data source....which is using the data management gateway.
    A few follow up questions if you have a second
    1. Do you know when PowerQuery data refresh will be supported? (just a general idea....weeks, months, next year?)
    2. Is there any other way to connect to (and be able to refresh) PowerBI data sources referencing "on prem" data via the data management gateway?  I tried using the oData feed URL in non-PowerQuery areas of excel (excel data tab, PowerPivot directly)
    but it didn't work.  If there's some other way to connect to and refresh on prem data, i'm all ears :D 
    Really appreciate your help, thanks for taking the time.

  • POWER QUERY Get External Data From File From Folder (Excel 2013)

    Hi,
    Beginner's question :
    What could be the use of the query on a folder : we just get a list of files with their path. What can we do with that?
    Thanks

    Hi,
    Do you want to combine data from multiple Excel Files in the same folder path into one table? If I understand correct, we can add a custom column to import the data.
    After we getting a list of files with their path, the Query Editor window will activate to show you a table containing a record for each file in the chosen directory. These will provide our function with the needed FilePath and FileName parameters. 
    Function sample: File name([Folder path],[Field name]
    For more detailed steps, please see the article:
    http://datapigtechnologies.com/blog/index.php/using-power-query-to-combine-data-from-multiple-excel-files-into-one-table/
    Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.
    George Zhao
    TechNet Community Support
    It's recommended to download and install
    Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
    programs.

  • How to use Power Query to load data from a single source into multiple tables

    Hi all,
    I have a requirement to load my data into three different data models using Power Query. Is that possible?
    My source is a SharePoint survey list, with similar questions like:
    1) Course lecturer - John Doe
    1a) The course was useful (rate 1 to 5)
    1b) The lecturer displayed good knowledge of topic (rate 1 to 5)
    2) Course Lecturer - Mary Brown
    2a) The course was useful (rate 1 to 5)
    2b) The lecturer displayed good knowledge of topic (rate 1 to 5)
    I would like to split the data into separate data models (one for John Doe; another for Mary Brown), so that I can compare the different lecturers. Other than running separate surveys for each of them, I thought of using Power Query to transform the data.
    Is it possible?
    Thanks.
    Regards
    GM

    Yes, this is possible.
    Start with a single query that returns you the data for all lecturers.
    Right-click on the "all lecturers" query in the queries pane, and choose Reference, once for each lecturer. This will create a query for each lecturer.
    Open the query for each lecturer and filter the data so that only that lecturer's results are visible.
    Click "Close & Load To..." for each lecturer's query  to load the data into the data model. This will create a data model table for each lecturer.
    If your question is more about how to transform such a survey list into a table that can be easily filtered, please provide an example of how the list shows up in Power Query.
    Ehren

  • New SP Custom List results in Power Query Error: Expression.Error: The key did not match any rows in the table.

    Hi,
    I'm running into something that seems to be a bug in Power Query. I've got my Power Query tables from SharePoint lists into Excel and Power Pivot data model. (Making sure the SharePoint site is in English and I replace MMD columns with normal columns). 
    But then, when I close down Excel, add another custom list to SharePoint, open the Excel workbook and want to add that new custom list using the same connection (via recent data sources).
    It shows the newly added custom list in the explorer but with an error:
    Nothing I can do to fix this; restarting Excel or PC; nothing. 
    The only way I got around this was by removing all my existing data sources and re-create them.. 
    after that, the same SharePoint custom list WAS read with Power Query without an issue.
    Thanks,
    Jeroen

    Its because of recent updates to Power Query.
    In recent updates internally their is some change in syntax for reading particular table from list (I faced this issue while reading data from Excel file, i.e After applying recent update of Power Query my Queries which were working fine before starting
    throwing same error that yo are facing).
    As solution their is small tweak to be done in existing queries, instead of recreating all queries. If you can provide me your complete old query from Advanced editor then I can point to tweak that need to be done.
    Thanks,
    Sagar K

  • Add actions to an existing query or function in Power Query

    Hello
    I have just started using Power Query and can see this will be an amazing tool to add to my tool set. However I am rather slow on the uptake and am having trouble with some of the basics of using Power Query
    I have successful created some queries and by following along on some articles at excelguru.ca/blog i have created a usable function or two
    The issue i have and can see this being something i will do a lot in the near future (especially while still learning), is that I need to go back to the query and function and edit it. In particular i need to either rename some columns to make them compatible
    with other data sources, or delete them.
    I can see that i can manually edit the query / function in the advanced editor, but i was wondering if it is possible to open the report up again and edit the columns in the same manner as when creating them in the first place?
    Thanks in advance for any help given.
    I am using Excel 2010, although do have access to Excel 2013 if needed, but as i am using PowerPivot in 2010 i would prefer to use PQ in 2010 also.
    Proportal

    OK - What had completely stumped me, and i had to set it aside to do other things, has just been solved!
    To explain - obviously editing Queries is simple - right click the query and edit... and then make the changes you want.
    For functions - the thing i was really struggling with is also fairly simple, but takes a few more steps.
    1 - copy the query from the advanced editor
    2 - create a new blank query from Get External Data on the Home Tab / From Other Sources / Blank Query
    3 - Past your function in and remove the function statement(s) and reinstate and file path information
    4 - Edit the columns etc as you want.
    5 - when completed paste the relevent sections back into the original function.
    Done.
    Hope this helps any others out there that were struggling with something so simple... 
    Moral of this - copy and save all your queries and functions in a text editor such as notepad++ so you can cut out a lot of the steps above...

  • Sharepoint 2013 support for power query

    Hi,
    I wanted to know whether SharePoint 2013 supports power query and can we refresh power query data from excel services.
    Thanks,
    Bhawna.

    Hi Bhawna.,
    According to your description, my understanding is that you want to know whether SharePoint 2013 support Power Query and refresh power query data from excel services.
    The  refresh of workbooks involving Power Query queries isn't currently available in SharePoint 2013.
    Here are some similar posts for you to take a look at:
    http://social.technet.microsoft.com/Forums/en-US/62b6384e-3120-40e2-9469-01c808530f55/does-sharepoint-2013-have-power-query-services-similar-to-power-pivot-and-power-view?forum=powerquery
    http://social.technet.microsoft.com/Forums/en-US/2da77984-5a97-4102-9119-97ecbc39f527/publish-power-query-workbook-to-sharepoint-2010?forum=powerquery
    In addition, I am not an expert for Power Query, you can create a new thread on the Power Query forum, more experts will assist you with this issue.
    Power Query forum:
    http://social.technet.microsoft.com/Forums/en-US/home?forum=powerquery
    I hope this helps.
    Thanks,
    Wendy
    Wendy Li
    TechNet Community Support

  • The Power Query ribbon and the Power Pivot ribbon are gone !

    Hello,
    I created in Excel 2013, using Power Query, two large tables (more than 4 000 000 rows) according this principle: << However,
    the query pane that shows up on the side while data is loading lets you disable "load to worksheet". The data will continue to download. Once the download is done, you can click on "Load to Data Model" which will then get the data into
    the underlying xVelocity/data model.>>
    I built two calculated columns in order to create PRimary and Foreign keys for relations.
    I saved the xlsx file several times
    I Closed the xlsx file
    I reopened the xlsx file but, bad news, the
    Power Query ribbon and the Power Pivot ribbon are gone !
    I can check the the data connections are still there.
    What happened? What did I miss ?
    Thanks for your help
    Error message when I refresh connections :

    When I restart the computer, Power
    Query and Power
    Pivot ribbons come back. But if I enter a
    formula, they go away again.
    What could be the solution?

  • Can I copy Power Query queries to other Excel workbook ?

    Can I copy Power Query queries to other Excel workbook ?
    I want to copy some Power Query queies to other Excel workbook.
    a workbook have some queries about population.
    a workbook have some queries about labor market.
    I want to build a workbook with population queries and labor market queries for sharing OneDrive.
    Regards,
    Yoshihiro Kawabata

    Thank you, Curt.
    Yes, Now I copy by M code.
    I want  more easy way.
    When sharing Query at Power BI, Excel automatically share all related queries by one click.
    Regards,
    Yoshihiro Kawabata 

  • What can Power BI Replace ?

    Being new to Power BI have few basic questions...
    1. What is Power BI suppose to replace from Legacy Microsoft BI tools - Performance Point Server? 
    2. Is Power BI an Enterprise analytic, reporting, dashboard tool ? Where does Power BI fits in Enterprise ?
    Is it suppose to replace SSRS, SSAS (TAB/MDM) and its supported security models...if no why not...why do Microsoft create so many BI tools instead merging BISM Model into one single platform to drive all analytic, reporting, dashboarding across desktop,
    mobile and cloud platform.

    Hi,
    The Power BI Designer is a client tool that will enable you to build reports against a wide range of on-prem and cloud-based data sources. It combines the capabilities of Power Query, Power Pivot and Power View into a single unified experience, optimized
    for the workload of creating rich, interactive reports.
    Reports that you create with the Power BI Designer can be published to the Power BI Dashboards Preview service, where you can create dashboards and share them with other users, so they can consume them easily on the web or mobile apps.
    We just released the first Preview version of the Designer a couple of months ago, and we're currently working on several new features. We're delivering monthly updates to this tool (as well as the service). You can stay tuned to these updates in our official
    blog (http://blogs.msdn.com/powerbi) or via "Update Notifications" when you launch the Power BI Designer and there is an update available to you.
    Regards,
    M.

Maybe you are looking for