Sequencing records by Date

Question

Hey guys, I have a DAX question i was hoping you could help me with.  I've loaded a bunch of phone calls into power pivot and I want to do some different analysis on them based on the number of times and the order in which they where called.  I
also want to look at things like at which sequence number do I get the best results,  where in the sequence do things really start to fall off, etc.   
How can i got about using DAX to assign a sequence number to the phone numbers based on the date/time that they were called?
In T-Sql I would use OVER and PARTITION BY the phone number ordering by the date.  I am trying to do this in DAX on fly as a calculated column but can't seem to figure it out.  The end result should be a list of phone numbers with a sequence 1,
2, 3 for each time the phone was called, in the order that it was called.

Answer 1

I tend to worry about correctness first. We can certainly optimize on the functions I have provided.
FILTER() is a single-threaded iterator. In the case of the above examples, it is iterating row-by-row over the entire table FactCalls and applying the test. The first optimization we'll make is as follows:
COUNTROWS(
CALCULATETABLE( 'Call Results'
, ALLEXCEPT( 'Call Results', 'Call Results'[PhoneNumber] )
, FILTER(
VALUES('Call Results'[DateTime])
, 'Call Results'[DateTime] < EARLIER( 'Call Results'[DateTime] )
) + 0
This allows us to iterate over just the unique values of [DateTime], rather than all values in the table.
ALLEXCEPT() allows us to hit the storage engine, rather than the formula engine for maintaining the filter context on [PhoneNumber]
This probably won't save us too much time, though since the cardinality of [DateTime] is likely very near that of the whole table.
Thus, we need to limit the size of the table we iterate over in FILTER() even further:
=COUNTROWS(
CALCULATETABLE(
'Call Results'
, ALLEXCEPT( 'Call Results', 'Call Results'[PhoneNumber] )
, FILTER(
CALCULATETABLE(
VALUES( 'Call Results'[DateTime] )
, ALLEXCEPT( 'Call Results', 'Call Results'[PhoneNumber] )
, 'Call Results'[DateTime] < EARLIER( 'Call Results'[DateTime] )
)+ 0
In this case, we essentially duplicate our CALCULATETABLE() call in determining the table we will pass to FILTER(). Thus, the number of rows we must iterate over is just the number of distinct [DateTime]s that belong to the [PhoneNumber] on the current row
of the table. Now we don't have to check every [DateTime] that exists for its value compared to the [DateTime] on the current row of the table.
If we want to not perform this operation on [PhoneNumbers], we need to put that test outside of our calculation:
=
IF(
LEN( 'Call Results' ) = 10
, COUNTROWS(
CALCULATETABLE(
FactCalls
, ALLEXCEPT( FactCalls, FactCalls[PhoneNumber] )
, FILTER(
CALCULATETABLE(
VALUES( FactCalls[DateTime] )
, ALLEXCEPT( FactCalls, FactCalls[PhoneNumber] )
, FactCalls[DateTime] < EARLIER( FactCalls[DateTime] )
) + 0
, BLANK()
This will only return a sequence for numbers that are 10 characters/digits long, else it will just give a blank.
A few notes:
I added a 0 to the results above. I neglected to do this before, but doing so forces the result to be an integer, rather than a BLANK() for the first call of any sequence. I prefer 0 indexing, so I've added 0, but if you prefer 1 indexing, then adding 1
will achieve the same effect.
I mentioned the storage and formula engines. This is a rich topic of discussion, but for the purposes of this discussion we can leave it at this: The storage engine is faster and runs on many threads (depending on data size), but can only perform simple
operations; the formula engine is slower and can only run on one thread (no matter the data size), but can perform complex operations and logic. FILTER() is strictly a formula engine function. You may wish to do more research on this topic yourself.
You are likely not IO-bound, but CPU-bound when performing this operation - so the fact that it's an in-memory technology doesn't matter much. If you were performing the same operations in SQL Server, the working set of data would all be loaded into memory
anyway and you would similarly be CPU-bound in performing your operations. The important considerations are parallelism and which data abstraction (columnar vs row) and which data manipulation language combine to create the fastest processing. It may very
well be that this processing is faster in SQL than DAX, and that is not so much a reflection on either engine, but a reflection on the specific problem.