My thoughts on testing DocumentDB

Question

Despite knowing DocumentDB won't be an option yet for my needs because of the lack of OrderBy and other known limitations in the Preview, I wanted to try it out and run some basic query tests against it to see what's already possible, how it performs, where
it lacks features and if it would make sense to consider DocumentDB as a future replacement for my current combined database Azure (SQL Server + Table Storage) solution.
I want to share my findings as a feedback on this preview.
My scenario
While the big picture is much more complex, for this post and my DocumentDB test, I reduced my app functionality to it's very basic requirement: Users can subscribe to news channels and have all articles of their subscriptions shown in a combined list. There
are thousands of news channels available and users may subscribe to 1 to 100s or even 1000s of them, while 1-100 is the common range of subscriptions. The app has tagging for read/unread articles, starred articles and everything can also be organized in folders
and users can filter their article lists by these tags - but I left all of these complexities out for now.
DocumentDB architecture
One collection for News Channels, one collection for Articles. I decided to split the channels from articles as there are some similarities in column names and this would have made issues in index design. I imported around 2.000 NewsChannel rows from my
SQLDB and around 3 million articles, filling up the Articles collection to nearly 10 GB of data.
A NewsChannel document looks like this:
id - I took the int value from my SQL database for this
Name
Uri
An Article document looks like this:
id - I also took the int value from my SQL database for this
NewsChannelId - in SQL DB, the foreign key from the NewsChannel table
Title
Content
Published - DateTime converted to Epoch
I put range indexes on id and Published as most of the time, I'd query for ranges of documents (greater than an id, newer than a specific published date, ...). I also excluded some columns from indexing, like Content in the articles document.
Test 1 - Get newest 50 articles for a single news channel
SELECT TOP 50 * FROM ArticlesWHERE NewsChannelId == [50]ORDER BY id DESC
I knew this would fail due to the lack of OrderBy. I tried to find a solution by custom indexes, but there is no way to define an index to be organized in descending order so newest entries would always be returned first. This would be enough as I not really
need ascending orders for articles, so this would have made up for the lack of OrderBy. But it does not seem to be possible.
Result: Impossible
Test 2 - Get newest 50 articles for all subscribed news channels of a user
SELECT TOP 50 * FROM ArticlesWHERE NewsChannelId IN ([1, 6, 100, 125, 210, ...])ORDER BY id DESC
This would be the most used query and it would have been very interesting to see how this could perform, but due it's similarity to Test 1, it's also not possible to do it. But a variant of it will be described in the next test (3).
Result: Impossible
Test 3 - Get any articles newer than a given article from all subscribed news channels of a user
This was the first test where I hoped to get some results. Each article document has a range index on id and its Published date, so this should be fast and nice. I seemed to have failed to create the range index for id correctly as DocumentDB complained
about id not being a range index - that sucked because to fix this I would have to recreate the whole database and re-import all data. But luckily, the index on Published was created correctly and it would do for testing this kind of query just as fine as
the id.
SELECT * FROM ArticlesWHERE NewsChannelId IN [1, 6, 100, 125, 210, ...]AND Published > [someDate]
Unfortunately, I found out there is no "contains" query supported in DocumentDB that would work like a WHERE IN query in SQL. But if I want to query articles for all subscribed channels, I will have to pass a list of NewsChannel IDs to the
query. That was really a surprise to me as something like this seems just as much as a base functionality like OrderBy.
Result: Impossible
Test 4 - Get any articles newer than a given article for a single news channel
Just as test 4, but only for 1 news channel - so finally here, DocumentDB will support my needs.
SELECT * FROM ArticlesWHERE NewsChannelId == [id]AND Published > [someDate]
And yes, this works, and the performance seems OK. But to my surprise, even if this just returns 5 documents and the query is well supported by the range index on Published, this has extremely high RU costs - depending on the query, somewhere between 2.500
and 6.000 in my tests - which would mean with 1 CU, I already will be throttled for such a simple query.
Result: Possible, quite fast but insanely high RU costs
Test 5 - Get a single article from a News Channel
As expected, this works like a charm. Fast and with 1 RU cost per query.
Result: Works great.
Other stuff I noticed:
For my scenario, I see no way to scale my DocumentDB. I already reached the limit of a single collection with only a fragment of all my data. I would need to do the partitioning myself, for example by having a single collection for each NewsChannel, like
I did in TableStorage where the NewsChannelId is the partition key - but due to the collection number limitations and even more due to the limitied query capabilitis INSIDE a single collection, I see now way how I could do performant queries if I would need
to query multiple, maybe even hundreds of different collections in one query.
Even if the space limit of a single collection would be raised to terabytes of data, I see the issue that I will run into serious performance problems as, as I understand, a collection can always be only on a single node. To support more load, I will be
required to split my data over multiple collections to have multiple nodes, but then again, this would not support my query needs. Please correct me if I'm seeing something wrong here.
Wrap up
Seeing that even my most basic query needs cannot be supported by DocumentDB right now, I'm a bit disappointed. OrderBy is coming, but it won't help without WHERE IN queries and even with them, I still don't know if this is something that will perform good
in combination and what, in such cases, the RU costs will look like if simple range queries with a small amount of documents returned already cost that much.
I'm looking forward what's happening next with DocumentDB, and I really hope I can replace my current solution with DocumentDB at some point, but currently, I don't see it. A good fit for me would be MongoDB, but it's not PAAS and it's hard and resource-intensive
to host ... so DocumentDB looked very nice at first sight. I really hope those issues will be resolved, and they will be resolved soon.
b.

Answer 1

Hi Aravind,
thank you very much for your detailed response.
Test 1: That's a good idea for a workaround, although it would get complicated when I want the top 50 documents from all subscribed news channels, which can be 100 or more (Test 2). The index documents can also get pretty large which might bring me to the
limit of a single document, needing to split it on multiple documents for a single news channel. However, for a proof of concept implementation with DocumentDB, this will do fine. I might try that :)
Test 2: Yes, but the ORs are limited to a maximum of 5 (?) currently, so not really an option as I need more most of the time.
Test 3: I will have a look at this and see how that performs!
Test 4: I used a classic UNIX epoch timestamp (seconds since 1970) and I also used a precision of 7 for the index. See below the code I used to create the index. So I think this should be OK. However, I'm glad to share the details of my account and
a sample query so you can have a look for yourself. I will contact you by Mail with details
articleCollection.IndexingPolicy.IncludedPaths.Add(new IndexingPath
IndexType = IndexType.Range,
Path = "/\"InsertedEpoch\"/?",
NumericPrecision = 7
As for partitioning - thanks for the article. For me, a fan-out on read strategy would be required if I would do my partitioning by News Channel ID ... but that's what giving me headaches. Given that it is not uncommon a user of my app has 100 or more
subscriptions, I would need to issue 100 parallel queries. I tried something like that for Azure Table Storage and found it to be a performance nightmare. That's why I currently use Table Storage as a pure document store but still do all computations of the
requested articles in SQL Server. But yes, I might have to put more thought into that and see how I can squeeze out the performance I need. Because SQL Server is fine and I can do a lot with indexes and indexed views to support my query scenarios - but
it has also it's scalability limits and the reason it still works good is that my App is in a testing/beta state and does not have the amount of data and users it will have when it is finally live. That's the reason I am searching for a NoSQL solution like
DocumentDB that should support my needs, mainly for scale-out, better.
Thanks again for your response and your suggestions, with that, I might be able to do a basic proof of concept implementation that supports my core features for some testing with DocumentDB and see how it's doing.
I will contact you per mail for the RU test data
Happy new year! :)
Bernhard