Slow, then fast execution of same query

Question

I have a test query:
dbxml> query 'count(collection()/Quots/QuotVerificationInstance/@lastModifiedDate[.="2009-09-07"])'
With this index:
Index: node-attribute-substring-string for node {}:lastModifiedDate
The first time I run the query in the shell, it takes a while:
Query - Finished query execution, time taken = 29656ms
The second time I run the query, it's a more acceptable speed:
Query - Finished query execution, time taken = 735ms
Can anyone explain the difference in search speed? The slow, fast pattern is pretty consistent (I've tried fresh shell sessions a few times and get the same initial slow search followed by the quicker one.)
What I'm trying to do is allow on-demand generation of a table showing the number of records modified per day and per month. The test date I'm searching for above is the date of insertion of records so it returns a comparatively large result set at the moment, but 29-odd seconds is way too slow to make the query comfortable. I've added a substring index because sometimes I want a partial date search (e.g. when counting records modified per month). I remember reading that a substring index doubles as an equality index. The date is stored simply as a string, not a date type.
What am I missing? I'd like to understand the difference in execution times, but basically I need it to run more quickly for simultaneous date queries. I'd be grateful for any pointers.
Tim

Answer 1

Hi Viyacheslav
I think it's perhaps too much trouble for us both for me to try and share the container -- my managers would want assurances about copyright (the data is part of a previously published dictionary), and I'd have to build a smaller one as the current file is > 500MB. But thanks for your interest.
My container setup is pretty simple. It's around 45000 small XML files added to a node container for a read only database (no environment). I have 16 indices supporting a range of different searches and a couple of reports, and the only index relevant to the @lastModifiedDate search is the node-attribute-substring-string index mentioned above (when I tried other edge or equality indexes I deleted the substring index). The query script I intended would be instantiated purely to run the one query and then die each time, so I don't see there being any chance of getting past an initial slow query if that's the behaviour. Because that first query finds around 37000 matches among the 45000 dates, it's understandable that it would take a while but I'm not sure why it speeds up in the shell from the second time round. An obvious suggestion would be that some form of optimisation is taking place in the shell, but I don't understand the workings of DBXML well enough to guess how. Even if this is the case, knowing that it happens wouldn't help me speed up my query due to the life cycle of the script.
Perhaps I should just generate these stats on a daily basis rather than generating them on-demand, or else try a different approach at getting the counts. But thanks again for the comments.
Tim