Serious performance problem - SELECT DISTINCT x.JDOCLASSX FROM x

Question

I am noticing a huge performance problem when trying to access a member that
is lazily loaded:
MonitorStatus previousStatus = m.getStatus();
This causes the following query to be executed:
SELECT DISTINCT MONITORSTATUSX.JDOCLASSX FROM MONITORSTATUSX
This table has 3 million records and this SQL statement takes 3 minutes to
execute! Even worse, my app heavily uses threads, so this statement is
executed in each of the 32 threads. As a result the application stops.
Is there any way that I can optimize this? And more importantly, can Kodo
handle a multithreaded app like this with a huge database? I've been having
a lot of performance problems since I've started doing stress & load
testing, and I'm thinking Kodo isn't ready for this type of application.
Thanks,
Michael

Answer 1

You can prevent this from happening by explicitly enumerating the valid
persistent types in a property. See
http://docs.solarmetric.com/manual.html#com.solarmetric.kodo.PersistentTypes
for details.
>
Inconveniently, this nugget of performance info is not listed in the
optimization guide. I'll add in an entry for it.This setting did in fact prevent the query from running which fixed the
problem. It definitely belongs in the optimization guide.
And more importantly, can Kodo
handle a multithreaded app like this with a huge database? I've beenhaving
a lot of performance problems since I've started doing stress & load
testing, and I'm thinking Kodo isn't ready for this type of application.I'd like to find out more information about details about your issues. We
do a decent amount of stress / load testing internally, but there are
always use cases that we don't test. Please send me an email (I'm assuming
that [email protected] is not really your address) and let's
figure out some way to do an analysis of what you're seeing.This email is just for posting to usenet, to avoid spam. I'm now running my
app through stress/load testing so I hope to discover any remaining issues
before going into production. As of this morning the system seems to be
performing quite well. Now the biggest performance problem for me is the
lack of what I think is called "outer join". I know you'll have this in 3.0
but I'm suprised you don't have this already because not having it really
affects performance. I already had to code one query by hand with JDBC due
to this. It was taking 15+ minutes with Kodo and with my JDBC version it
only takes a few seconds. There are lots of anti-JDO people and performance
issues like this really give them ammunition. Overall I just have the
impression that Kodo hasn't been used on many really large scale projects
with databases that have millions of records.
Thanks for configuration fix,
Michael