V2.2.1 is waisting connections / kodo 2.2.1 usable?

Question

Hi,
Summary of the problem: Since 2.2.1, the deepness of the object mapping
seems to have a direct correlation to the minimum number of connections
needed to execute any query. Insufficient connections cause a kodo system
lock.
I'm currently moving from version 2.1.3 to 2.2.1 and realized a behaviour
that seems to be quite problematc:
If the pool-part of system.prefs is missing, any deepness of object mapping
can be resolved by version 2.2.1. Good.
If the pool-part of system.prefs is provided with min=1/max=1 and the query
accesses an 1:1 mapping, the target objects of the query can be resolved.
Also good.
If the pool-part of system.prefs is provided with min=1/max=1 and the query
accesses an 1:n mapping, the target objects can't be provided because kodo
locks. Very bad.
If I change the max pool size to '2', the query can be executed.
We are currently working with a more complex extend respectively class
graph. Querying our class graph on the root level was no problem with kodo
2.1.3, but using kodo 2.2.1 we need a pool size of _13_ connections to query
some objects!!!! I wouldn't call this an expected behaviour.
If I don't use the kodo connection pooling and change the data source by
calling JDBCPersistenceManagerFactory.setConnectionFactoryName(jndiPath) to
use own connection pools, the problem is exactly the same: The pool must
have at least _13_ connections for our extent, or more generally spoken, at
least 2 connections for querying a 1:n mapping. I didn't check the number
for n:m, nor the exact number for first class objects as part of other FCOs
depending on their mapping.
So, as a summary, it seems as if the nesting of SQL-Statements and their
connection caching during execution has changed in 2.2.1 in a way that seems
quite problematic (or even unusable) not only for complex queries.
If a team is working on the same project it is usually also working on the
same test database which is sometimes also used by other development teams.
Opening _13_ or even more connections per connection pool (often more than
just one connection pool is needed) per team member to a central database
will easily lead to resource problems. This won't be the case on production
time, but code has to be developed prior, hasn't it :)
Did I oversee anything? Or is the problem know by anyone?
Cheers,
Christian Elsen

Answer 1

Hi Patrick,
> Summary of the problem: Since 2.2.1, the deepness of the object
> mapping seems to have a direct correlation to the minimum number
> of connections needed to execute any query. Insufficient
> connections cause a kodo system lock.
It was a long try-and-error-research, but I found an indicator to the
problem: first class objects that belong to the default-fetch-group.
In the attached example I use a class D with an Arraylist of class E
objects and query for all D objects. The pool size is always min=1/max=1.
-case 1: the list of E objects of class D does not belong to
the default fetch group => The query is executed as expected
-case 2: the list of E objects of class D belongs to
the default fetch group => The query is locked until the
timeout duration for the pool elapses (default=10sec).
If I increase the pool max to 2, the query is executed as expected.
I didn't try a default-fetch-group-recursion, but I expect a recursive
behaviour: Having an FCO in the default-fetch-group that has an FCO in
the default-fetch-group that has an FCO in the default-fetch-group....
leads to an increased number of connections needed to execute a query.
We currently work on a quite complex mapping and have to query for the
'root' of the Extend is some cases => therefore we today need a pool
with 14! connections to execute the query. The problem arised with kodo
version 2.2.1, in 2.1.3 one connection was enough to execute the query.
I wonder if the problem depends just on version 2.2.1 or maybe on our
environment, but the observations point to the kodo implementation.
In production this behaviour won't be acceptable, as I described in my
first comment. It would be nice to hear if the results are reproducable.
If they are, we maybe have to look for other jdo implementations, as our
production date is in about 3 weeks, although we're quite satisfied with
the implementation (not regarding the described big problem).
Regards,
Christian
PS: Are there any guidelines of how to size the pools depending on the
used mapping?
I cannot reproduce your problem. I'm trying with a test case with three
classes: A, B, and C. A has a list of B's and B has a list of C's. Both
of these lists are one-to-manys. I run a query on A whose filter digs
through B into C. pool/min is set to 0, and pool/max is set to 1. The
query works.
I've attached my test class and package.jdo. Can you alter this to make
it reproduce your problem, or provide me with code that displays the
behavior that you described?
Further, what database are you running against? I've only run my test
against InstantDB.
-Patrick