My experience migrating Kodo 3.4 to 4.1

Question

Hello Stefan,
I struggled with Kodo 4.0 and gave it up. Kodo 4.1 seems to be a much better
release. I migrated my app in a day. First I managed to run it against 3.4
metadata with some property file changes (migration docs are not very good
and miss few things but may be if you use Kodo automated migration tools it
will do for you what I was doing manually) . If you use lots of custom
filed mappings (especially non-trivial mappings) allocate much more time for
conversion - the whole thing has changed. I have not had chance to convert
my mapping and had to bandate it with externalizers and other things for
now. One thing you lose in kodo3.4 mode is ability to query by interface
since now it must be explicetly.
Couple of tips here
- kodo.Log: kodo(DefaultLevel=INFO, SQL=TRACE...) kodo is no longer valid
logger name
- kodo.jdbc.DBDictionary: oracle(BatchLimit=30) BatchLimit=30 is no longer a
valid option use
kodo.jdbc.SQLFactory: (BatchLimit=50) instead
- kodo.PersistentClasses= ... is no longer works use
kodo.MetaDataFactory: kodo3(Types=....) in kodo3 mode or
kodo.MetaDataFactory: (Types=...) in jdo2 mode
- Any SQL with DATE column is no longer batched leading to 2-3 times
performance drop. The decision swa made based on bugs in oracle 9 drivers in
batching mode. If you have latest drivers (and database) from my experience
you will not have any problems. So to reenable it you can register your own
instance of AdvancedSQL (now a factored out part of DatabaseDictionary):
kodo.jdbc.SQLFactory: (BatchLimit=50,
AdvancedSQL=com.peacetech.jdo.kodo.kodo4.patch.OracleAdvancedSQL)
where OracleAdvancedSQL could look like:
public class OracleAdvancedSQL extends kodo.jdbc.sql.OracleAdvancedSQL {
@Override public boolean canBatch(Column col) {
switch (col.getType()) {
case Types.DATE:
return true;
default:
return super.canBatch(col);
- I have not tested read performance much since I was concentrating on
writes. But write performance even with batch enabled does not seems to be
not up to 3.4 level I observed a consistent 30-40% decrease in performance
while persisting large graph complex of fairly small objects. I ran 3.4 and
4.1 versions side by side against a dedicated Oracle 10 server and noticed
performance decrease of 30-40%
SQL generated by both versions was very close if not identical (I only did
spotcheck) but incase of INSERT you would not expect it any different anyway
I tried profiling 4.1 version and found some significant hot spots but could
not dechipher them in any reasonable time because of huge depth of stacks
and lack of source code. I might try it again if I have time because
performance is critical for us.
- I have not tried any new/advanced features yet. including new mappings,
detachment, data cache, quality of eager fetch so I can not give you any
good feedback on that. At least I can say that this release worth trying -
after migration my application worked as expected except for lower
performance
I also have to say I do not know how well automated migration of kodo 3.4
metadata to jdo2 metadata works (if it exists) because I use my model driven
code generator and I just developed JDO2 plugin for it and regenerated my
metadata from the model (I did not have to gegenerate my java classes of
course)
Alex
Then I created native JDO2 mappings and everything

Answer 1

Denis,
Could you email it to me please shurik at peacetech dot com
Thanks
Alex
"Denis Sukhoroslov" <[email protected]> wrote in message
news:[email protected]...
Alex,
The issue was in version 3.4.1. BEA has provided a patch, no new version.
Denis.
"Alex Roytman" <[email protected]> wrote in message
news:[email protected]...
Denis,
In which version did you observe it and which version fixed it?
Thank you
Alex
"Denis Sukhoroslov" <[email protected]> wrote in message
news:[email protected]...
I don't know, I didn't tried 4.1 yet. It is possible that this issue
was'n exist in kodo 4.x at all.
"Christiaan" <[email protected]> wrote in message
news:[email protected]...
Nice! Is it also solved for 4.1?
regards,
Christiaan
"Denis Sukhoroslov" <[email protected]> wrote in message
news:[email protected]...
Finally, BEA has solved the issue I mentioned. Reading cached PCs
which have embedded objects become much faster (about 10 times in my
tests).
Thank you very much to all why was involved in this job.
Denis.
"Denis Sukhoroslov" <[email protected]> wrote in message
news:[email protected]...
Hi Alex,
I know about default-fetch-group, of course I marked these embedded
fields properly. You're right, it is not a cache miss but an
unnecessary fetch from DB. It's strange that nobody has found this
before. I managed to create a stanalone test case and send it to BEA
support. They agree that it is a bug, but still can't fix the issue.
The test is quite small, so if anyone interested I can send it here.
Denis.
"Alex Roytman" <[email protected]> wrote in message
news:[email protected]...
Hi Denis,
That's very strange. All custom fields such enums etc. are
essentially mapped onto regular JDO mandated types. I use it all the
time and have not observed this behavior but I might have missed it
of course. I have a suspicion that what you are seeing is not cache
misses but rather fetches outside of default fetch group. Keep in
mind that Kodo does not fetch any custom field as part of default
fetch group unless you explicetly specified it in your package.jdo
So, try to mark all your custom mapped fields with
default-fetch-group="true" and I suspect all your extra database
selects will disspear.
Read speed, inded, is always the critical part. I just have not had
chance to play with 4.1 reads enough to say if it is faster or
slower. There are more ways to optimize reads (various flavors of
eager fetches, cutom optimized mapping of collections including
embedding ...) but very few optimizations for updates
Alex
"Denis Sukhoroslov" <[email protected]> wrote in message
news:[email protected]...
Hi Alex.
My question is out of this topic, but it looks like you may have an
answer. BEA support did nothing for the last few months.
We still use kodo 3.4.1. DB is Sybase 12.5.x. In our app we're very
concerned on performance as well. But, we do have much more reads
then writes. So, we're trying to cache as much as possible. Kodo
distributed cache works quite good. At least, it presents better
performance then Gemfire and Tangosol, on the same use cases. But
we found a bug in its caching mechanism: when you have a persistent
class and this class has an embedded attribute of some
non-primitive type (like enum or just a simple complex type with
one or two attributes in it) kodo bypasses cache and performs a
select to DB each time. Have you seen this? Is it possible to solve
via custom mapping, what do you think?
Thanks. Denis.
"Alex Roytman" <[email protected]> wrote in message
news:[email protected]...
Hello Stefan,
I struggled with Kodo 4.0 and gave it up. Kodo 4.1 seems to be a
much better release. I migrated my app in a day. First I managed
to run it against 3.4 metadata with some property file changes
(migration docs are not very good and miss few things but may be
if you use Kodo automated migration tools it will do for you what
I was doing manually) . If you use lots of custom filed mappings
(especially non-trivial mappings) allocate much more time for
conversion - the whole thing has changed. I have not had chance to
convert my mapping and had to bandate it with externalizers and
other things for now. One thing you lose in kodo3.4 mode is
ability to query by interface since now it must be explicetly.
Couple of tips here
- kodo.Log: kodo(DefaultLevel=INFO, SQL=TRACE...) kodo is no
longer valid logger name
- kodo.jdbc.DBDictionary: oracle(BatchLimit=30) BatchLimit=30 is
no longer a valid option use
kodo.jdbc.SQLFactory: (BatchLimit=50) instead
- kodo.PersistentClasses= ... is no longer works use
kodo.MetaDataFactory: kodo3(Types=....) in kodo3 mode or
kodo.MetaDataFactory: (Types=...) in jdo2 mode
- Any SQL with DATE column is no longer batched leading to 2-3
times performance drop. The decision swa made based on bugs in
oracle 9 drivers in batching mode. If you have latest drivers (and
database) from my experience you will not have any problems. So to
reenable it you can register your own instance of AdvancedSQL (now
a factored out part of DatabaseDictionary):
kodo.jdbc.SQLFactory: (BatchLimit=50,
AdvancedSQL=com.peacetech.jdo.kodo.kodo4.patch.OracleAdvancedSQL)
where OracleAdvancedSQL could look like:
public class OracleAdvancedSQL extends
kodo.jdbc.sql.OracleAdvancedSQL {
@Override public boolean canBatch(Column col) {
switch (col.getType()) {
case Types.DATE:
return true;
default:
return super.canBatch(col);
- I have not tested read performance much since I was
concentrating on writes. But write performance even with batch
enabled does not seems to be not up to 3.4 level I observed a
consistent 30-40% decrease in performance while persisting large
graph complex of fairly small objects. I ran 3.4 and 4.1 versions
side by side against a dedicated Oracle 10 server and noticed
performance decrease of 30-40%
SQL generated by both versions was very close if not identical (I
only did spotcheck) but incase of INSERT you would not expect it
any different anyway :-)
I tried profiling 4.1 version and found some significant hot spots
but could not dechipher them in any reasonable time because of
huge depth of stacks and lack of source code. I might try it
again if I have time because performance is critical for us.
- I have not tried any new/advanced features yet. including new
mappings, detachment, data cache, quality of eager fetch so I can
not give you any good feedback on that. At least I can say that
this release worth trying - after migration my application worked
as expected except for lower performance
I also have to say I do not know how well automated migration of
kodo 3.4 metadata to jdo2 metadata works (if it exists) because I
use my model driven code generator and I just developed JDO2
plugin for it and regenerated my metadata from the model (I did
not have to gegenerate my java classes of course)
Alex
Then I created native JDO2 mappings and everything