Help: looking for Serialization suggestions (on strategies and products)

Question

Hello,
I'm looking for suggestions on how to tackle a Serialization issue. Suggestions on tools to use, and approaches for "do it yourself" coding are both needed.
I need to serialize a large evolving codebase with a minimum of rewriting (overriding readObject... for each class is out of the question, likewise embeding metadata comments for all classes is also out of the question). Currently the codebase is using Java's default Object Serialization (to a binary array). This is completely unacceptable for the long term (object versioning and migrations are a nightmare).
As such I've determined that JSX/JSX2 looks like a feasable solution (the pricing is nice; the mechanism wrappers readObject, writeObject (so a minimum of recoding); they automatically map objects back and forth and don't require Objects to be constrained (with meta data, being a given format (such as beans...))). I was leaning towards Object -> XML because:
1) it is easy to read and therefore parse / upgrade / version
2) it would fit into many different types of databases without need of specalized decompositions (just put the whole object in a data cell)
however, management really wants to go Object -> tables. I am a bit leery about this but I keep hearing "we have no plans to move to any data technology other than RDB". Given those assumptions, it kind of pulls me in another direction. Namely a table serialization would have greatly enhanced performance (one could scan for objects with the highest value of some instance var quite quickly; with the other scheme it would be rather expensive...). So now I'm looking for suggestions on products that serialize Object -> table. The product at www.objectmatter.com looks like it might work well, but the licencing seems expensive. JSX/JSX2 might have Object -> table capability in the future but not now.
Likewise, loathe as I am to reinvent a complex wheel, I am looking for strategies on rolling my own object -> table mapping code.
On the one hand I think I might implement the serialization in a way similar to JSX (wrapper readObject...) as this would seem to give a direct route to the relevant instance variables.... On the other hand this would be even more slow than Java's current serialization, and I think I might go a bit blind parsing the binary stream. I could write some "stand-alone" that scans .java or .class files and auto generates a mapping file, then I could use some tool like Castor (however, given the complexities of some of my classes I think they might break Castor). The big problem is that my classes are complex and ugly (inner classes, anon classes...) so I get the feeling developing an algorithm for serializing them will be difficult and error prone. I was thinking of using Reflection and some guesses on getter/setters but now I'm thinking of using Java's security model (basically to turn all instance variables into "public" ones for my serialization routine).

Answer 1

An update:
Actually Castor didn't look as "full featured" as I originally thought...
So for now I'm trying to write my own persistance manager. Its along the same lines as the article "Using Reflection to Automatically Map Objects to a Database" (which can be found at URL http://www.ajug.org/meetings/2001/DBMap.html ).
The cool thing is that I have permission to make my work open source (I think that will help my company and others). My project goals are:
1) create the persistance manager
-the manager can be easily extended for any backing DB (within reason); so not just RDB but LDAP.... This will enable the user to easily "plug & play" DB technologies.
-the manager will have a very simple (and limited) API similar to
serialization. Basically one will be able to save, delete, and
retrieve objects without needing predefined DB-Object mapping
files.
Likewise the user will be insulated from DB specific details.
I am going to add a "context" which is basically a level of
transactionalism (either the backing db supports it or you code it
into your db wrapper layer).
There will NOT be much in the way of query support.
2) hook into existing "bridge" technologies
-when I get done I want to see if I can get my auto-mapping & my
persistence manager... to "cooperate" with stuff like Castor,
Hibernate, OJB, TJDO.... This is a much lower priority but I see
it as strategic for the long term growth of the project (it will
enable users to transition from "dumb" auto-maps to "smart" custom
ones seemlessly
I'm going to open another thread about this on the forum.