How to use Java NIO to implement disk cache for serialized java objects

Question

Hi,
I have a cache (implemented as hahstable etc.) that contains java objects (mostly strings) and swaps objects from runtime memory to the disk and back based on some algorithms. Currently, the reading and writing from the disk is implemented using java.io.* package i.e. fileInputstream and FileOutputStream. Essentially, I serialize the java object and write to the disk and the deserialize and give it back to the Hashtable cache.
The performance of swapping from disk to memory is kinda slow. I have read that memory mapping would improve the performance.
My idea is to do the following:
Have one big file mapped to memory. I write the serialized objects to different portions of the file and then read those portions when needed. I can use the MappedByteBuffer for that but then I have the following questions. I will not store objects in the hashtable anymore.
1. How do I delete things from the cache in the above design i.e. how do I delete portions of a mapped file?
2. How do I serialize objects using ByteBuffers and then deserialize them? I guess this shouldn't be hard but just want to confirm.
Do you think this is the right design or should I change? Right now using the old io package, I have a separate file for each object. When using the NIO package, I want to store all objects in a single file in different portions of the file, is that the right way to go?
As you can see, I am beginner in memory mapped io and need help.

Answer 1

Have one big file mapped to memory. I write the serialized objects to different portions of the file and then read those portions when needed. I can use the MappedByteBufferThis is a good idea, one that I have worked on. It involves quite a bit of manipulation with temporary buffers and a deep working knowledge of object serialization.
1. How do I delete things from the cache in the above design i.e. how do I delete portions of a mapped file?The best way to handle this is do a two-step process, cutting the file into two pieces and gluing it back together where the original one is...
2. How do I serialize objects using ByteBuffers and then deserialize them? I guess this shouldn't be hard but just want to confirm.It is hard. Wrapping the streams and making the IO work properly is not the challenge however. The hard part comes in hacking the object streams. The object input/output streams use a ClassDescriptor object which only gets written once/ read once. This shouldn't be a problem if you will read/write the entire file at once, but will bring you grief if you want random access to your objects. You will also need an indexing mechanism to support random access.
Do you think this is the right design or should I change? Right now using the old io package, I have a separate file for each object. When using the NIO package, I want to store all objects in a single file in different portions of the file, is that the right way to go?I guess it depends on your needs. Do you require random access to objects? NIO provides some performance gains, but mostly for very large amounts of data (>10M in my experience).
You can always write all your objects into the same file using normal io techniques and you can still generate an index and acheive random access. It might be easier...
Good luck