Java library for large list sorts in small amount of memory

Question

Hi All
Wondering whether anybody would know of a ready made java library for sorting large lists of objects. We will write ourselves if necessary but I'm hoping this is the sort of problem that has been solved a million times and may hopefully have been wrapped behind some nice interface
What we need to do is something along the line of:
Load a large list with objects (duh!)
If that list fits into an allowed amount of memory great. Sort it and end of story.
If it won't then have some mechanism for spilling over onto disk and perform the best sort algorithm possible given some spill over onto disk.
These sorts need to be used in scenarios such as a random selection of 10% of the records in a 100G unsorted file and delivered sorted, with the jobs doing the sorts being limited to as little as 64MB of memory (these are extremes but ones that happen quite regularly).
Some of you might think "just use a database". Been there. Done that. And for reasons beyond the scope of this post its a no goer.
Thanking you for all your help in advance.

Answer 1

Of course this kind of sorting was common back in days of yore when we had mainframes with less than 1MB or ram.
The classic algorithm uses serveral files.
1) Load as much as will fit
2) sort it
3) Write it to a temporary file
4) Repeat from 1, altnerating output between two or three temp files.
Your temporary files then each contain a series of sorted sequences (try saying that three times fast).
You then merge these sequences into longer sequences, creating more temp files. Then merge the longer sequences into even longer ones, and so on until you're left with just the one sequence.
Merging doesn't require any significant memory use however long the sequences. You only need to hold one record from each temp file being merged.