GZIP compressing input stream?

Question

I have an API which takes an InputStream for feeding in large binary data.
Suppose I want to write a wrapper around that API which GZIPs the data on the fly. This would require some kind of InputStream which will compress the data with GZIP on the fly. However, GZIPInputStream can only decompress on the fly.
Does anyone know of a way to do the exact opposite?

Answer 1

So, don't store the entire file in memory. Storing a compressed version of the file isn't much of a
solution.You're not paying attention. I said I can't FIT the data in memory. If I can't fit it in memory, why would I be trying to store it in memory? Hence why I want to store it in a DATABASE.
If you can't grasp the most simple of uses for a database, there isn't a lot of hope for you in this field in the future.
You actually can use a ByteArrayOutputStream, to generate the compressed bytes, as I described, > but you apparently we're paying attention, which at this point is all I expect from you.Okay, how? If I simply put all the uncompressed bytes into the GZIPOutputStream and that output stream is attached to the ByteArrayOutputStream, I will run out of memory.
Code, please. In other words, prove that what you say is possible.
You are allowed to. It's debatable whether it's a good idea. Anyway this is the first time you
mentioned JDBC. Back when we wanted to help you (before you called us idiots) that might
have been useful information. It's easier to help people when you have information about what
the hell theyr'e trying to do.I did say that we have an API which takes an input stream, and that this was the reason why I had to pass in an input stream. It doesn't matter whether it's JDBC, or anything else. What is important is that it takes an input stream.
Actually, it's pretty cheap. It's probably cheaper than the cost of the extra RAM you'll need doing
extraneous compression/decompression.Actually, compression is generally fairly inexpensive, RAM-wise. You lose some CPU, but CPU is even cheaper than the amounts of disk space which we would otherwise need for this.
We're storing gigabytes of data, and
I would rather not have to actually store gigabytes
of data.So, you're not actually storing it now?We're storing it compressed, which makes it take less space. You do know what compression is, right?
A redesign might end up removing GZIP entirely. But maybe not. It's hard to say because you still
haven't given a lot of details, and given your attitude I'm certain your estimation of the project is
entirely wrong anyway. At this point I don't care either way. Good luck with your project to create
an unmaintainable buggy monstrosity.Actually, this is trying to make the project more maintainable. Right now, we have multiple components each doing the gzipping in their own sections. I'm trying to make a component which can generically take a data stream, and store it into the database compressed.
You're partially right, though. A redesign would remove GZIP entirely -- I would use Deflate as-is. Deflate has convenient methods for working with smaller chunks of data, and I could borrow much of the code from InflateInputStream and modify it to create DeflateInputStream. The next version of our app, once we're willing to break compatibility, will probably do this.