Is there a memory limitation to byte arrays?

Everyone,
I have a huge(16MB) CMYK source file and I am trying to convert it to a sRGB /JPEG file and then compress it.
When I try to read that file into a byte array it chokes and throws an OutOfMemoryException. Is there a way around this?
thanks,
Barat.

The data looks something like this:
CCCCC...MMMMM...YYYYY...KKKKK...
CCCCC...MMMMM...YYYYY...KKKKK...
where each letter represents one byte.
For example, the first cyan byte has cyan information
for 8 pixels -- each bit represents cyan or no cyan, etc.
The point is to get this data in a viewable form on screen.
Have tried increasing the heap to 256
Of course the problem is in a large array that holds the output information, which is ~130M
The original file size is 16M but this was expanded to eight times the size for the PixelInterleavedSampleModel
where one data element (a byte in this case) represents one sample of a pixel
There must be a better way - any suggestions?

Similar Messages

How to save a value in a byte array, retrieve it and display it?

Hi,
I am doing a project for my data structures class that involves saving a value (given in String format) in a byte array (our 'memory'). Initially I just tried casting character by character into the byte array and casting back to char[] for retrieval (using .toString() to return what's supposed to be the original string), but this did not work. I tried the .getBytes() method, applying it to the string and then trying to recover it by placing the contents of the 'memory' in a byte array and applying toString(), but that didn't work either. I looked a bit and found this class, CharsetEncoder and CharsetDecoder. I tried to use these but when I try the compiler tells me I cannot instantiate CharsetDecoder because it is an abstract class. At this point I'm at a loss as to what I can do. Is there any way to place a string in a byte array and then recover the string in it's original format? For example, I might save a value in my particular class of "456". But when I try to recover the value from my 'memory' i.e. the byte array, it comes out like [gnue@hnju.... or something similar. I need it to output the original string ("456").
Below is my code as it is right now, for the methods setValue and getValue.
Thanks!
public void setValue(String value) throws InvalidValueException {
     // stores the given value in the memory area assigned to the variable
          if(this.type.isValidValue(value)){
               bytes = value.getBytes();
               int i,j,k;
               int l=0;//might be wrong?
               int ad=address-(address%4);
               mem.readWord(ad);
               reg=mem.getDataRegister();
               if((address%4)+bytes.length-1<4){
                    for(i=address%4;i<address%4+bytes.length;i++)
                         reg.setByte(i, bytes[i]);
                    mem.setDataRegister(reg);
                    mem.writeWord(ad);
               else if((address%4)+bytes.length-1>=4){
                    if(address%4!=0){
                         for(i=address%4;i<4;i++){
                              reg.setByte(i, bytes);
                              l++;
                         mem.setDataRegister(reg);
                         mem.writeWord(ad);
                         ad+=mem.WORDSIZE;
                    while(ad<address+bytes.length-(address+bytes.length)%4){
                         for(j=0;j<4;j++){
                              reg.setByte(j, bytes[j+l]);
                              l++;
                         mem.setDataRegister(reg);
                         mem.writeWord(ad);
                         ad+=mem.WORDSIZE;
                    if((address+bytes.length)%4!=0){
                         mem.readWord(ad);
                         reg=mem.getDataRegister();
                         for(k=0;k<(address+bytes.length)%4;k++){
                              reg.setByte(k, bytes[k+l]);
                              l++;
                         mem.setDataRegister(reg);
                         mem.writeWord(ad);
               else
                    throw new InvalidValueException("The value passed is not valid.");
     /** Gets the current value of the variable.
     @return current value converted to String
     public String getValue() {
          // returns the current value stored in the corresponding memory area
          // value is converted to String
          bytes=new byte[this.getType().getSize()];
          int i,j,k;
          int l=0;//might be wrong?
          int ad=address-(address%4);
          mem.readWord(ad);
          reg=mem.getDataRegister();
          if((address%4)+bytes.length-1<4){
               for(i=address%4;i<address%4+bytes.length;i++)
                    bytes[i] = reg.readByte(i);
          else if((address%4)+bytes.length-1>=4){
               if(address%4!=0){
                    for(i=address%4;i<4;i++){
                         bytes[i] = reg.readByte(i);
                         l++;
                    ad+=mem.WORDSIZE;
               mem.readWord(ad);
               reg=mem.getDataRegister();
               while(ad<address+bytes.length-(address+bytes.length)%4){
                    for(j=0;j<4;j++){
                         bytes[j+l] = reg.readByte(j);
                         l++;
                    ad+=mem.WORDSIZE;
               if((address+bytes.length)%4!=0){
                    mem.readWord(ad);
                    reg=mem.getDataRegister();
                    for(k=0;k<(address+bytes.length)%4;k++){
                         bytes[k+l] = reg.readByte(k);
                         l++;
          return bytes.toString();

You can certainly put it into a byte array and then construct a new String from that byte array. Just calling toString doesn't mean you'll automatically get a meaningful string out of it. Arrays do not override the toString method, so the use the one inherited from object.
Look at String's constructors.

Open report using memorystream or byte array

Dear All,
I am using .Net4.0 with CR13, is it possible to load a report document using a memory stream or byte array in stead from a physical file from disk?
If not, will this be made available?
regards

By default when you use the .Load() API CR creates a temp file in your users
%Users%\AppData\Local\Temp folder so CR requires access to that folder at least. And therefore it also requires Virus and other permissions to use it.
Bottom line is there is no Load or Open report from FileStream Method.
Add it to Idea Place, search first though, someone may have already requested it.
Don

Concatenate byte arrays

I have a Socket that is sending data accross a connection. I want to keep reading more data into an array. Is there a way to add more data into an array, or do I have to create a new one every time.
I'd like to do something like this:
byte[] b = new byte[10];
istrm.read( b2);
b += b2; <----- I know this doesn't work
I have been writing it like this:
InputStream istrm = sock.getInputStream();
byte[] b = new byte[ istrm.available()];
istrtrm.read( b);
but I don't know if that gets all of the data. If it blocks for a while, does that mean more data will come down the stream? I just want to make sure the data all gets there and put into 1 byte array.
Thanks in advance.

You cannot alter the size of an array object. You must allocate another array and copy the contents from the original. A better way would probably be to have a list of arrays, keeping track of the total received. THen create a single array and copy the data from the smaller arrays. You could have this happen whenever the list length exceeds a chosen boundary to avoid having too many small array objects referenced in memory.
eg
byte[] b1 = new byte[100];
byte[] b2 = new byte[50];
List list = new LinkedList();
int totalSize = 0;
list.add(b1);
totalSize += b1.length();
list.add(b2);
totalSize += b2.length();
byte[] b = new byte[totalSize];
int currentPos = 0;
for(Iterator iterator = list.iterator(); iterator.hasNext(); )
byte[] temp = (byte[]) iterator.next();
for(int count = 0; count < temp.length; count++)
b[currentPos++] = temp[count];
or something like that... (untested)
Hope that helps...
Talden

Changing a byte in a byte array???? HELP

Hey I have the following code
     String plainString = new String();
     byte[] byt = new byte[0];
     byte change;
     try
          int length = (int)(keyf.length());
          byt = new byte[(length++)];
          keyf.readFully(byt);
          keyf.seek(0);
     }catch(IOException e){System.err.println(e);}And I want to edit the byt array at position (length--) and add the character "," there. Also changing the byt array at its current position to eof. Anyone know what to do???

Make a new array with a size of length+1.byt = new byte[(length++)];
keyf.readFully(byt);
byte[] b2 = new byte[byt.length+1];
System.arraycopy(byt, 0, b2, 0, byt.length);
b2[b2.length - 1] = (byte)',';

Is there a null value that I can put into a byte array?

I have a byte[] that I'm trying to make smaller, at the moment, in order to do so, i'm writing it byte-by-byte to another byte[] called temp. Both are set to the same size, because I don't know exactly what the initial array will compress to.
For example, my method will write a single byte that will tell the decompressor to carry out the next instruction 5 times (eg aaaaa = 5a), but after the instruction, I want to set the 4 a's afterwards to an empty value so that I can then iterate through temp, finding out how long to make the output byte[] by counting how many null's there are.
Eclipse is telling me null is not possible to use, I was just wondering if there is an equivalent I can use?

That's an idea!
The only thing is then when I come to iterate through the byte array to write it to my output array, it throws up an error that I'm trying to compare a byte to a byte[]:
          int next = 0;
          byte[] n = new byte[0];
          for (int i = 0; i < temp.length; i++) {
               if (temp[i] != n) {
                    output[next] = temp;
                    next++;
          return output;

Byte array in argument ? There's no ObjectFactory with an @XmlElementDecl

Hello everyone,
I'm trying to upload a file through my webservice but when i set up the call to the webservice from my client using
webservices.ChansonTestService service = new webservices.ChansonTestService();
            webservices.ChansonTest port = service.getChansonTestPort();I get the following exception :
Caused by: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
There's no ObjectFactory with an @XmlElementDecl for the element {http://webservices/}fichier.
        this problem is related to the following location:
                at protected javax.xml.bind.JAXBElement webservices.PutFichier.fichier
                at webservices.PutFichierThis is my webservice method :
@WebMethod(operationName = "putFichier")
    @Oneway
    public void putFichier(@WebParam(name = "fichier")
    byte[] fichier) {
        try {
       File mp3 = new File("/home/schulman/Desktop/dl01.mp3");
       FileOutputStream fo = new FileOutputStream(mp3);
       fo.write(fichier);
       catch (IOException ioe) {
           ioe.printStackTrace();
    }Do i have to set up something else to be able to use byte array as argument ? i thought it was mapped using base64 by default...
I'm using glassfish v2, and i'm open to any suggestion !!
Regards
Edited by: Schulman on Nov 15, 2007 2:38 AM

hi,
first generate the classes by using the jdk 1.6 wsimport command
wsimport -keep <directory> -s <directory> wsdl url.

Is there such a thing as a byte array

I have an array of chars in my c code which I need to pass an array to from my Java code. I was just wondering what the equivalent is in Java is there such a thing as a byte[]

You have declared param as an int and therefore the compiler notice that the assignment to that byte array may result in loss of some bits. To prevent the warning you need to do an explicit cast to byte:
m_writeBuffer[0] = (byte)(param>>8);
m_writeBuffer[1] = (byte)(param & 0x00FF);Not that you would have to do the same even if param was declared as byte because java performs all operation on bytes as int.

Return byte array

Hello i have a function that return a byte array. I did it this way but i m wondering if it is the good way, so if someone can help improve it. Thank you.
try
            InputStream is = new BufferedInputStream(new FileInputStream(nomFichierDemande));
            byte[] readData = null;
            while (is.available() > 0)
                readData = new byte[is.available()];
                is.read(readData);
return readData;
        catch (IOException e)
            System.out.println("probleme lors de lecture de l'image");
            e.printStackTrace();
        }the source for my stream is an image file, bmp, gif or png, will it work the same for all kinds of file ?
Thanks for help.
Message was edited by:
JusteUneQuestion
Message was edited by:
JusteUneQuestion

(a) You should use File.length() rather than
InputStream.available(): that's not what it's for.Hello i change the code like this :
InputStream is = new BufferedInputStream(new FileInputStream(imageFile));
                byte[] readData = null;
                readData = new byte[(int)imageFile.length()];
                is.read(readData);
                return readData;Is it like this that u mean i have to use File.length, or was it only for the while loop ? I dont think that i need this while loop finally.
(b) You should reconsider the whole idea of loading
the entire file into memory. It doesn't scale. Sooner
or later you'll encounter a file that doesn't fit
into memory - then what?Can you give me a better idea? i thought about this but dont really know how to do to improve it. There is a limitation in the size of image anyway but i d like to have a good code in case for later.
With this code i could read gif file but i m having problem for bmp so maybe its because it doesnt scale like you said.
Message was edited by:
JusteUneQuestion

Best way to append null bytes to a byte array?

I have a fixed length byte array and need to add two null bytes to its end. I have an implementation that I wrote to do this, though I was wondering whether someone could improve on what I have:
public byte[] appendNullBytes( byte[] bytes ) {
   byte[] copyBytes = new byte[bytes.length+2];
   System.arraycopy(bytes, 0, copyBytes, 0, bytes.length);
   copyBytes[copyBytes.length-1] = 0;
   copyBytes[copyBytes.length-2] = 0;
   return copyBytes
}

ajmasx wrote:
jverd wrote:
ajmasx wrote:
jverd wrote:
Those aren't null bytes, they're zero bytes. A bytes can never have the value null in Java.I was using the term null interchangeably with 'zero', Which is not generally applicable in Java.Well it all depends what you are doing:
- A null is a zero value.Not in general in Java. Null is null and zero is zero.
- char c = '\0' defines a null character.Yes, that's the only use of "null" for primitive types.
- A null pointer is where the address is zero and not pointing to some location in memory.Nope. A null pointer is simply one that doesn't point to any value. Nothing in the spec says it's zero.
- A null reference is more or less the same thing (there are certain difference in the details).Nope. A null pointer and null reference in Java are the same thing.
- A null terminated string is one where null defines the end of the string.This does not exist in Java. As the above smartass pointed out, you can put the null character at the end of a String, but that character is part of the String, not a terminator.
>
If you are dealing with files and other types of I/O then this concept most certainly exists. Not in Java.
Just because you are limiting yourself to high-level use of Java, does not mean they aren't present in Java.The fact that null is precisely defined in the JLS and does not include these things means they aren't present in Java.

How Do I Load An Animated GIF Into PictureBox Control From Byte Array?

I'm having a problem with loading an animated GIF int a picturebox control in a C# program after it has been converted to a base64 string, then converted to a byte array, then written to binary, then converted to a base64 string again, then converted to
a byte array, then loaded into a memory stream and finally into a picturebox control.
Here's the step-by-step code I've written:
1. First I open an animated GIF from a file and load it directly into a picturebox control. It animates just fine.
2. Next I convert the image in the picturebox control (pbTitlePageImage) to a base64 string as shown in the code below:
                if (pbTitlePageImage.Image != null)
                    string Image2BConverted;
                    using (Bitmap bm = new Bitmap(pbTitlePageImage.Image))
                        using (MemoryStream ms = new MemoryStream())
                            bm.Save(ms, ImageFormat.Jpeg);
                            Image2BConverted = Convert.ToBase64String(ms.ToArray());
                            GameInfo.TitlePageImage = Image2BConverted;
                            ms.Close();
                            GameInfo.TitlePageImagePresent = true;
                            ProjectNeedsSaving = true;
3. Then I write the base64 string to a binary file using FileStream and BinaryWriter.
4. Next I get the image from the binary file using FileStream and BinaryReader and assign it to a string variable. It is now a base64 string again.
5. Next I load the base64 string into a byte array, then I load it into StreamReader and finally into the picturebox control (pbGameImages) as shown in the code below:
byte[] TitlePageImageBuffer = Convert.FromBase64String(GameInfo.TitlePageImage);
                        MemoryStream memTitlePageImageStream = new MemoryStream(TitlePageImageBuffer, 0, TitlePageImageBuffer.Length);
                        memTitlePageImageStream.Write(TitlePageImageBuffer, 0, TitlePageImageBuffer.Length);
                        memTitlePageImageStream.Position = 0;
                        pbGameImages.Image = Image.FromStream(memTitlePageImageStream, true);
                        memTitlePageImageStream.Close();
                        memTitlePageImageStream = null;
                        TitlePageImageBuffer = null;
This step-by-step will work with all image file types except animated GIFs (standard GIFs work fine). It looks like it's just taking one frame from the animation and loading it into the picturebox. I need to be able to load the entire animation. Does any of
the code above cause the animation to be lost? Any ideas?

There is an ImageAnimator so you may not need to use byte array instead.
ImageAnimator.Animate Method
http://msdn.microsoft.com/en-us/library/system.drawing.imageanimator.animate(v=vs.110).aspx
chanmm
chanmm

How can i convert object to byte array very*100 fast?

i need to transfer a object by datagram packet in embeded system.
i make a code fallowing sequence.
1) convert object to byte array ( i append object attribute to byte[] sequencailly )
2) send the byte array by datagram packet ( by JNI )
but, it's not satisfied my requirement.
it must be finished in 1ms.
but, converting is spending 2ms.
network speed is not bottleneck. ( transfer time is 0.3ms and packet size is 4096 bytes )
Using ObjectOutputStream is very slow, so i'm using this way.
is there antoher way? or how can i improve?
Edited by: JongpilKim on May 17, 2009 10:48 PM
Edited by: JongpilKim on May 17, 2009 10:51 PM
Edited by: JongpilKim on May 17, 2009 10:53 PM

thanks a lot for your reply.
now, i use udp socket for communication, but, i must use hardware pci communication later.
so, i wrap the communication logic to use jni.
for convert a object to byte array,
i used ObjectInputStream before, but it was so slow.
so, i change the implementation to use byte array directly, like ByteBuffer.
ex)
public class ByteArrayHelper {
private byte[] buf = new byte[1024];
int idx = 0;
public void putInt(int val){
buf[idx++] = (byte)(val & 0xff);
buf[idx++] = (byte)((val>>8) & 0xff);
buf[idx++] = (byte)((val>>16) & 0xff);
buf[idx++] = (byte)((val>>24) & 0xff);
public void putDouble(double val){ .... }
public void putFloat(float val){ ... }
public byte[] toByteArray(){ return this.buf; }
public class PacketData {
priavte int a;
private int b;
public byte[] getByteArray(){
ByteArrayHelper helper = new ByteArrayHelper();
helper.putInt(a);
helper.putInt(b);
return helper.toByteArray();
but, it's not enough.
is there another way to send a object data?
in java language, i can't access memory directly.
in c language, if i use struct, i can send struct data to copy memory by socket and it's very fast.
Edited by: JongpilKim on May 18, 2009 5:26 PM

Maximum size of a byte array?

Hi,
I'm trying to read in a file into a byte array as following:
byte[]inputformat = new byte[1280];
FileInputStream in= new FileInputStream(file);
BufferedInputStream buffer= new BufferedInputStream(in);
int len = buffer.read(inputformat);
The problem appears when I try to access the 1024th array-element (inputformat[1024]) afterwards. Is there a limitation in the byte-arrays, or prehaps in the buffer.read-method?
All comments are welcomed!
Cheers,
Slafs

I executed the following code:
import java.io.*;
public class BufferTest {
    public static void main(String[] args) {
        try {
            byte[] inputFormat = new byte[1280];
            File inFile = new File(args[0]);
            FileInputStream in = new FileInputStream(inFile);
            BufferedInputStream bis = new BufferedInputStream(in);
            int len = bis.read(inputFormat);
            System.out.println("Read " + len + " bytes");
        catch (IOException e) {
            System.out.println("Error: " + e.toString());
}To produce the following output:
$ java BufferTest snapshot1.png
Read 1280 bytesSo, I am not sure where your problem lies, but I don't think it's in the code you posted.
Good luck
Lee

This byte array is not garbage collected?

imageBytes[] does not seem to ever be garbage collected. I've pinpointed a SINGLE LINE that, when commented out, will prevent the memory leak. What could possibly be wrong with this line?
Running this code results in a "OutOfMemoryError: Java heap space" after roughly a minute on my machine. I'm using JRE version 1.5.0_08.
import javax.imageio.ImageIO;
import java.awt.*;
import java.awt.image.*;
import java.io.*;
// creates images to send to ImageConsumer
public class ImageProducer {
     static BufferedImage bufferedImage;
     static Robot robo;
     static byte[] imageBytes;
     public static void main(String args[]) {
          try {
               robo = new Robot();
          } catch(Exception e) {
               e.printStackTrace();
          PipedOutputStream pos = new PipedOutputStream();
          new ImageConsumer(pos).start();
          bufferedImage = robo.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
          try {
               ObjectOutputStream oostream = new ObjectOutputStream(pos);
               ByteArrayOutputStream baos = new ByteArrayOutputStream();
               ImageIO.write(bufferedImage, "JPG", baos);
               baos.flush();
               imageBytes = baos.toByteArray();
               while (true) {
                    ImageIO.write(bufferedImage, "JPG", baos);
                    baos.flush();
                    // THIS SEEMS TO BE WHERE THE MEMORY LEAK OCCURS: imageBytes
                    // If you comment the following line out, there will be no
                    // memory leak. Why? I ask that you help me solve this.
                    imageBytes = baos.toByteArray();
                    baos.reset();
                    oostream.writeObject(imageBytes);
                    pos.flush();
          } catch (Exception e) {
               e.printStackTrace();
// This thread reads the image data into bImg
class ImageConsumer extends Thread {
     BufferedImage bImg;
     PipedInputStream pis;
     ImageConsumer(PipedOutputStream pos) {
          try {
               pis = new PipedInputStream(pos);
          } catch (IOException e) { e.printStackTrace();}
     public void run() {
          try {
               ObjectInputStream oinstream = new ObjectInputStream(pis);
               while (true) {
                    byte[] imageBytes = (byte[])oinstream.readObject();
                    ByteArrayInputStream bais = new ByteArrayInputStream(imageBytes);
                    bImg = ImageIO.read(bais);
          } catch (Exception e) {e.printStackTrace();}
}

while (true) {
     ImageIO.write(bufferedImage, "JPG", baos);
     baos.flush();
     // THIS SEEMS TO BE WHERE THE MEMORY LEAK OCCURS: imageBytes
     // If you comment the following line out, there will be no
     // memory leak. Why? I ask that you help me solve this.
     imageBytes = baos.toByteArray();
     baos.reset();
     oostream.writeObject(imageBytes);
     pos.flush();
}I have only briefly gone through the code, but why are you flushing it right before calling that line. Won't the byte array returned always be empty right after flushing the stream?

Bit permutation inside a byte array

Hi everyone,
I'm trying to implement DES algorithm, i want to permute specified bits inside an 8byte array.
I tought about extract every bits from the byte array inside a 64 byte array and then do the permutation.
But i don't see how to put these bits back into an 8byte array to return the result of encryption/decryption.
This is what i've done so far (inspired by sources i found on the internet) :
// extends 8 byte array into a 64 byte array
public static byte[] extendsByteArray(byte[] bytes) {
byte[] tab = bytes;
short len = (short)(desBlockSize * (short)8);
byte[] bits = new byte[len];
short i,j;
for (i = (short)(desBlockSize - (short)1); i >=0; i--) {
for (j = (short)0; j < desBlockSize; j++)
bits[--len] = (byte) ((tab[i] >> j) & 0x01);
return bits;
// 64 byte array to 8 byte array
public static byte[] transformByteArray(byte[] bits){
byte[] tab = bits;
byte[] bytes = new byte[desBlockSize];
short i,j = (short)0;
for ( i = (short)0; i < (short)8; i++){
bytes[i] = (byte)((bits[j] * (short)2^7) + (bits[j+(short)1] * (short)2^6)+
               (bits[j+(short)2] * (short)2^5) + (bits[j+(short)3] * (short)2^4)
               + (bits[j+(short)4] * (short)2^3) + (bits[j+(short)5] * (short)2^2)
               + (bits[j+(short)6] * (short)2) + (bits[j+(short)7]));
j +=(short)7;
return bits;
This implementation doesn't seems to work, do u have some ideas why it doesn't work or is there an easier solution to permute bits as i'm stuck with this fo the moment.
Thanks in advance for your answers,
Julien

If you are using JavaCard please do not use multiplications as you used in your code. Integer multiplications are very slow in any platform that does not implement them in hardware, and the virtual machine that runs in JavaCards usually does not do any optimizations or Just-In-Time compiling - simply interprets the codes. Memory and CPU power are at a premium in Java Cards. Even if your Java Card has a hardware multiplier, usually it is reserved for RSA operations - it is not available for general use for Java programs.
Instead of using x * 2 raised to the sixth power, use x << 6. It is faster and generates fewer bytecodes.

Is there a memory limitation to byte arrays?

Similar Messages

Maybe you are looking for