Read file with size 2Go : Pb

I'm using a loop with Read Vi to seek in the file. It works well, But When my file is greater than 2 GB, I got a memory error (mémoire insuffisante pour terminer l'opération) when I try to read for example 263000 singles data.
This error appears when I want to read my 263000 values (not when seeking).
I don't understand why, because it runs well with a file less than 2 Gb even if I want to read more values.
Please find attached the example vi. Choose a file and run it : if your file size is less than 2 Gb (but have enough datas (ex : file size 10 Mb), it's ok, but if you run it with a file size more than 2 Gb you have the memory error.
Help please ...
Eddy DUCHENE
12 F Chemin de Boutary
69300 CALUIRE ET CUIRE
[email protected]
Attachments:
SeekBigFile_tst.vi ‏61 KB

Hi Eddy,
There are all kinds of fun issues with files greater than 2 GB. The reason is that files greater than 2 GB can't be directly indexed by using a 32 bit integer (the largest integer type LabVIEW currently has). There are a number of solutions. If you are using an NTFS file system, you should be able to continue reading and writing sequentially as long as you don't wire anything up to the position inputs, however, this usually isn't really practical, since it gives you no random access, and accessing 2 GB of data sequentially is going to be kind of tiring.
Anyhow, the best solution to your problem would be to use a 64-bit file library. You can install one of these, the HWS library, from most newer versions of the Driver CD which ships with almost all NI products. For a detailed description of the issue and the possible solutions, check out this tutorial:
http://zone.ni.com/devzone/conceptd.nsf/webmain/6A56C174EABA7BBD86256E58005D9712
Regards,
Ryan K.

Similar Messages

Outofmemory while creating a new object of file with size more than 100 MB

I have created an application which generates a report by getting the data from our archived files (.zip file).By the time, the application is reaching a file with size more than 100 mb, it is running out fo memory while creating the object of that particular file. Can some one help me by tellin if there id way to resolve this issue?
Thanks in advance

If you're getting OutOfMemoryError, the simplest thing to try is to give the VM more memory at startup. For Sun's VM, I believe the default is 64 MB. You can increase this by using the -X args that control the heap size. For example: java -Xms128m -Xmx256m ... etc. ... This says start with 128 MB of heap, and allow it to grow up to 256 MB.
One thing to consider, though, is do you need that much stuff in memory at once? A more intelligent approach might be to unpack the archive to the file system, and then read a file at a time or a line at a time or a whatever at a time is appropriate for the processing you need to do.

Couldn't Upload file with size more than 2K

Hi,
i am developing an webapplication where in i have to upload a file.we are using hibernate for data storing and retrieval.problem comes when i am uploading a file with size more than 2k bytes.if the file size lesser than 2k it works fine.i found that its a bug with thin jdbc driver version 9.but i am using 10g jdbc driver.still problem exists. i am not suppose to use oci drivers.
working on
OS: windows Xp.
Apps :weblogic8.1
DB: oracle 9i
if anyone has solution plz mailme at [email protected]

I'm not sure where the issue would be. Are you saying that you are using a 9i driver to access a 10g database? If so, download the newer driver and add it to your WEB-INF/lib directory. Remove the existing driver.
If, on the other hand, you are using a 10g driver for a 9i database, I would not expect problems. However, you could always download the older driver and try it out.
Or am I missing something?
- Saish

The md5-sparc cannot open a file with size 2GB

Hi,
I would need to check the integrity of the file with size > 2GB with md5-sparc but I still get the error message "error opening file". Is there an update of this program which can open the large files, too? I have tried to search on web but without success.
Thanks a lot Milan

digest -a md5 seems to work:
bash-3.00$ ls -l sol-10-u3-ga-sparc-dvd.iso
-rw-r--r--   1 root     root     4207411200 Apr 6 2007 sol-10-u3-ga-sparc-dvd.iso
bash-3.00$ digest -a md5 sol-10-u3-ga-sparc-dvd.iso
6d046d5d95ac1cce58e742abcdd35238
bash-3.00$ cat /etc/release
                       Solaris 10 11/06 s10s_u3wos_10 SPARC
           Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
                        Use is subject to license terms.
                           Assembled 14 November 2006

Action with new file with size from selection

Hello,
I try to record an action that takes a copy of the current image and creates a new file with the same dimensions. The action I record always has the file size of the new image hardcoded in it. I tried to set Preset to Clipboard, but this is not recorded in the action. This is with CS3.
How can I create a new file that takes the current image size into account?
Thanks for your help,
Daniel

You can record the menu 'Image-Duplicate' to make a copy of the whole image without recording the size. If the doc only has one layer you can also record the flyout menu of the layer's panel to duplicate the layer to a new document. The size will not be recorded.

Read file with nio and flush with servlet

Can I read file, by using java.nio (for example by FileInputStream.getChannel()) and then flush the file content thru the servlet?
I kwow about reading without java.nio, get a byte array and then flush it by httpservletresponse writer or outputstream, but I wish to know if it is possibile with java.nio reading.
thanks

I'm doing it only for file reading..
protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
        response.setContentType("text/html;charset=UTF-8");
        PrintWriter out = response.getWriter();
        FileInputStream fis = null;
        try
            String path = "/var/hello.txt";
            fis = new FileInputStream(path);
            FileChannel channel = fis.getChannel();
            ByteBuffer bb = ByteBuffer.allocate((int) channel.size());
            channel.read(bb);
            byte[] data2 = bb.array();
            channel.close();
            out.write(new String(data2));
        } catch (FileNotFoundException ex)
            ex.printStackTrace();
        } finally
            try
                fis.close();
            } catch (IOException ex)
ex.printStackTrace();
            out.close();
    }

Read files with no app suffixes (.pdf, .doc. xls, etc ...)

Hi,
On a Macintosh, files do not have suffixes (.pdf, .xls or .doc, etc ...) to identify the application that created the file. How can I open a file on my iPhone if there is no suffixes ?
Thanks in advance.

You're welcome.
The files on your Mac include the file extension, which you can choose to show or not show with the files on your Mac.
Since I don't email files to Mac users only, I'm in the habit of including the file extension with all email attachments.
I'm not aware of a way to force open a file with the iPhone.

How to read files with codepage UTF16LE with "Sender File Adapter"

Hi everybody,
I am using a XI-Filesender-Adapter to get a (UTF-16LE encoded) file und process it in XI-Mapping.
This is my File-Content:
Cost Centre,Cost Code,Page Count (B&W),Page Count (Colour),Job Count
Unknown,Lexmark,"37,480",334,"11,968"
Unknown,Unknown,312,0,177
110000,Lexmark,128,228,43
The HEX-representation of this content is:
FF FE 43 00 6F 00 73 00 74 00 20 00 43 00 65 00
(The starting 2 Bytes FF FE represent UTF16-LE )
And this is the payload I get for mapping:
<?xml version="1.0" encoding="utf-8" ?>
<ns:MT_POM_KOSTEN xmlns:ns="http://aua.com/pom">
<POM_REC>
<COSTCENTER>uFEFFCost Centre</COSTCENTER>
<COSTCODE>Cost Code</COSTCODE>
<PAGECOUNT_BW>Page Count (B&W)</PAGECOUNT_BW>
<PAGECOUNT_COL>Page Count (Colour)</PAGECOUNT_COL>
<JOBCOUNT>Job Count</JOBCOUNT>
</POM_REC>
<POM_REC>
<COSTCENTER>Unknown</COSTCENTER>
<COSTCODE>Lexmark</COSTCODE>
<PAGECOUNT_BW>37,480</PAGECOUNT_BW>
<PAGECOUNT_COL>334</PAGECOUNT_COL>
<JOBCOUNT>11,968</JOBCOUNT>
</POM_REC>
<POM_REC>
<COSTCENTER>Unknown</COSTCENTER>
<COSTCODE>Unknown</COSTCODE>
<PAGECOUNT_BW>312</PAGECOUNT_BW>
<PAGECOUNT_COL>0</PAGECOUNT_COL>
<JOBCOUNT>177</JOBCOUNT>
</POM_REC>
<POM_REC>
<COSTCENTER>110000</COSTCENTER>
<COSTCODE>Lexmark</COSTCODE>
<PAGECOUNT_BW>128</PAGECOUNT_BW>
<PAGECOUNT_COL>228</PAGECOUNT_COL>
<JOBCOUNT>43</JOBCOUNT>
<POM_REC>
I can see the correct strings (for example Cost Centre) in payload, but the string-comparison in the user defines function cannot recognize the equality of the strings:
for (int i =0; i < a.length; i++) {
if (
(a<i>.equals("Cost Centre")) )
result.addSuppress();
else
result.addValue("");
Actually I am using UTF-8 as codepage in Fileadapter (and Text as type)
When I try to use UTF16 (or UTF-16LE) as Codepage, I am getting unreadable characters.
I also tried binary, UTF16-BE,...
The only way is to covert the file to ANSI before I use it with XI. Then my function does work correctly.
Does anybody have an idea, how I can read a UTF16-LE File and process it correctly in XI?
I am using XI 7.00 0023 and JSDK 1.4.2-34
Thanks a lot
Armin

Hello Armin,
I have gone through some SAP notes and blogs to find solution to your problem, here is what I found
1. SAP NOTE 821267
q) How do I correctly configure the File Encoding used by the File
Adapter?
Flat Files with File Content Conversion
For a File Sender channel, configure the encoding of the source
file. The file will be interpreted according to the configured
encoding and converted to XML with an UTF-8 encoding.
For a File Receiver channel, configure the encoding to match
the encoding you would like to be written to the target flat
file.
- Flat Files without File Content Conversion
Whether to configure an encoding in this case depends on if you
want to pass through the file "as is", e.g. within a File
Sender to File Receiver scenario, or if you want to convert the
file's encoding on its way through the Integration Server. For
"as is" processing, configure both the sender and the receiver
using the File Type setting "Binary".
To apply an encoding conversion, configure the respective
source and target encoding in both the sender and receiver
channel.
Important: Configuring an encoding in the receiver channel
will only lead to the expected results if the payload sent to
the receiver channel is in UTF-8 format (e.g., by having
specified an encoding conversion in the Sender channel).
So as per this note if you configure the encoding scheme of sender communication channel to UTF-16LE, adapter should be able to convert it to UTF-8 by default. But you have posted that this encoding scheme is leading to unreadable charcters
2) SAP note 880173
This speaks of use of module XMLAnonymizerBean which can be applied to XML payload to change its encoding.
3) How to guide on encoding : http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79fb42?QuickLink=index&overridelayout=true
4) SAP note:960663
http://help.sap.com/saphelp_nw04/helpdata/en/45/da2deb47812e98e10000000a155369/content.htm
TextCodePageConversion Bean details which might solve your problem.
5) Finally if nothing above works you need a java mapping code to convert to target XML structure without any File content conversion. The mapping will convert the received file to proper target XML in "UTF-8" encoding. Please let us know if you need help on the code with this final option.
Regards
Anupam

Specifying byte stream type for Read File with Type Descriptor

Hi.
I'm trying to write a VI that reads an image file format that can have
different datatypes. What I have so far is that I open the file, read
the header, and get the width, height, number of frames, and datatype.
I calculate number of pixels by nrows*ncols*nframes with no problem,
but I'm not sure how to tell Read File the correct datatype to read
the data into.
I tried using a Case structure where I have a Read File in each case
with the correct type constant as input for that case. The problem is
that the tunnel graduates the datatype to the highest representation.
I found in Application Note 154 the discussion about Type Descriptors.
Is there a way to generate a Type Descriptor and output it from a C
ase
structure? I tried just returning the value (e.g. 0x0402 for a Word),
but Read File will just see that the byte stream type is a uint32.
Is there any other way to do this?
Thanks for any help.

I converted the code to LabVIEW 6.1 for you and attached it below.
Don't worry about being a newbie. We all start there. Keep asking this type of question and you won't stay there long.
As you are discovering, being strictly typed means that you must rewrite code even for a simple data type change, or convert everything to the same data type first. For image data, conversion can result in a lot of extra space being wasted. Use a modified version of the GLV_WaveformBuffer.vi to hold your data. Use the array functions, which operate inline, to add to and delete the data wires in the buffer. This allows you to save several different data types. You will need several different inputs and outputs to handle these data types. I ha
ve also attached a similar file created for exactly the problem you have - storing arrays of different data types (data from NI-SCOPE devices, in this case - can be float, I8, I16, or I32).
Routines that take any type work in one of two ways. LabVIEW primitives, such as plus and minus operators, work by figuring out the type and doing the right thing in the C code layer of the LabVIEW environment. Users of LabVIEW can't do this. Users can make polymorphic VIs. Polymorphic VIs are actually a single VI for every data type that are referenced by a "wrapper", the polymorphic VI. Users still need to write a different VI for every data type they need.
Take home message - if you need to work with different data types, you will need to rewrite your code for every data type or convert your data to a common data type. Polymorphic VIs and case statements are your friend.
Let me know if you need more help.
This account is no longer active. Contact ShadesOfGray for current posts and information.
Attachments:
GigaLabVIEW61.zip ‏362 KB
sfpScpChan_Waveform_Buffer.zip ‏74 KB

Reading file with BufferedReader

Hi i have a problem iam trying to make a linked lists, where i read a file and
each word in that file is putt into the list.
but the problem is when i use a while loop and the eadLine() method.
like this
<method >
String word;
try .....
while((word = file.readLine() ) ! =null){
word = file.readLine();
The problem is that readLine() only reads the first word in the file and doesnt make the rest of the list
any suggestions??
here is the rest of the code:
import java.io.*;
class Lister
public static void main(String []args) throws Exception {
          Listene l = new Listene();
          l.initiate(args);
class Listene {
private BufferedReader file;
Listerr list = new Listerr();
public void initiate(String args[]) throws IOException
     String filename =args[0];
     String word;
     try
          file = new BufferedReader(new FileReader(filename));
          while( ( word = file.readLine( ) ) != null ){
          word = file.readLine();
          //System.out.println(word);
          list.insert(ord);
          shutdown();
     catch (FileNotFoundException e)
          System.out.println("Minila-filen kan ikke �pnes: " + filename);
public void shutdown()
     if (file != null)
          try
               file.close();
          catch (Exception e)
               System.out.println("F�r ikke lukket Minila-filen") ;
class Listerr{
Node liste =null;
Node next;
void insert(Object element){
     if(liste==null){
     liste=new Node(element);
     System.out.println("setter inn element: "+element);
     else{
     Node n = liste;
     while(n.neste!=null){
          n=n.neste;
     n.neste=new Node(element);
     System.out.println("setter inn element: "+n.element);
class Node{
     Object element;
     Node neste;
     public Node(Object element2){
     element=element2;
     neste=null;
}

Hi, I assume you are trying to read WORDs seperated by newline. i.e., each WORD occuring in a newline in the input-file. If that is the case, the code you have written should work perfectly. One reason why it may fail is when the newline character inserted by the input-file editor is not the one as listed by System.getProperty("line.separator");
One small change to your code to avoid redundant calls to BufferFileReader::readline() can be done as follows...
String word = null;
try
while( (word = file.readLine()) != null )
list.inert(word);
catch (IOException ex)
//System.out.println("Could not read file: " + filename + ex.getMessage());
}

Printing Adobe Reader files with a DeskJet 6540 and OS X.6

Ever since I upgraded to Mac OS X.6, I've had problems printing .pdf files. No matter what I teil the printer, it only prints one copy of one page of the document. I've upgraded to Adobe Reader 9.2.0. Is this a driver problem? OS X.6 is supposed to take care of that. Adobe Reader is the only software that causes this problem, and of course Adobe does not provide support. Any ideas?

Hi There...I'm having the same problem, although I'm using XP Pro with Service Pack 3 and an OfficeJet 6500 , e709n ...and
it was working until this past couple of weeks. I can't print ANY PDF, doesn't matter if it's old or new... If I "do" a preview, all I see is a line of print across the top of the page.
Anything else will print other than PDF's, so of course it's driving me nuts! I've even reinstalled the driver for the printer; but it didn't make any difference except for my time and frustration. I hope someone can give us an answer soon...there are important docs I need to print out.

How to read file with JNI?

Hello,
I want to implement a native function that has as a parameter a File type. For example:
public native void readFile(File f);
But as far as I know the JNI does not support File type, or I am wrong? One way to solv this problem i of course to convert the file into an array of chars end then to implement some method like:
public native void readFile(char [] f);
So if there is another way of solving this problem, please help me.
Thank you in advance.:)

I would recommend against passing a File object as a paramater. The implication of trying to do that is that you will be using the File object, which means you will be making JNI calls back into the JVM in order to do what you need to do. If there are very many of those calls, well, no fun!
If you do it by hand, then yes, it can be very difficult and time-consuming. Jace, http://jace.reyelts.com/jace, (a free, open-source project) let's you do this sort of stuff in your native code very easily. For example,
* A C++ function that accepts a java.io.File and reads its contents.
void readFile( java::io::File& file ) {
const int bufferSize = 512;
JArray<JByte> buffer( bufferSize );
try {
    FileInputStream input( file );
    for ( int numBytesRead = 0; bytesRead != -1; ) {
      numBytesRead = input.read( buffer, 0, bufferSize );
      // Do something with the buffer
catch ( IOException& ioe ) {
    cout << ioe << endl;
} is just as easy to write as straight Java, and it's all implemented using standard JNI function calls under the covers. For example, the above code results in the invocation of several JNI functions, like, NewObject, NewByteArray, CallIntMethod, FindClass, GetMethodID, ExceptionOccurred, etc...
God bless,
-Toby Reyelts

Permission denied when read file with setGID

I'm using a NFS system. I noticed that once I set setGID to a file, e.g.
chmod g+s myfileThe file become un-readable (even root user is not able to access it). if I try to read, i got:
tail myfile
cannot open `myfile' for reading: Permission deniedWhat could be problem? using ls -l or getfacl, I can see owner,group, other all have read permission though:
ls -l
-rw-r-Sr--+ 1 oracle dba 5 Sep 19 07:51 myfile
getfacl myfile
# file: myfile
# owner: oracle
# group: dba
user::rw-
group::r--
mask::rwx
other::r--

Using the nosuid option is a good idea and you should consider using this with all NFS mounted disks. It means that the server's root user cannot make a suid-root program on the file system, log in to the client as a normal user and then use the suid-root program to become root on the client too. http://tldp.org/HOWTO/NFS-HOWTO/security.html
You could ask your system administrator to check file permissions on the parent directory, but it will require a more in-depth review of your configuration to analyze what's going on, and this is not possible unless you provide more information.

Tomcat 5 under NetBeans 3.6 read files with wrong path

The problem is quite simple:
I have XSLT file which I read in my servlet. Under Tomcat 4.1.29 (and earlier till 4.0.1) everything works fine, but Tomcat 5 under NetBeans 3.6 produces an error while he tries to read a file not from a directory
$CATALINA_HOME/webapps/..
but from a dir
$CATALINA_HOME/bin/webapps/...
anybody knows how to fix it? Why he wants to start reading from "bin" directory
P.S. Tomcat runs under WIn2000

Actually, sorry, it's more complicated than that... the rules for <xsl:include> say that by default the transformer will look in the same directory as the XSLT file that's doing this include. But that only works if the transformer knows what directory that is. So if you give the transformer an InputStream containing the XSLT (e.g. by using getResourceAsStream), it has no way of knowing what directory it came from. Use the version that creates a File object for the XSLT, and wrap that in a StreamSource when passing it to the transformer. That works for me.

How to Write BUFFER & Read TEXT with Encrypt file ?

I'm using Windows Phone 8.1 RT.
I have a issue :
- I write a BUFFER encrypted to file. After, I read file with TEXT. It's throw exception : No mapping for the Unicode character exists in the target multi-byte code page. (//ERROR 2)
- I write a TEXT encrypted to file. After, I read file with BUFFER. It's throw exception : The supplied user buffer is not valid for the requested operation. (//ERROR 1)
Code Write Buffer & Read Text.
//Write Textstring msg = EncryptText.Text;
//ERROR 1 - Use 1 or 2
await WriteTextAsync(this.file, msg);
//ERROR 1
//Read Buffer
string msg;
//ERROR 1 - Use 1 or 2
IBuffer buffer = await ReadBufferAsync(this.file);
StreamReader stream = new StreamReader(buffer.AsStream());
msg = stream.ReadToEnd();
//ERROR 1
Code Encrypt-Decypt.
public static string EncryptString(string msg)
var bufferMsg = CryptographicBuffer.ConvertStringToBinary(msg, BinaryStringEncoding.Utf8);
var bufferMsgEncrypted = Encrypt(bufferMsg);
var msgEncrypted = CryptographicBuffer.EncodeToBase64String(bufferMsgEncrypted);
return msgEncrypted;
}public static IAsyncAction WriteTextAsync(IStorageFile file, string msg)
return FileIO.WriteTextAsync(file, EncryptString(msg));
public static IBuffer Decrypt(IBuffer bufferMsg)
var key = CreateKey(KEY);
var aes = SymmetricKeyAlgorithmProvider.OpenAlgorithm(SymmetricAlgorithmNames.AesCbcPkcs7);
var symetricKey = aes.CreateSymmetricKey(key);
var bufferMsgDecrypted = CryptographicEngine.Decrypt(symetricKey, bufferMsg, null);
return bufferMsgDecrypted;
}public static IAsyncOperation<IBuffer> ReadBufferAsync(IStorageFile file)
var buffer = FileIO.ReadBufferAsync(file);
Task<IBuffer> result = Task.Run<IBuffer>(async () =>
var Buffer = await buffer;
return Decrypt(Buffer);
return result.AsAsyncOperation();
Link demo code :
https://drive.google.com/file/d/0B_cS3IYO936_akE0cmI4bExJMjh0RU9qR3RvWDBWWElZWC1z/view?usp=sharing

Please provide a working app so this can be tested. You can upload to OneDrive and share a link here.
Matt Small - Microsoft Escalation Engineer - Forum Moderator
If my reply answers your question, please mark this post as answered.
NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined
objects and unknown namespaces.

Read file with size 2Go : Pb

Similar Messages

Maybe you are looking for