Read file in chunks

hello,
as many of you know i'm trying to solve a problem memory with labview 7.1
many of you told me to read file in chunks and i'm trying to do it...
but i have some questions...
in attachment you find see my VI (it's incomplete): i would read my files 1 Mb for each step (now my file is 80 Mb)...i would ask you what i have to link to count input of the readfile vi in the true case (the last for-cycle-step, where i have to save the last part of my file).
Should it be correct done in this way?
Have i done the save-process in the correct way? (initialize the array knowing the array size and the using the arrayreplacesubset vi in the for loop)
thanks all
Using LabVIEW 7.1
Attachments:
ReadFileInChunks.vi ‏58 KB

EOF function give me the size of the file and not the offset of the EOF?
so in which way could i stop the while cycle?
yes,,,i'm reading the file line by line and the i'll concatenate strings (1000 in the example) to obtain a chunk of data
if i put in the count input of the read file function (with the line mode True) i don't obtain all lines in the output...i don't know why
you can see the image in attachment
in fact was to solve my problem that i tryed to concatanate strings...if i could receive in the output 1000 lines in a automatic way should be better...
and yes...in my case i'll make my operation in the case structure, when i obtaine 1000 lines
Using LabVIEW 7.1
Attachments:
examplee.png ‏43 KB

Similar Messages

JSF : Reading files greater than 200MB

Hi,
I am using JSF with tomahawk in my application.
From the jsp i am calling a POJO to read a file (in a zip file) based on a condition. I am reading the whole file and returning back the full file...obviously i hit a java.lang.OutofMemory exception.
I want to do something like send back each line (or chunk) that i read from the file back to the calling page.
Here is the code that I have
JSP:
<pre>
<%
     out.print(readObject.readFromZipFile(strPathToZipFile, logFileName);
%>
</pre>Here is the mehod readFromZipFile from the readObject class:
public String readFromZipFile(String zipFilePath, String filePath) {
     StringBuffer sbFile = new StringBuffer();
     byte data[] = new byte[blockSize];
     BufferedInputStream is = null;
     try {
          ZipFile zipfile = new ZipFile(archivePath);
          ZipEntry entry = (ZipEntry) zipfile.getEntry(filePath);
          is = new BufferedInputStream(zipfile.getInputStream(entry));
          while (bytesToRead > 0) {
               data = null;
               data = new byte[bytesToRead];
                 sbFile.append(new String(data));
          is.close();
     } catch(Exception e) {
          log.error(e);
     } finally {
          is = null;
          data = null;
     return sbFile.toString();
}I am reading the file and sending the data back as a string. Is there a way i can send the data back in chunks to the JSP / JSF and flush the data so that i dont hit the out of memory exception?
Any help is highly appreciated.

Hi BalusC,
I tried this big time...i am getting roasted !!!
This is what i tried in the servlet
//get the size of the file
int iFile = file.size();
//start reading file from 5000 bytes
int iPart = 5000;
//read the file in part increments of 1000 bytes
//so that we dont read the entire file at one shot
while(iPart < iFile){
//read the file from the given (start bytes)
//to the (start bytes + 1000)
response.getWriter().append
          (readZipFile(strPath,file,iPart,iPart+1000));
//increment iPart to read the next 1000 bytes , next time around
iPart = iPart + 1000;
//flush the writer so that you can clear it
response.getWriter().flush();
}I still get a out of memory error...
BTW i also tried response.getOutputStream().write()...
This is apart from the read file method given above...what am i doing wrong here?

IMAQ Read File error -1074396159

Hi NI,
I am getting "IMAQ Read File" error -1074396159 when trying to read a large image. I get this error when trying to read a ~400MB image on a very clean (not many extraneous applications) Windows 7 computer with 4GB or RAM. I tried this on two other not very clean Windows 7 computers with 4GB of RAM and 2GB of RAM respectively. On the latter two computers I do not get this error. I found this knowledgebase which explains one possible cause of this error. Are there any other causes for this error? Are there any workarounds?
Thanks,
Jonathan-LV

That KB is incorrect, as the memory only needs to be virtually contiguous inside the process address space, not physically contiguous in RAM. This is not affected by how many programs are loaded (with the very slight exception that some read-only pages of DLLs can be shared cross-process in certain cases). This is really a function of how fragmented the memory space is in the process itself. The problem is that a normal 32-bit app on 32-bit Windows only has 2GB of virtual memory space. 500MB is 25% of the entire space. DLLs tend to get loaded scattered throughout the memory space for various reasons and other memory allocations that have different lifetimes cause fragmentation. The number of multi-hundred-MB chunks available is very small in a large app like LabVIEW.
Some options to help:
-Moving to a 64-bit OS (still using 32-bit LabVIEW) will change that 2GB address space to 4GB with no changes to your app
-Moving to both a 64-bit OS plus 64-bit LabVIEW will change that 4GB address space to 8TB (virtually unlimited)
Eric

HOW TO DELETE ADOBE READER FILE FROM ADOBE READER

HOW TO DELETE ADOBE READER FILE FROM ADOBE READER@ !@

Your question make no sense. What exactly are you trying to do?

Reading Files from an FTP location based on Timestamp

Hi,
I have a requirement of reading files from an FTP location. The files should be read in the order they were saved in the folder. Can someone tell me how to solve this?
Thanks,
Ravi

Hi Arik,
Thanks for your reply. I tried doing what u said, but it isn't working. My requirement is to List the files in the folder using FTP adapter and get the file name with the least file stamp from the FTP adapter and pass the FileName dynamically to another FTP adapter which reads the file based on the FileName. When i tested after including the property (for listing the files) you mentioned, it is giving the following error (copied from flow trace):
<messages>
<input>
<Invoke_getFileListFTP_FileListing_InputVariable>
<part name="Empty" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<empty xmlns="http://xmlns.oracle.com/pcbpel/adapter/ftp/APP_SequentialFileTransfer/PRJ_SequentialFileTransfer/WS_GetFileList_FTP"/>
</part>
</Invoke_getFileListFTP_FileListing_InputVariable>
</input>
<fault>
<bpelFault>
<faultType>0</faultType>
<bindingFault xmlns="http://schemas.oracle.com/bpel/extension">
<part name="summary">
<summary>Exception occured when binding was invoked. Exception occured
during invocation of JCA binding: "JCA Binding execute of
Reference operation 'FileListing' failed due to: Exception
occured when binding was invoked. Exception occured during
invocation of JCA binding: "Could not instantiate
InteractionSpec
oracle.tip.adapter.ftp.outbound.FTPListInteractionSpec due
to: Cannot set JCA WSDL Property. Error while setting JCA
WSDL Property. Property setListSorter is not defined for
oracle.tip.adapter.ftp.outbound.FTPListInteractionSpec Please
verify the spelling of the property. ". The invoked JCA
adapter raised a resource exception. Please examine the above
error message carefully to determine a resolution. ". The
invoked JCA adapter raised a resource exception. Please
examine the above error message carefully to determine a
resolution.</summary>
</part>
<part name="detail">
<detail>Cannot set JCA WSDL Property. Error while setting JCA WSDL
Property. Property setListSorter is not defined for
oracle.tip.adapter.ftp.outbound.FTPListInteractionSpec Please
verify the spelling of the property.</detail>
</part>
<part name="code">
<code>null</code>
</part>
</bindingFault>
</bpelFault>
</fault>
<faultType>
<message>0</message>
</faultType>
</messages>
Thanks,
Ravi
Edited by: Ravi Santosh on Apr 24, 2012 11:02 PM

Reading file from an external folder.

Iam tring to read an external file from the C:\ drive. Given below is the code.
DATA LV_XLS(100) TYPE C.
DATA LV_CONTENT TYPE XSTRING.
LV_XLS = 'C:\XML\Report.xls'
READ DATASET LV_XLS INTO LV_CONTENT.
But when i execute iam getting an error - FILE NOT FOUND. How do i read file from an external folder. My requirement is to strictly read from an external folder. Please help.
If this is not possible please advice as to how the file could be moved to the SAP root directory. Iam not able to locate the root.

hi,
This is the code to read a excel file from the local system.
Read data set you are using in your code corresponds to files on application server thats the reason its throwing an error.
* Parameter variable declaration for browsing the file location       *
PARAMETERS:
p_file TYPE ibipparms-path OBLIGATORY.
data:
it_text TYPE truxs_t_text_data .
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
CALL FUNCTION 'F4_FILENAME'
   EXPORTING
     program_name        = syst-cprog
*   DYNPRO_NUMBER       = SYST-DYNNR
     field_name          = ' '
   IMPORTING
     file_name           = p_file.
START-OF-SELECTION.
CALL FUNCTION 'TEXT_CONVERT_XLS_TO_SAP'
    EXPORTING
*   I_FIELD_SEPERATOR          =
*   I_LINE_HEADER              =
      i_tab_raw_data             = it_text
      i_filename                 = p_file
    TABLES
      i_tab_converted_data       = t_cust
EXCEPTIONS
    conversion_failed          = 1
    OTHERS                     = 2
IF sy-subrc <> 0.
* MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
*         WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
This is to read a text file from local system.
CALL FUNCTION 'GUI_UPLOAD'
    EXPORTING
    filename                      = 'D:\FILEINTERFACE\file.TXT'
*    FILETYPE                      = 'ASC'
     has_field_separator           = 'X'
*    HEADER_LENGTH                 = 0
*    READ_BY_LINE                  = 'X'
*    DAT_MODE                      = ' '
*    CODEPAGE                      = ' '
*    IGNORE_CERR                   = ABAP_TRUE
*    REPLACEMENT                   = '#'
* IMPORTING
*    FILELENGTH                    =
*    HEADER                        =
    TABLES
      data_tab                      = t_kna1
* EXCEPTIONS
*    FILE_OPEN_ERROR               = 1
*    FILE_READ_ERROR               = 2
*    NO_BATCH                      = 3
*    GUI_REFUSE_FILETRANSFER       = 4
*    INVALID_TYPE                  = 5
*    NO_AUTHORITY                  = 6
*    UNKNOWN_ERROR                 = 7
*    BAD_DATA_FORMAT               = 8
*    HEADER_NOT_ALLOWED            = 9
*    SEPARATOR_NOT_ALLOWED         = 10
*    HEADER_TOO_LONG               = 11
*    UNKNOWN_DP_ERROR              = 12
*    ACCESS_DENIED                 = 13
*    DP_OUT_OF_MEMORY              = 14
*    DISK_FULL                     = 15
*    DP_TIMEOUT                    = 16
*    OTHERS                        = 17
IF sy-subrc <> 0.
* MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
*         WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
PS. In both of the above mentioned function modules make sure that the file structure is similar to the internal tables used.
Thanks
Sharath

Reading file from a folder : File dir = new File(URL+aFile_to_add);

Hi,
Trying to read files from a dir on my site using a JSP and bean. I've tried nearly every option for the filepath I can think of and it doesn't seem to be finding the file...
Here's the bit:
String URL = "images\\wedding_photos\\";
File dir = new File(URL+aFile_to_add);
String[] files = dir.list();
It's working locally, but I obviously had to change the path from c://... to the above.
The full code is listed below in the addWedding Method
The method removes the .jpg from the file name and then writes the new file name to a database
Thanks for any insight.
Regards
Jim
public String addWedding(String aFile_to_add)
String feedback = "unset in scrubWedding";
try
String URL = "images\\wedding_photos\\";
Class.forName("org.gjt.mm.mysql.Driver");
java.sql.Connection connection = java.sql.DriverManager.getConnection("jdbc:mysql://localhost/rhwedd2_shop?user=**********&password=*******");
java.sql.Statement statement = connection.createStatement();
File dir = new File(URL+aFile_to_add);
String[] files = dir.list();
if (files == null)
feedback = "Sorry, couldn't find the file "+aFile_to_add+"...no files have been added.";
else
for (int i=0; i<files.length; i++)
// Get filename of file or directory
String filename = files;
String no_extension;
//convert string array element into a char array
char [] charsfilename = filename.toCharArray();
int newlength = charsfilename.length -4;
no_extension = String.valueOf(charsfilename,0,newlength);
filesAdded = filesAdded+", "+no_extension;
// Show the new file names in stack trace without extension
//System.out.println("String no_extension " + no_extension );
statement.executeUpdate("INSERT INTO items VALUES ('"+no_extension+"',5,'"+aFile_to_add+"')");
// write to table with item_id and weddingid
feedback = "The following files have been added..."+filesAdded;
if (statement != null )
statement.close();
if ( connection != null )
connection.close();
catch(Exception e)
feedback = "Into the catch";
e.printStackTrace(System.err) ;
return feedback;
}//end addWedding

String URL = "images\\wedding_photos\\";
File dir = new File(URL+aFile_to_add); ok, ok, I'll answer...
1) What gives you the impression that the relative directory "images\\..." exists (or, in other words, what makes you think your current directory is the parent of "images"?

Unable to read file from application server

Hi guys,
I am reading file(could be any extension) from application server,but some time i am successfuly able to read file and sometime unable to read,why its happening .
my code is here
OPEN DATASET E_FILE FOR INPUT IN BINARY MODE . "
IF SY-SUBRC = 0.
    DO .
      READ DATASET E_FILE INTO GS_PDF_TAB.
      IF SY-SUBRC = 0.
        APPEND GS_PDF_TAB TO GT_PDF_TAB.
      ELSE.
        EXIT.
      ENDIF.
    ENDDO.
    CLOSE DATASET E_FILE.
ENDIF.
Thanks
Ankur Sharma

Hi,
What actually happens? Do you get a short dump? Do you get a return code ne 0? Does it run fine but you get no data in your table?
We aren't mind-readers and can't help much without more information.
Try using transaction AL11 to see if you access the files you are trying to open.
Gareth.

Uploading and reading file from application server

Hi
My problem is when am uploading a file to application server it is getting stored in
usr/sap/transyp1/prod/in   directory
after that i want to read that file from application server to update database
when using below code it is showing some other directory in f4 help
DATA: lv_hostname TYPE msxxlist-name.
DATA: lv_server TYPE bank_dte_jc_servername.
PARAMETERS: p_file TYPE rlgrap-filename.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
CALL FUNCTION 'BANK_API_SYS_GET_CURR_SERVER'
IMPORTING
e_server = lv_server.
lv_hostname = lv_server.
CALL FUNCTION 'F4_DXFILENAME_4_DYNP'
EXPORTING
dynpfield_filename = 'P_FILE'
dyname = sy-cprog
dynumb = '1000'
filetype = 'P'
location = 'A'
server = lv_hostname.
experts could you please help me out
Thanks & Regards
Nagesh.Paruchuri

User Transaction file. You will get all logical file path names.
used following fucntion module to read file name and use command open dataset to read the file.
CALL FUNCTION 'FILE_GET_NAME'
       EXPORTING
            CLIENT           = SY-MANDT
            LOGICAL_FILENAME = C_LOGICAL_FILENAME
            OPERATING_SYSTEM = SY-OPSYS
            PARAMETER_1      = P_IN_FILENAME
       IMPORTING
            FILE_NAME        = P_OUT_FILENAME
       EXCEPTIONS
            FILE_NOT_FOUND   = 1
            OTHERS           = 2.
OPEN DATASET P_OPEN_FILE ENCODING UTF-8 IN TEXT MODE FOR OUTPUT.
IF SY-SUBRC <> 0.
    MESSAGE E000(38) WITH 'Error in Opening file: ' V_PHY_FILENAME.
ENDIF.

Open Data Set Error while trying to read file from non SAP server

Hi all,
is it possible to read data from non-SAP application Sever?
I'm using OPEN DATASET p_filin FOR INPUT IN LEGACY TEXT MODE CODE PAGE '1504',
Where p_filin is other Windows server.Our applicition server is under Unix.Is it a problem?
I make test to read file from SAP application server and it was ok.So how to call other server?
Thanks!

Hi,
Yes it is possible to read data from a non SAP server through the statement OPEN DATASET.
The important thing to check is that the SAP Server got enough access to the non SAP server so it can perform a reading/writing process depending on your needs.
You should contact your network administrator and BASIS to help you check the permissions. This can be pretty tricky, specially if the servers are in different domains.
Regards,
Gilberto Li

How to combine both DAQ AI signal, write and read file in single VI

Hi
I am the new user of LabVIEW version 7.1 for testing automation application. I have to measure 33 signals ( mostly analog like temp, pressure, etc...) from NI USB 6210 DAQ system and write in master file for future verfication.From real data or from master file back up have to write one more file if only the signal reaches steady state , which will used for analysis and same signals to be read from this file parallely & make a waveform and/or table display format.
Pl. help me to shortout this problem
note: I have plan to ugrade labVIEW version 2011 shortly, so let me know doing parrel acquistion write and read file for data analysis in same VI in version 7.1......

Parallel operations in LabVIEW are very simple. Just code it in parallel and it will work.
Try taking a look at some of the examples in the NI Example Finder (Help > Find Examples). There you will find example for writing to and reading from files, as well as data acquistion in parallel with other operations.
You might need a producer/consumer architecture is you are acquiring data very quickly.
Chris
Certified LabVIEW Architect
Certified TestStand Architect

Error ORA-20003: Cannot read file/... in Cash Management?

Dear All.
I am having this Error in the Cash Management Module on the bank statement load process
Error ORA-20003: Cannot read file /software/d01/oracle/interface/finance/inbound/email.9Nemail.13111507.3445704.cnv.
How do I resolve this error.
Thank you.

Hi Lucy,
The error is probably due to one of the reasons:
1. There is no file in the location: /software/d01/oracle/interface/finance/inbound/email.9Nemail.13111507.3445704.cnv.
2. No permission on the file respective file.
Please get the assistance from the DBA to check on this.
Thanks &
Best Regards,

[Urgent] How to read files from different directories?

I am new to Java Programming, I would like to know how to read files from directories other than the current one? (example as follows)
ProjectDirectory
|--MainDirectory
|--MainProgram.java
|--SupplementDirectory
|--SupplementProgram.java
|--Pictures
|--Image.gif
What should I write in the MainProgram.java so that I can use the supplementProgram.java from MainProgram and read the Image.gif file from the MainProgram.java?
Thanks

Run through the I/O tutorial here. It should get you up to speed on this sort of thing...

Can the aTV read files from an iTunes library on my nas drive without my computer being on

I have my itunes library on a My Book Live nas drive that contains all of my digital movies and TV shows. I want to be able to watch my movies from the library without a computer running. I understand that Apple has chosen to chain you to the computer and your itunes account through Home Sharing but I can not see why they couldn't allow you to access a nas drive without a computer running because your aTV is tied to your itunes account also so why do they force you to run a computer 24/7 365 in order to access the files you have bought and paid for.
I have a love/hate for Apple because I love the quality of the products they make but I hate how they make all of the digital files you purchase only accessible through a computer running Home Sharing even though the files are not stored on the computer.
I have tried several things to get this to work without a computer but no luck. The only way I have found to be able to do this is to Jailbreak your aTV and install 3rd party software (which I don't want to do) that will allow you to by pass the computer and Home Sharing (which is dumber than ****) because that is all I want to do is eliminate the need of a computer running wasting electricity (I thought Apple was supposed to be an enviro friendly company, guess not). I am not trying to steal or give someone else my purchased movies and using the aTV and an internet connection they could still be able to tie your media to your itunes account so I just don't get there thinking, or lack of thinking I should say.
Is their a way to do this without Jailbreaking or modifying your aTV?
If not is Apple ever going to wake up and allow there customers to actually USE the media they purchase from them without Home Sharing and a computer running?

_Skull wrote:
Can the aTV read files from an iTunes library on my nas drive without my computer being on
no it can't. you need itunes open and running

How can I transfer an adobe reader file to Microsoft word?

I have Windows Vista on my machine. I really need to transfer an Adobe reader file into Microsoft word or at least be able to type inside of Adobe. Just wondering how can I do this task or am I just not able to?
Any help or suggestions out there..?

That would require using Adobe Acrobat. It can create Word docs from PDF's.
The bad news is, most of the time it doesn't do a very good job at it.
With Acrobat, you could also add form fields that you can fill in which would be much better.

Read file in chunks

Similar Messages

Maybe you are looking for