How can i open large data?
he code in the screenhot opens realy all data
formats, but only small data, when the data becomes too large, LabView
say: "Not enough Memory to complite this operation". But I have enough
Memory. This appear already by a data-largeness of fwe bytes (maybe
more than 10 MB).
I have 1 Gbyte Memory, also enough, but when i
will open a small data with only 10 MByte it say's to me that i have
not enough memory.
Ok when i would open a data with more than
1Gbyte, for example a zip-file, i can understand it, but the data are
realy not large.
Can somebody say waht is wrong?
ThX
Attachments:
Read all files.vi.png 12 KB
You should also read the tutorial Managing Large Data Sets in LabVIEW. Some things you will learn:
Your current code is making several copies of your data. The tutorial will teach you how to find them and eliminate them. The tutorial has not been updated for LabVIEW 8.5 yet, and there are several enhancements in LabVIEW 8.5. The updated version is posted below. The code examples did not change.
For best speed in reading from the disk, you want to use 65,000 byte chunks.
Store your data in a single-element queue. This will give you best performance for a large array.
Store your data as a set of arrays instead of one array (this has been mentioned above). You can break it up into several single-element queues or save it as an array of clusters, each cluster containing a sub-array of the data. The cluster acts sort of like a handle in C.
Let us know if you need more help.
This account is no longer active. Contact ShadesOfGray for current posts and information.
Attachments:
Memory Management in LabVIEW.doc 132 KB
Similar Messages
-
How can I open files sent as attachments from Windows that appear as "win.dat" when they arrive. I used to use TNEF but that does not work anymore.
Try this:
http://www.joshjacob.com/mac-development/tnef.php
(TNEF's Enough - a good little app for this very purpose) -
How can i open WINMAIL.DAT attached to e-mails ???
Hi,
anybody knows how to open attached WINMAIL.DAT in e-mails (Yosemite 10.10.3)
Thanks.See the suggestion by thomas_r.
Get a copy of TNEF
HOW TO OPEN WINDAT FILES -
how can i open a PDF bank statement in "numbers" so that the rows and columns contain properly aligned data from statement?
Numbers can store pdfs pages or clippings but does not directly open pdf files. To get the bank statement into Numbers as a table I would open the bank statment in Preview (or Skim) or some pdf viewer.
Then hold the option key while selecting a column of data.
Then copy
Then switch to numbers and paste the column into a table
Then repeat for the other columns in the pdf document
It would be easier (in my opinion) to download the QFX or CSV version from your bank -
How can i open a DOC or TXT file and insert the data into table?
How can i open a DOC or TXT file and insert the data into table?
I have a doc file . the doc include some columns and some rows.(for example 'ID,Name,Date,...').
I'd like open DOC file and I'd like insert them into the table with same columns.
Thanks.Use the SQL*Loader utility or the UTL_FILE package.
-
My Iphone is deactivate, I have photos with out back up, How can I open with out loosing my data.
My Iphone is deactivate, I have photos with out back up, How can I open with out loosing my data.
Aussienana wrote:
OK, so I went and explored what this site said about Mackeeper. But before I delete it can you tell me:
If we don't need anti virus program and if the one that came with the machine is supposed to do the job then what does that say about the viruses that Mackeeper says it found? Were they not there and the program made them up? Or were they there and Apple didn't deal with them?
Thanks for all feedback.
They were not there and MacKeeper made them up. Delete it completely. I have seen it cripple people's Macs.
Uninstall MacKeeper
Pete -
Hi i have a learning HTML file with a data file include a swf files how can i open it in my ipad?
hi i have a learning HTML file with a data file include a swf files how can i open it in my ipad?
iPad do not support flash natively.
Look in the app store, some browsers do support flash with limited functionality. -
How can I open the 'time date browse button'?
Hello!
How can I open the 'time date browse dialog' from the time stamp control in my block diagram? I tryed it with the property node but couldn't find this item to open the 'time/date browse dialog'!
Thanks in advance for your helpThere is no way to programitically bring up the dialog window for the timestamp control. I recommend building your own dialog window VI that will work by opening when you want the user to set the time and when it closes, it can programatically update the timestamp control. It will not be as pretty but it will be just as functional.
-
How can i open my cellular data. I dont have the cellular network in my settings
how can i open my cellular data. I dont have the cellular network in my settings. Im using Iphone 4s bought in Japan and ulocked here in the philippines using Globe carrier
Are you trying to set an APN setting on it?
Settings>Cellular>Cellular Data Network -
After updating my iphone through iTunes I cant open it. It ask me to restore my phone. How can I open it without restoring ?
Go into your iTunes preferences and select devices (on a Mac it's iTunes > Preferences > Devices; on PC it's Edit > Preferences > Devices). This will show you any and all iTunes backups you have, including your phone. If you don't see any backups, then you don't have one in iTunes.
With iCloud, if you've got a backup you can view it using an iOS device that's logged in with your iCloud (apple ID) account. You go into Settings > iCloud > Storage & Backup > Manage Storage and see if your phone is listed there. If it is, then you have a backup for it. You can click on it to see how large the backup is and what is backed up.
Don't be afraid to restore it. The worst that can happen is that you lose data. If you have a backup, then you can get that data back. If not, then you will have to start over, but your phone will still work. -
How can I load my data faster? Is there a SQL solution instead of PL/SQL?
11.2.0.2
Solaris 10 sparc
I need to backfill invoices from a customer. The raw data has 3.1 million records. I have used pl/sql to load these invoices into our system (dev), however, our issue is the amount of time it's taking to run the load - effectively running at approx 4 hours. (Raw data has been loaded into a staging table)
My research keeps coming back to one concept: sql is faster than pl/sql. Where I'm stuck is the need to programmatically load the data. The invoice table has a sequence on it (primary key = invoice_id)...the invoice_header and invoice_address tables use the invoice_id as a foreign key. So my script takes advantage of knowing the primary key and uses that on the subsequent inserts to the subordinate invoice_header and invoice_address tables, respectively.
My script is below. What I'm asking is if there are other ideas on the quickest way to load this data...what am I not considering? I have to load the data in dev, qa, then production so the sequences and such change between the environments. I've dummied down the code to protect the customer; syntax and correctness of the code posted here (on the forum) is moot...it's only posted to give the framework for what I currently have.
Any advice would be greatly appreciated; how can I load the data faster knowing that I need to know sequence values for inserts into other tables?
DECLARE
v_inv_id invoice.invoice_id%TYPE;
v_inv_addr_id invoice_address.invoice_address_id%TYPE;
errString invoice_errors.sqlerrmsg%TYPE;
v_guid VARCHAR2 (128);
v_str VARCHAR2 (256);
v_err_loc NUMBER;
v_count NUMBER := 0;
l_start_time NUMBER;
TYPE rec IS RECORD
BILLING_TYPE VARCHAR2 (256),
CURRENCY VARCHAR2 (256),
BILLING_DOCUMENT VARCHAR2 (256),
DROP_SHIP_IND VARCHAR2 (256),
TO_PO_NUMBER VARCHAR2 (256),
TO_PURCHASE_ORDER VARCHAR2 (256),
DUE_DATE DATE,
BILL_DATE DATE,
TAX_AMT VARCHAR2 (256),
PAYER_CUSTOMER VARCHAR2 (256),
TO_ACCT_NO VARCHAR2 (256),
BILL_TO_ACCT_NO VARCHAR2 (256),
NET_AMOUNT VARCHAR2 (256),
NET_AMOUNT_CURRENCY VARCHAR2 (256),
ORDER_DT DATE,
TO_CUSTOMER VARCHAR2 (256),
TO_NAME VARCHAR2 (256),
FRANCHISES VARCHAR2 (4000),
UPDT_DT DATE
TYPE tab IS TABLE OF rec
INDEX BY BINARY_INTEGER;
pltab tab;
CURSOR c
IS
SELECT billing_type,
currency,
billing_document,
drop_ship_ind,
to_po_number,
to_purchase_order,
due_date,
bill_date,
tax_amt,
payer_customer,
to_acct_no,
bill_to_acct_no,
net_amount,
net_amount_currency,
order_dt,
to_customer,
to_name,
franchises,
updt_dt
FROM BACKFILL_INVOICES;
BEGIN
l_start_time := DBMS_UTILITY.get_time;
OPEN c;
LOOP
FETCH c
BULK COLLECT INTO pltab
LIMIT 1000;
v_err_loc := 1;
FOR i IN 1 .. pltab.COUNT
LOOP
BEGIN
v_inv_id := SEQ_INVOICE_ID.NEXTVAL;
v_guid := 'import' || TO_CHAR (CURRENT_TIMESTAMP, 'hhmissff');
v_str := str_parser (pltab (i).FRANCHISES); --function to string parse - this could be done in advance, yes.
v_err_loc := 2;
v_count := v_count + 1;
INSERT INTO invoice nologging
VALUES (v_inv_id,
pltab (i).BILL_DATE,
v_guid,
'111111',
'NONE',
TO_TIMESTAMP (pltab (i).BILL_DATE),
TO_TIMESTAMP (pltab (i).UPDT_DT),
'READ',
'PAPER',
pltab (i).payer_customer,
v_str,
'111111');
v_err_loc := 3;
INSERT INTO invoice_header nologging
VALUES (v_inv_id,
TRIM (LEADING 0 FROM pltab (i).billing_document), --invoice_num
NULL,
pltab (i).BILL_DATE, --invoice_date
pltab (i).TO_PO_NUMBER,
NULL,
pltab (i).net_amount,
NULL,
pltab (i).tax_amt,
NULL,
NULL,
pltab (i).due_date,
NULL,
NULL,
NULL,
NULL,
NULL,
TO_TIMESTAMP (SYSDATE),
TO_TIMESTAMP (SYSDATE),
PLTAB (I).NET_AMOUNT_CURRENCY,
(SELECT i.bc_value
FROM invsvc_owner.billing_codes i
WHERE i.bc_name = PLTAB (I).BILLING_TYPE),
PLTAB (I).BILL_DATE);
v_err_loc := 4;
INSERT INTO invoice_address nologging
VALUES (invsvc_owner.SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
v_inv_id,
'BLAH INITIAL',
pltab (i).BILL_DATE,
NULL,
pltab (i).to_acct_no,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
SYSTIMESTAMP,
NULL);
v_err_loc := 5;
INSERT INTO invoice_address nologging
VALUES ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
v_inv_id,
'BLAH',
pltab (i).BILL_DATE,
NULL,
pltab (i).TO_ACCT_NO,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
SYSTIMESTAMP,
NULL);
v_err_loc := 6;
INSERT INTO invoice_address nologging
VALUES ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
v_inv_id,
'BLAH2',
pltab (i).BILL_DATE,
NULL,
pltab (i).TO_CUSTOMER,
pltab (i).to_name,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
SYSTIMESTAMP,
NULL);
v_err_loc := 7;
INSERT INTO invoice_address nologging
VALUES ( SEQ_INVOICE_ADDRESS_ID.NEXTVAL,
v_inv_id,
'BLAH3',
pltab (i).BILL_DATE,
NULL,
'SOME PROPRIETARY DATA',
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
NULL,
SYSTIMESTAMP,
NULL);
v_err_loc := 8;
INSERT
INTO invoice_event nologging (id,
eid,
root_eid,
invoice_number,
event_type,
event_email_address,
event_ts)
VALUES ( SEQ_INVOICE_EVENT_ID.NEXTVAL,
'111111',
'222222',
TRIM (LEADING 0 FROM pltab (i).billing_document),
'READ',
'some_user@some_company.com',
SYSTIMESTAMP);
v_err_loc := 9;
INSERT INTO backfill_invoice_mapping
VALUES (v_inv_id,
v_guid,
pltab (i).billing_document,
pltab (i).payer_customer,
pltab (i).net_amount);
IF v_count = 10000
THEN
COMMIT;
END IF;
EXCEPTION
WHEN OTHERS
THEN
errString := SQLERRM;
INSERT INTO backfill_invoice_errors
VALUES (
pltab (i).billing_document,
pltab (i).payer_customer,
errString || ' ' || v_err_loc
COMMIT;
END;
END LOOP;
v_err_loc := 10;
INSERT INTO backfill_invoice_timing
VALUES (
ROUND ( (DBMS_UTILITY.get_time - l_start_time) / 100,
2)
|| ' seconds.',
(SELECT COUNT (1)
FROM backfill_invoice_mapping),
(SELECT COUNT (1)
FROM backfill_invoice_errors),
SYSDATE
COMMIT;
EXIT WHEN c%NOTFOUND;
END LOOP;
COMMIT;
EXCEPTION
WHEN OTHERS
THEN
errString := SQLERRM;
INSERT INTO backfill_invoice_errors
VALUES (NULL, NULL, errString || ' ' || v_err_loc);
COMMIT;
END;Hello
You could use insert all in your case and make use of sequence.NEXTVAL and sequence.CURRVAL like so (excuse any typos - I can't test without table definitions). I've done the first 2 tables, so it's just a matter of adding the rest in...
INSERT ALL
INTO invoice nologging
VALUES ( SEQ_INVOICE_ID.NEXTVAL,
BILL_DATE,
my_guid,
'111111',
'NONE',
CAST(BILL_DATE AS TIMESTAMP),
CAST(UPDT_DT AS TIMESTAMP),
'READ',
'PAPER',
payer_customer,
parsed_francises,
'111111'
INTO invoice_header
VALUES ( SEQ_INVOICE_ID.CURRVAL,
TRIM (LEADING 0 FROM billing_document), --invoice_num
NULL,
BILL_DATE, --invoice_date
TO_PO_NUMBER,
NULL,
net_amount,
NULL,
tax_amt,
NULL,
NULL,
due_date,
NULL,
NULL,
NULL,
NULL,
NULL,
SYSTIMESTAMP,
SYSTIMESTAMP,
NET_AMOUNT_CURRENCY,
bc_value,
BILL_DATE)
SELECT
src.billing_type,
src.currency,
src.billing_document,
src.drop_ship_ind,
src.to_po_number,
src.to_purchase_order,
src.due_date,
src.bill_date,
src.tax_amt,
src.payer_customer,
src.to_acct_no,
src.bill_to_acct_no,
src.net_amount,
src.net_amount_currency,
src.order_dt,
src.to_customer,
src.to_name,
src.franchises,
src.updt_dt,
str_parser (src.FRANCHISES) parsed_franchises,
'import' || TO_CHAR (CURRENT_TIMESTAMP, 'hhmissff') my_guid,
i.bc_value
FROM BACKFILL_INVOICES src,
invsvc_owner.billing_codes i
WHERE i.bc_name = src.BILLING_TYPE;Some things to note
1. Don't commit in a loop - you only add to the run time and load on the box ultimately reducing scalability and removing transactional integrity. Commit once at the end of the job.
2. Make sure you specify the list of columns you are inserting into as well as the values or columns you are selecting. This is good practice as it protects your code from compilation issues in the event of new columns being added to tables. Also it makes it very clear what you are inserting where.
3. If you use WHEN OTHERS THEN... to log something, make sure you either rollback or raise the exception. What you have done in your code is say - I don't care what the problem is, just commit whatever has been done. This is not good practice.
HTH
David
Edited by: Bravid on Oct 13, 2011 4:35 PM -
How can I select a date to view in Calendar while I am in month view?
I am looking at month view in Calendar on my iPad. I touch a date. Intuitively, I expect that date to open up in Day or Week view, but it doesn't! On my iphone, it works just fine. Am I missing something?
How can I select a date to view while in Month view and have it go to Day or Week view?
Thanks for any help.I would try this:
In Lr import one photo into a different folder that you can see in the list.
Then in your OS (mac Finder / Win Explorer) move the photo to the folder that you cannot see.
In Lr the photo will now appear with a question mark on its frame. Click directly on the question mark and select <Locate photo>. Navigate to the folder, open it, and click on the photo.
Maybe this will help Lr to recognize the folder.
If this doesn't help try renaming the folder in your OS. -
How can I get a date picker or wheel to show up in the form to make filling in a form with a lot of
how can I get a date picker or wheel to show up in the form to make filling in a form with a lot of dates more easy?
There is no built-in date picker available for forms created in Acrobat. There are some third-party solutions involving JavaScript (either a large collection of fields or a custom dialog) though. Also, text fields with date formatting will use a wheel type date picker on Android/iOS devices with Adobe Reader.
-
How can I extract a dates column from an ascii file
I am extracting data from an ascii file and the rest of the data is extracted using read from spreadsheet, but the date appears as just the day e.g. 22 for 22nd, how can I get the date, format in ascii is e.g 22/04/2006, can I do this by specifying something at format of read from spreadsheet ? Cheers
EmilyHi,
I found the other day that there are instructions included inside the Read From Spreadsheet vi on how to modify it slightly to read strings rather than doubles.
If you drop the Read From Spreadsheet vi onto the block diagram and double click on it, you will open the front panel for the vi. If you then select the block diagram (for Read From Spreadsheet) you will see an 'instructions' frame below the main 'code', and the 3rd paragraph tells you how to modify the VI to read the spreadsheet as a set of strings rather then doubles. Save the vi as a new filename when you've edited it or you will overwrite the existing vi!
Using the newly modified vi you can read in your entire spreadsheet, extract the dates as strings, then do any conversion necessary to the other columns of the sheet to change them into the types of value you require (eg doubles).
I hope this helps.
Mark
Applications Engineer
National Instruments -
How can i restore my data synchronised from a serv...
I use Nokia C3-00, And in My country synchronisation with server is not available except i perform phone switch or with a PC...But in thesame country of mine there is a server people use to synchronise...And when i tried it on my phone with the settings given to me...I synchronised my contacts,notes and calender...and the phone said 'synchonisation sucessfull' but after i restored my phone coz it had an error..and restoring it deleted my data(contacts,notes etc)...Now my question is, how can i restore my data(contacts,notes) that i synchronised ? Coz i checked d phone...i did not see an option to restore d data.....And if i can't, i'm sorry but nokia put that 'sync with server' feature just for fun ...
TSNA - My system crashed 1/2 way through opening up Parallels. I use Parallels for Quicken & Quickbooks only. My system locked up & I had to power down to get any response on the screen. When it started back up I got a screen that said Restore from Time Machine, Safe Startup (or some such words) and one other choice. NONE of which would do anything. - No response to any clicking, entering, up down etc. I called over my Apple friend and he could not get it going,. Ended up having to Reload O/S from the startup disks and reformat my hard drive. Reformatted because I wanted to get rid of Parallels which has painfully slowed down my entire system. I never keep it open unless I am working with Quickbooks or Quicken. Therefore I had decided to start using Boot Camp anyway. So I would have had to reformat my hard drive to do that regardless of this freeze up of my system.
This whole thing has taken days to get my documents off Time Capsule. Time Machine would then not restore to my "new" or reformatted hard drive, so all the files on the Time Capsule were hidden. Total nightmare !
Because now I can't get iPhoto APP back on my system as it is not available and have to use the new PHOTO which I hear is not good.
Maybe you are looking for
-
** Not able to access Runtime Constants in ABAP XSL
Hi friends, We have written one ABAP XSLT program. (XSLT program runs in ABAP Engine). This program is for the interface Mapping in XI. Source Interface occurrence is 1 and target interface occurrence is 0..unbounded. ie. We use Message Split to send
-
I have one more doubt for GL creation which is asked by the clien TDS on contractors they created GL accounts Tax category as > input tax allowed and also they have not ticked only balances in local currency but the same for other TDS on rent ,TDS on
-
Does SSH come installed with Solaris 10 SPARC
How to enable SSH on new Solaris 10 box (SPARC) Thanks
-
Any fix to Lion 10.7.5 update problems?
I have a 2009 macbook pro that has always run perfectly. That is until I just downloaded the Lion 10.7.5 "update". What is did was make me very disappointed in Apple. Safari opens but that's it. It won't navigate anywhere. Blank screen. Also, I
-
After running into an error with itunes I figured I would fix it by uninstalling all apple software, (i.e. itunes, apple downloader, quicktime, and bonjour). After the removal I went to instal itunes 8.0 where when it was just about complete I was gi