Streaming data into a HDF5 file using mathscript

Hi Everybody,
I am developing a data logging system with a requirement to stream data into HDF5 files. I have previously written scripts in Matlab to stream the data using commands such as H5D.write etc. I do not seem to be able to access such low level commands in Mathscript. Is it possible to install any package to allow the use of these HDF5 commands in Mathscript?
Thanks,
Peter

I also asked this question but there seems no definite solution...
Anyway, here are two possibilities
1) Share the directory holding the file among all nodes
2) Run you procedure on a specific node

Similar Messages

How do you stream data to a HDF5 file?

I have managed to collect data to a hdf5 file but only for the first block of data (100, 1000, 10,000) but I would like to stream data until the hard disk is full or the user stops it. I have placed the HDF5 *.vi's, dll's, on my 'C' Drive.

There are at least 4 HDF5 APIs available for LabVIEW. Which one are you using? If you downloaded the sfpFile API from the NI Website, the example I attached below (which also uses NI-SCOPE) should work for you. You will probably need to relink to the appropriate API VIs when you load it. If you do not have NI-SCOPE, the example will still give you the info you need. Treat the scope VI as a generic data source and ignore the VI linking errors from NI-SCOPE VIs on load (you can get the NI-SCOPE driver here).
At each iteration, you need to do the following:
Get the current dataspace from the data on disk.
Extend this dataspace by the amount you will be adding (assumes an extensible data set, you can preallocate the size beforehand if you know what it is, which will make this step unnecessary).
Select a hyperslab on this extended dataspace corresponding to the data you will be writing.
Using the the new dataspace, write your data to disk.
This is a lot more complex than it could be, but that is the nature of HDF5 (very low level, very powerful). Let us know if you need more information. Please let us know what version of LabVIEW you are using, if you do.
This account is no longer active. Contact ShadesOfGray for current posts and information.
Attachments:
sfpFileEX Stream To Disk from NI-Scope.vi ‏234 KB

How to edit/update data into an XML file using Flex and Actionscript

I can read an external xml file, please see the code below:
protected function button1_clickHandler(event:MouseEvent):void
var GrowthChartsDataGrid:XML;
var loader:URLLoader = new URLLoader();
var request:URLRequest = new URLRequest("../GrowthChartsDataGrid.xml");
    loader.load(request);
    loader.addEventListener(Event.COMPLETE, onComplete)
function onComplete (event:Event)
     var loader:URLLoader = URLLoader(event.target);
     GrowthChartsDataGrid = new XML(loader.data);
     GrowthChartsDataGrid.GrowthChartGridView += <Month> {txtMonth.text} <Weight> {txtWeight.text} </Weight> </Month>
     texttesting.text = GrowthChartsDataGrid.toString();
I can read an XML file and can add an extra node and can display it in a text filed. But I want to update the XML file contents which will come from the txtMonth and txtWeight text boxes.
Please any suggestions???

First of all thanks for your quick reply .
I actually want to add another node inthe xml file. Files is at a local location and i can read the file and add an extra node (but I cant store this extra node in the actual XML file). But I want to save XML with the extra node.
For exmaple,
my current xml is:
<?xml version="1.0" encoding="utf-8"?>
<GrowthChartGridView>
    <Month> 1
    <Weight>3.5</Weight></Month>
<Month> 2
    <Weight>3.9</Weight></Month>
<Month> 3
    <Weight>4.5</Weight></Month>
</GrowthChartGridView>
and at run time, I can create a new node using the data from two textboxes at button click event.
     GrowthChartsDataGrid.GrowthChartGridView += <Month> {txtMonth.text} <Weight> {txtWeight.text} </Weight> </Month>
Now what I want to do is, I want to add this node back in to my XML. Therefore, the result I am looking for is, my local XML should update like this.
<?xml version="1.0" encoding="utf-8"?>
<GrowthChartGridView>
    <Month> 1
    <Weight>3.5</Weight></Month>
<Month> 2
    <Weight>3.9</Weight></Month>
<Month> 3
    <Weight>4.5</Weight></Month>
<Month> {txtMonth.text} <Weight> {txtWeight.text} </Weight> </Month> //I can read data from text boxes so its fine but I can not store this in to                                                                                                          my original XML
</GrowthChartGridView>
Thanks

Can I use Bridge to export image data into a .txt file?

I have a folder of images and I would like to export the File Name, Resolution, Dimensions and Color Mode for each file into one text file. Can I use Bridge to export image data into a .txt file?

Hello
You may try the following AppleScript script. It will ask you to choose a root folder where to start searching for *.map files and then create a CSV file named "out.csv" on desktop which you may import to Excel.
set f to (choose folder with prompt "Choose the root folder to start searching")'s POSIX path
if f ends with "/" then set f to f's text 1 thru -2
do shell script "/usr/bin/perl -CSDA -w <<'EOF' - " & f's quoted form & " > ~/Desktop/out.csv
use strict;
use open IN => ':crlf';
chdir $ARGV[0] or die qq($!);
local $/ = qq(\\0);
my @ff = map {chomp; $_} qx(find . -type f -iname '*.map' -print0);
local $/ = qq(\\n);
#     CSV spec
#     - record separator is CRLF
#     - field separator is comma
#     - every field is quoted
#     - text encoding is UTF-8
local $\\ = qq(\\015\\012);    # CRLF
local $, = qq(,);            # COMMA
# print column header row
my @dd = ('column 1', 'column 2', 'column 3', 'column 4', 'column 5', 'column 6');
print map { s/\"/\"\"/og; qq(\").$_.qq(\"); } @dd;
# print data row per each file
while (@ff) {
    my $f = shift @ff;    # file path
    if ( ! open(IN, '<', $f) ) {
        warn qq(Failed to open $f: $!);
        next;
    $f =~ s%^.*/%%og;    # file name
    @dd = ('', $f, '', '', '', '');
    while (<IN>) {
        chomp;
        $dd[0] = \"$2/$1/$3\" if m%Link Time\\s+=\\s+([0-9]{2})/([0-9]{2})/([0-9]{4})%o;
        ($dd[2] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of CODE\\s/o;
        ($dd[3] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of DATA\\s/o;
        ($dd[4] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of XDATA\\s/o;
        ($dd[5] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of FARCODE\\s/o;
        last unless grep { /^$/ } @dd;
    close IN;
    print map { s/\"/\"\"/og; qq(\").$_.qq(\"); } @dd;
EOF
Hope this may help,
H

Is it possible to have 2 different output config XML files and index the data into 2 endeca apps using the same indexing component ProductCatalogSimpleIndexingAdmin

Hi ,
We have a catalog that defines 2 types of products (they have too many different properties), so wanted to keep them on two different MDEX engines and serve the applications requests. Here DB catalog and front end ATG application is same for both the MDEX instances.
Is it possible to have 2 different output config XML files and index the data into 2 endeca apps using the same indexing component ProductCatalogSimpleIndexingAdmin?
Thanks
Dev

Hi, also have had some problem some monthes ago - I created separete component ProductCatalogSimpleIndexingAdminSecond. After that one of my colleage gave me some advice:
The creating separate component like ProductCatalogSimpleIndexingAdmin for the second IOC is possible way for resolving your situation. But I afraid that this way will be required creating mane duplicates for already existed components.
In my opinion the better way is the following:
starting from AssemblerApplicationConfiguration and ApplicationConfiguration component. It contains details for connecting between ATG and Endeca. Of course you should configure different components for different Endeca Apps.
After that:
Find all components that uses AssemblerApplicationConfiguration and ApplicationConfiguration. Customize these components for using one or another *Configuration component depending on what index works. (many variants released it: the most simple global custom component with flag.)
Then customize the existed ProductCatalogSimpleIndexingAdmin. Using one or another IOC and setting the flag in global custom component when index started. You can add some methods into your custom ProductCatalogSimpleIndexingAdmin like:
Execute baseline index for both IOC (one by one)
Execute baseline for IOC 1
Execute baseline for IOC 2.
Note: you should be afraid about incremental (partial) index in this configuration. But resolving conflicts in incremental index should be done after full implementation these changes.
Regards

Split records into Multiple csv files using a Threshold percentage

Hi Gurus,
I have a requirement to split the data into two csv file from a table using a threshold value(in Percentage) .
Assume that If my source select query of interface fetches 2000 records , I will provide a threshold value like 20%.
I need to generate a csv1 with 400 records(20% of 2000) and the rest of the records into another csv2.
For implementing this I am trying to use the following process.
1) Create a procedure with the select query to get the count of records.
Total Records count: select count(1) from source_table <Joins> <Lookups> <Conditions>;
2) Calculate the Record count to first CSV using the threshold_value.
CSV1_Count=Total records count /threshold_value
3) Create a view that fetches the CSV1_Count(400) records for CSV1 as follows.
Create view CSV1_view as select Col1,Col2,Col3 from source_table <Joins> <Lookups> <Conditions>
Where rownum<=CSV1_Count;
4) Generate CSV1 file using View 'CSV1_View'
5) Generate CSV2 File using the Interface with same select statement (with columns ) to generate a CSV.
select Col1,Col2,Col3 from source_table ST <Joins> <Lookups> <Conditions>
Left outer join (Select Col1 from CSV1_View ) CS on CS.Col1=ST.Col1 where CS.Col1 is null;
Which gives the Total records minus the CS1_View records.
The above process seems a bit complex and very simple . If any changes in my Interface I also need to change the procedure (counts the no:of records).
Please provide your comments and feedback about this and looking for your inputs for any new simple approach or fine tune the above approach.
Thanks,
Arjun

Arjun,
This are my thoughts and Lets do it in 3 Steps
Step 1. ODI Procedure
Drop table Temp_20 ;
Create table Temp_20 as select * from table where rownum < ( SELECT TRUNC( COUNT(1) /5) FROM TABLE ) .
[ ** This way iam fetching approx 20% of the table data and loading into Temp table . 1/5 th is 20% so i am dividing count by 5
I don't believe View will help you especially with RowNum as if you run the same query with rownum < N the rows order might differ . so Temp table is great ]
Step 2 . Use OdiSqlUnload with select columns from temp_20
Step 3 . Use again OdiSqlUnload with select columns from table where ( uk keys ) not in ( selecy uk_keys from temp_20)
[** this way you can pick the remaining 80% ** and the data will be not repeat itself across 20% and 80% , as might happen with view ]
what do you think ?

How to read a tab seperated data from a text file using utl_file

Hi,
How to read a tab seperated data from a text file using utl_file...
I know if we use UTL_FILE.get_line we can read the whole line...but i need to read the tab separated value separately.....
Thanks in advance...
Naveen

Naveen Nishad wrote:
How to read a tab seperated data from a text file using utl_file...
I know if we use UTL_FILE.get_line we can read the whole line...but i need to read the tab separated value separately.....If it's a text file then UTL_FILE will only allow you to read it a line at a time. It is then up to you to split that string up (search for split string on this forum for methods) into it's individual components.
If the text file contains a standard structure on each line, i.e. it is a fixed delimited structure, then you could use external tables to read the data instead.

Cannot insert date into numbers 3.1 using ctl shift d

can no longer insert date into numbers 3.1 using control shift d

Hi Gerald,
If you are inserting the current date a lot, and want a simple keyboard combination to do it rather than fiddling with formulas in Numbers, here is an easy way that takes advantage of some neat functionality built into every Mac that is much easier than it sounds. It requires the investment of just a few minutes of time to make a one-time setup of an "Automator service." Thereafter inserting a date is just a menu pick or keyboard combination.
1. Open Automator (in your applications folder) and choose Service as document type:
2. Drag 'Run AppleScript' action from the left over into the right pane:
3. Copy the following script and paste it into the right pane (replacing all of the default text in the pane):
on run
          set date_ to ((current date) as string)
set the clipboard to the date_
          tell application "Numbers"
activate
                    tell application "System Events"
keystroke "v" using {option down, shift down, command down}
                    end tell
          end tell
end run
4. At the top of the right pane choose 'no input' for 'Service receives' and 'Numbers.app' for 'in'. (On my machine I have the old Numbers renamed to Numbers09 so the script doesn't get confused).
After you click the hammer icon (to "compile") your right panel should now look like this:
5. Name the service "Today" or similar and move it to the Library > Services folder. (If it doesn't save automatically to that location you can File > Export in Automator, choose the 'Desktop' for 'Where', then in Finder open a new window and hold down the option key and choose Go > Library > Services, and drag Today.workflow from the Desktop into that folder.)
That's it. The service is ready to go. Thereafter, whenever you want to insert the current date, just pick Today from the Services menu:
6. Or, to make this even easier to access, choose Services Preferences and assign a keyboard shortcut to the service:
SG
Troubleshooting notes:
This should "just work." But if it doesn't for some reason, try one or more of the following:
- Rename the old Numbers (the one in the iWork '09 folder under Applications) to Numbers09 or similar.
- Check to make sure you aren't using the same keyboard shortcut for something else.
- Remove 'Day of the week' from your "Full" date format in System Preferences > Language & Region.
- Try revising permissions under System Preferences > Security & Privacy > Accessibility > Privacy.

Delete data from a flat file using PL/SQL -- Please help.. urgent

Hi All,
We are writing data to a flat file using Text_IO.Put_Line command. We want to delete data / record from that file after data is written onto it.
Please let us know if this is possible.
Thanks in advance.
Vaishali

There's nothing in UTL_FILE to do this, so your options are either to write a Java stored procedure to do it or to use whatever mechanism the host operating system supports for editing files.
Alternatively, you could write a PL/SQL procedure to read each line in from the file and then write them out to a second file, discarding the line you don't want along the way.

Error Loading Data into a flat file

I am recieving the following error when loading data into a flat file from a flat file. SQL 2005 is my back end DB. If I cut the file iin half approx 500K rows my ODI interface works fine. Not sure what to look at.. I rebuit the interface which before was just dying giving no error and now I am getting this.
Thanks
java.lang.Exception
     at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
     at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
     at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
     at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
     at com.sunopsis.dwg.cmd.e.i(e.java)
     at com.sunopsis.dwg.cmd.g.y(g.java)
     at com.sunopsis.dwg.cmd.e.run(e.java)
     at java.lang.Thread.run(Unknown Source)

Figured it out, found similar post that stated changing the HEAP size
Increase the page size in odiparams.bat in the bin folder and restart Designer.
For eg:
set ODI_INIT_HEAP=128m
set ODI_MAX_HEAP=1024m

Capturing EPC error List into a local file using BDC

Can anyone let me know the Code for downloading the EPC check Results into a local File using BDC...[<b></b>

You can use the function module
EXTENDED_PROGRAM_CHECK to get the errors and warnigs of a epc for a program.
then you can download that internal table information using GUI_DOWNLOAD fm.
Regards,
Ravi

How to convert a HTML files into a text file using Java

Hi guys...!
I was wondering if there is a way to convert a HTML file into a text file using java programing language. Likewise I would also like to know if there is a way to convert any type of file (excel, power point, and word) into text using java.
By the way, I really appreciated the help that you guys gave me on my previous topic on how to extract tests from a pdf file.
Thank you....

HTML files are already text files. What do you mean you want to convert them?
I think if you search the web, you can find things for converting those MS Office files to text (or extracting text from them, as I assume you mean).

I want to import a MS Word Index into a .pdf file using Adobe Pro. How do I do that?

I want to import a MS Word Index into a .pdf file using Adobe Pro. It is for a book. The book has been converted to a .pdf for printing, but it needs an Index for the last page of the book. I tried to embed a new index (starting from scratch but no luck. Any suggestions? Thank you for your time.

Now I want to search this "myArrayStrings.get(r)" in another file. How to do that?Store the returned String. Do the same thing you did to read the first file to read the second file (i.e. read it in line-by-line and store the lines in a list). Then check if that list contains the String you stored from the first list.
E.g. (using your code):
String name = myArrayStrings.get(r); // assuming r is initialized somewhere before this
if ( otherList.contains( name ) ) { // assuming otherList is your second list
// do something
}

How to get DocSet property values in a SharePoint library into a CSV file using Powershell

Hi,
How to get DocSet property values in a SharePoint library into a CSV file using Powershell?
Any help would be greatly appreciated.
Thank you.
AA.

Hi AOK,
Would you please post your current script and the issue for more effcient support.
In addition, to manage document set in sharepoint please refer to this script to start:
### Load SharePoint SnapIn
2.if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null)
3.{
4. Add-PSSnapin Microsoft.SharePoint.PowerShell
5.}
6.### Load SharePoint Object Model
7.[System.Reflection.Assembly]::LoadWithPartialName(“Microsoft.SharePoint”)
8.
9.### Get web and list
10.$web = Get-SPWeb http://myweb
11.$list = $web.Lists["List with Document Sets"]
12.
13.### Get Document Set Content Type from list
14.$cType = $list.ContentTypes["Document Set Content Type Name"]
15.
16.### Create Document Set Properties Hashtable
17.[Hashtable]$docsetProperties = @{"DocumentSetDescription"="A Document Set"}
18.$docsetProperties = @{"CustomColumn1"="Value 1"}
19.$docsetProperties = @{"CustomColum2"="Value2"}
20. ### Add all your Columns for your Document Set
21.
22.### Create new Document Set
23.$newDocumentSet = [Microsoft.Office.DocumentManagement.DocumentSets.DocumentSet]::Create($list.RootFolder,"Document Set Title",$cType.Id,$docsetProperties)
24.$web.Dispose()
http://www.letssharepoint.com/2011/06/document-sets-und-powershell.html
If there is anything else regarding this issue, please feel free to post back.
Best Regards,
Anna Wang
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

Need to read data from pipe separated file using POJO?

Hi,
I need to read data from pipe separated file using POJO.
There is config.properties file which consists of the
data mapping.
Can you help me with sample code or help?
Regards
Regards
Taton
Edited by: Taton on Mar 7, 2009 4:41 PM

It's not possible to read from a file without using classes from the core API*. You'll have to get clarification from your instructor as to which classes are and are not allowed.
[http://java.sun.com/docs/books/tutorial/essential/io/]
*Unless you write a bunch of JNI code to replicate what the java.io classes are doing.

Streaming data into a HDF5 file using mathscript

Similar Messages

Maybe you are looking for