Extracting specific data from multiple text files to single CSV
Hello,
Unfortunately my background is not scripting so I am struggling to piece together a powershell script to achieve the below. Hoping an experienced powershell scripter can provide the answer. Thanks in advance.
I have a folder containing approx. 2000 label type files that I need to extract certain information from to index a product catalog. Steps to be performed within the script as I see are:
1. Search folder for *.job file types
2. Search the files for certain criteria and where matched return into single CSV file
3. End result should be a single CSV with column headings:
a) DESCRIPTION
b) MODEL
c) BARCODE
Try:
# Script to extract data from .job files and report it in CSV
# Sam Boutros - 8/24/2014
# http://superwidgets.wordpress.com/category/powershell/
$CSV = ".\myfile.csv" # Change this filename\path as needed
$Folders = "d:\sandbox" # You can add multiple search folders as "c:\folder1","\\server\share\folder2"
# End Data entry section
if (-not (Test-Path -Path $CSV)) {
Write-Output """Description"",""Model"",""Barcode""" | Out-File -FilePath $CSV -Encoding ascii
$Files = Get-ChildItem -Path $Folders -Include *.job -Force -Recurse
foreach ($File in $Files) {
$FileContent = Get-Content -Path $File
$Keyword = "viewkind4"
if ($FileContent -match $Keyword) {
for ($i=0; $i -lt $FileContent.Count; $i++) {
if ($FileContent[$i] -match $Keyword) {
$Description = $FileContent[$i].Split("\")[$FileContent[$i].Split("\").Count-1]
} else {
Write-Host "Keyword $Keyword not found in file $File" -ForegroundColor Yellow
$Keyword = "Code:"
if ($FileContent -match $Keyword) {
for ($i=0; $i -lt $FileContent.Count; $i++) {
if ($FileContent[$i]-match $Keyword) {
$Parts = $FileContent[$i].Split(" ")
for ($j=0; $j -lt $Parts.Count; $j++) {
if ($Parts[$j] -match $Keyword) {
$Model = $Parts[$j+1].Trim()
$Model = $Model.Split("\")[$Model.Split("\").Count-1]
} else {
Write-Host "Keyword $Keyword not found in file $File" -ForegroundColor Yellow
$Keyword = "9313"
if ($FileContent -match $Keyword) {
for ($i=0; $i -lt $FileContent.Count; $i++) {
if ($FileContent[$i] -match "9313") {
$Index = $FileContent[$i].IndexOf("9313")
$Barcode = $null
for ($j=0; $j -lt 12; $j++) {
$Barcode += $FileContent[$i][($Index+$j)]
} else {
Write-Host "Keyword $Keyword not found in file $File" -ForegroundColor Yellow
Write-Output "File: '$File', Description: '$Description', Model: '$Model', Barcode: '$Barcode'"
Write-Output """$Description"",""$Model"",""$Barcode""" | Out-File -FilePath $CSV -Append -Encoding ascii
Sam Boutros, Senior Consultant, Software Logic, KOP, PA http://superwidgets.wordpress.com (Please take a moment to Vote as Helpful and/or Mark as Answer, where applicable)
Similar Messages
-
How can I use Automator to extract specific Data from a text file?
I have several hundred text files that contain a bunch of information. I only need six values from each file and ideally I need them as columns in an excel file.
How can I use Automator to extract specific Data from the text files and either create a new text file or excel file with the info? I have looked all over but can't find a solution. If anyone could please help I would be eternally grateful!!! If there is another, better solution than automator, please let me know!
Example of File Contents:
Link Time =
DD/MMM/YYYY
Random
Text
161 179
bytes of CODE memory (+ 68 range fill )
16 789
bytes of DATA memory (+ 59 absolute )
1 875
bytes of XDATA memory (+ 1 855 absolute )
90 783
bytes of FARCODE memory
What I would like to have as a final file:
EXCEL COLUMN1
Column 2
Column3
Column4
Column5
Column6
MM/DD/YYYY
filename1
161179
16789
1875
90783
MM/DD/YYYY
filename2
xxxxxx
xxxxx
xxxx
xxxxx
MM/DD/YYYY
filename3
xxxxxx
xxxxx
xxxx
xxxxx
Is this possible? I can't imagine having to go through each and every file one by one. Please help!!!Hello
You may try the following AppleScript script. It will ask you to choose a root folder where to start searching for *.map files and then create a CSV file named "out.csv" on desktop which you may import to Excel.
set f to (choose folder with prompt "Choose the root folder to start searching")'s POSIX path
if f ends with "/" then set f to f's text 1 thru -2
do shell script "/usr/bin/perl -CSDA -w <<'EOF' - " & f's quoted form & " > ~/Desktop/out.csv
use strict;
use open IN => ':crlf';
chdir $ARGV[0] or die qq($!);
local $/ = qq(\\0);
my @ff = map {chomp; $_} qx(find . -type f -iname '*.map' -print0);
local $/ = qq(\\n);
# CSV spec
# - record separator is CRLF
# - field separator is comma
# - every field is quoted
# - text encoding is UTF-8
local $\\ = qq(\\015\\012); # CRLF
local $, = qq(,); # COMMA
# print column header row
my @dd = ('column 1', 'column 2', 'column 3', 'column 4', 'column 5', 'column 6');
print map { s/\"/\"\"/og; qq(\").$_.qq(\"); } @dd;
# print data row per each file
while (@ff) {
my $f = shift @ff; # file path
if ( ! open(IN, '<', $f) ) {
warn qq(Failed to open $f: $!);
next;
$f =~ s%^.*/%%og; # file name
@dd = ('', $f, '', '', '', '');
while (<IN>) {
chomp;
$dd[0] = \"$2/$1/$3\" if m%Link Time\\s+=\\s+([0-9]{2})/([0-9]{2})/([0-9]{4})%o;
($dd[2] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of CODE\\s/o;
($dd[3] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of DATA\\s/o;
($dd[4] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of XDATA\\s/o;
($dd[5] = $1) =~ s/ //g if m/([0-9 ]+)\\s+bytes of FARCODE\\s/o;
last unless grep { /^$/ } @dd;
close IN;
print map { s/\"/\"\"/og; qq(\").$_.qq(\"); } @dd;
EOF
Hope this may help,
H -
Average Data from multiple text files
I am new to labVIEW hence a little help is appreciated:
I have a 100 txt files with two columns (tab separated) for X and Y value.
I need to average the Y values to generate one single txt file and generate X versus Y graph.
So how do I read the data from these text files? (without having to select each one of them individually) and how to average the data and create a XY graph from it?
Thanks in advance
Solved!
Go to Solution.Use X-Y Graph.
You were able to get two arrays of X and Y points from text file right?
Gaurav k
CLD Certified !!!!!
Do not forget to Mark solution and to give Kudo if problem is solved. -
How to retrieve specific data from a text file
Hi, everyone
For my project it is required that a parameter file is read at the beginning, in order for certain variables to be initialized with specific values that change with the user.
At the moment, the way it is done is the following: The values at a specified sequence in a text file are read and saved in an array and the elements of the array are retrieved according to their index.
The problem with this implementation is, that if for some reason the format of the file changes, e.g. we want to use a parameter file from a previous version of the program that has the values for the same variables but in a different order, the only way to have the right values for the parameters is to change everything accordingly, which is really time wasting.
Could someone suggest a different implementation that would make the reading of the different values independent from their order in the file, e.g. by scanning the file for specific strings and reading the value after the string?
Thank you very much.
P.S. I have attached a screenshot of the routine I am using now.
Solved!
Go to Solution.
Attachments:
read parameter file.JPG 180 KBHi panagiov,
Find attached Folders.
Method 1: in this you can search for each variable separately. You can use "Config file vis" to get all keys(Variables) at once and then you can use for loop to get their values. Or you can access individual values as shown in this code.
Method 2: here you will get all data at once. You will get Variable and Data (2D array) You need to search variables as and when required.
I hope you will understand these methods.
Best of luck
Gaurav k
CLD Certified !!!!!
Do not forget to Mark solution and to give Kudo if problem is solved.
Attachments:
Method 1.zip 7 KB
Method 2.zip 9 KB -
How can I extract the metadata from multiple image files into a .csv?
Hi.
I'm looking for what I believe is a simple task but I am not sure how to go about it. I want to get the relevant data (seen in the screen grab below) into a spread sheet. I need to get the File Name, Size, Dimensions, Resolution and Kind into the spreadsheet so I can better analyze the data for my purposes. I have tried to find EXIF and IPTC extractors but only come across products that show me the info with no way of transferring it. I figured there must be a way to do this in the Finder. Maybe it's a simple Automator task (which I am not very good at) or a Script.
Can anyone guide me thru or suggest a tool that would work?
I was given a suggestion in another Disscussion to use a Plug-In for iPhoto. I found an old one but it dosnt have all of the metadata I need. It's missing the File Name and Resolution as options.
http://www.tc.umn.edu/~erick205/Projects/Metadata%20Export/
Thanks,
StephenNevermind, I found it.
+Inspector > Metrics > File Info: > drag the file's icon to the desktop+
cheers -
Hi,
I have to implement the following scenario in SSIS but don't know how to do since I never worked with SSIS before. Please help me.
I have 20 different text files in a single folder and 20 different tables corresponding to each text file in SQL Server 2008 R2 Database. I need to extract the data from each text file and
load the data into corresponding table in Sql Server Database. Please guide me in how many ways I can do this and which is the best way to implement this job. Actually I have to automate this job. Few files are in same format(with same column names
and datatypes) where others are not.
1. Do I need to create 20 different projects ?
or
Can I implement this in only one project by having 20 packages?
or
Can I do this in one project with only one package?
Thanks in advance.As I said I don't know how to use object data type, I just given a shot as below. I know the following code has errors can you please correct it for me.
Public
Sub Main()
' Add your code here
Dim f1
As FileStream
Dim s1
As StreamReader
Dim date1
As
Object
Dim rline
As
String
Dim Filelist(1)
As
String
Dim FileName
As
String
Dim i
As
Integer
i = 1
date1 =
Filelist(0) =
"XYZ"
Filelist(1) =
"123"
For
Each FileName
In Filelist
f1 = File.OpenRead(FileName)
s1 = File.OpenText(FileName)
rline = s1.ReadLine
While
Not rline
Is
Nothing
If Left(rline, 4) =
"DATE"
Then
date1 (i)= Mid(rline, 7, 8)
i = i + 1
Exit
While
End
If
rline = s1.ReadLine
End
While
Next
Dts.Variables(
"date").Value = date1(1)
Dts.Variables(
"date1").Value = date1(2)
Dts.TaskResult = ScriptResults.Success
End
Sub -
How to extract different date format in text file
well, iam new for using regex api(regular expression), In my project i want to extract the different format of date from the text file... date format will be 3rd june 2004, 03-06-2004, june 3rd and so on....
can any body give me regular expression to extract the date from the text file...
i will be very grateful..
kareemdate format will be 3rd june 2004, 03-06-2004, june 3rd and so on....The only way to do this (without implementing a "mind reader") is to determine in advance all the possible date formats that are possible in your input, and try to interpret each date using each of those formats until one of them passes.
It's easy enough to handle june 3rd vs 3 june vs 3rd june, but 6/3 vs 3/6, of course, is ambiguous. -
Hi i am new to labview
I want to extract data from a text file and display it on the front panel.
How do i proceed??
I have attached a file for your brief idea...
Attachments:
extract.jpg 3797 KBRoopeshV wrote:
Hi,
The below code shows how to read from txt file and display in the perticular fields.
Why have you used waveform?
Regards,
Roopesh
There are so many things wrong with this VI, I'm not even sure where to start.
Hard-coding paths that point to your user folder on the block diagram. What if somebody else tries to run it? They'll get an error. What if somebody tries to run this on Windows 7? They'll get an error. What if somebody tries to run this on a Mac or Linux? They'll get an error.
Not using Read From Spreadsheet File.
Use of local variables to populate an array.
Cannot insert values into an empty array.
What if there's a line missing from the text file? Now your data will not line up. Your case structure does handle this.
Also, how does this answer the poster's question? -
Can I use Visual Basic to covert form user data from multiple .pdf files to a single .csv file? If so, how?
You can automate Acrobat using IAC (InterApplication Communications), as documented in the Acrobat SDK. Your program could loop through a collection of PDFs, load them in Acrobat, extract the form data from each, and generate a CSV file that contains the data.
Acrobat can also do this with its "Merge Data Files into Spreadsheet" function, but this is a manual process. -
Error in importing data from multiple ASCii files, Concatenat​e
I am trying to use the "Importing Data From Multiple ASCII FIles.VBS" and the "Concatenate Groups.VBS" scripts downloaded from here, http://zone.ni.com/devzone/cda/epd/p/id/3870. When I run the importing data script and test it by highlighting the example data that comes with the download I get an error. "Error in <Importing Data From Multiple ASCII Files.VBS> (Line:60, Column: 11): Variabls is undefined: 'AscIIAssocSet' "
The offending line of code is " Call AsciiAssocSet(FilePaths(i), StpFilePath) ' assign STP file "
Screenshots of the error are attached.
Can anyone tell me what this error is? Am I just doing something wrong? Getting quite frustrated.
Thanks in advance.
Attachments:
concactenate1.png 261 KB
concactenate2.png 243 KBPlease have a look at the DIAdem Example to concanate channels. Its delivered with DIAdem.
Examples > Creating Scripts > Scripts > Appending Channels to Each Other
Please use a dataplugin (File -> Text DataPlugin Wizard...) or the csv plugin to load your data.
Greetings
Andreas
P.S.: The one you have downloaded needs the old Ascii Wizard that is not activated by default in newer DIAdem versions.
If you want to use it anyway:
Settings -> Options -> Extensions -> GPI Extensions ...
Add gfsasci.dll -
Adding data from a text file into a JTextArea
I'm working on a blogging program and I need to add data from a text file named messages.txt into a JTextArea named messages. How do I go about doing this?
then you need to be more specific in what you need help with because those are two commonly used options for reading data in from a file. what have you tried? where is your code failing? what kind of errors are you getting? of course it looks like others in the forum have already reprimanded you for failing to post properly.
i have used both FileReader and the nio package for cfg and ini files which are both just txt files as well as massive data files. so tell me how they dont help you? -
How to load Test data from a Text file in ECATT
Hi,
I have created a test configuration with a test script, system data container, and test data container.
I have done the recording of a transaction and created the script. Parameterization is done for the script and have imported those parameters from script in to the data container.
I am trying to load a the data from a text file on the local work-station. The data is not being read.
Please explain this in detail (step by step) as I am very new to ECATT.
I am trying this on SAP ECC 6.0 IDES server.
Thanks in Advance
Vikas PatilPlease explain this in detail (step by step) as I am very new to ECATT.
Thanks in Advance
Vikas Patil -
How do you read data from a text file into a JTextArea?
I'm working on a blogging program and I need to add data from a text file named messages.txt into a JTextArea named messages. How do I go about doing this?
Student_Coder wrote:
1) Read the file messages.txt into a String
2) Initialize messages with the String as the textSwing text components are designed to use Unix-style linefeeds (\n) as line separators. If the text file happens to use a different style, like DOS's carriage-return+linefeed (\r\n), it needs to be converted. The read() method does that, and it saves the info about the line separator style in the Document so the write() method can re-convert it.
lethalwire wrote:
They have 2 different ways of importing documents in this link:
http://java.sun.com/docs/books/tutorial/uiswing/components/editorpane.html
Neither of those methods applies to JTextAreas. -
I would like to plott data from a text file in the same way as a media player does from a video file. I’m not sure how to create the pointer slide function. The vi could look something like the attached jpg.
Please, can some one help me?
Martin
Attachments:
Plotting from a text file like a media player example.jpg 61 KBHI Martin,
i am not realy sure what you want!?!?
i think you want to display only a part of the values you read from XYZ
so what you can do:
write all the values in an array.
the size of the array is the max. value of the slide bar
now you can select a part of the array (e.g. values from 100 to 200) and display this with a graph
the other option is to use the history function of the graphes
regards
timo -
How to parse data from a text file with no convenient delimiters?
I need to read data from a text file. This file contains one line of data with the repeating pattern "time 00 ADVar2: ___ Height: ____ time 01 ADVar2: ___ Height: ___ ..." I need LabView to parse out the "time" and "height" values, build an array with the values, and graph the correlation on an X&Y plot. Does Labview have an automated way to read to the input data file and parse out the correct values, even without convenient delimiters? Thank you.
You actually do have a convenient delimiter: "time". Thus, you can make an array using that as the delimiter. Only caveat is that the first array element will be empty. Then you can conveniently use the Scan From String function in a for-loop. Something like this:
Message Edited by smercurio_fc on 11-21-2008 03:13 PM
Attachments:
Example_VI.png 9 KB
Maybe you are looking for
-
Help with vertical spry menu bar submenu displaying incorrectly!
I'm new to Dreamweaver and am learning as I go. I am having a problem with the vertical spry menu bar submenu in the Live View. When I scroll over the spry menu bar option that has a submenu, the submenu appears in the upper right hand corner of my s
-
Clarification on JMSConnectionFactory
Hi All, I am a newbie to weblogic JMS. I want to know what exactly is a ConnectionFactory that is created within a JMS Module. Should there be one ConnectionFactory created for each JMSModule to access the queues/topics in that JMS Module? I have rea
-
How to shift object slightly in keynote?
How to shift object slightly in keynote? It always jumps to be alighned with other objects.
-
About the MacBook Pro 13'3 Keyboard Backlight
I am a new macbook pro user. about the Keyboard backlight. it is off now. how can i ON it again? it will be on automatically by the sensor??
-
I accidentally deleted the app, which had precious videos on it. Now I don't know where to find the videos, but I want to get them back. How do I do that?