Use of applescript in numbers to automatically analyze data?

Question

Hi all,
I am trying to figure out if it is possible to use applescript and numbers to achieve the following:
(1) Import a CSV file
(2) Work out an average and standard deviation for particular columns of data in that CSV file
(3) Copy and paste out those averages and standard deviations to a separate numbers file, along with the name of the original data file, so in the end I have a file that contains a number of rows, each one containing the results for each data file and the file name.
The files (from an analytical instrument) are all the same, so ideally I could use automator and/or applescript to automatically run through a directory of csv files to achieve this.
I know only a small amount of very basic programming. I don't want someone to write this for me, but if someone could offer some pointers or examples that would be very helpful. Or tell me that what I am trying to do is foolish, and use something else instead.
Thanks

Answer 1

As I got a sample file, I was able to build a script matching the full requirements.
Here is a revised version which may be useful for other users.
--[SCRIPT average&stdev_CSV]
Enregistrer le script en tant que Script : average&stdev_CSV.scpt
déplacer le fichier ainsi créé dans le dossier
<VolumeDeDémarrage>:Users:<votreCompte>:Library:Scripts:Applications:Numbers:
Il vous faudra peut-être créer le dossier Numbers et peut-être même le dossier Applications.
aller au menu Scripts , choisir Numbers puis choisir average&stdev_CSV
Sélectionner un dossier contenant des fichiers CSV.
Le script ouvre ces fichiers dans Numbers,
calcule les valeurs MOYENNE() et ECARTYPE()
écrit celles-ci avec le nom du document dans un fichier texte.
Après traitement de tous les fichiers le fichier texte et ouvert dans Numbers.
On peut également enregistrer le script en tant que progicile (application sou 10.6.x).
Il suffira alors de glisser/déposer l'icone du dossier à traiter sur celle du script_application.
--=====
L'aide du Finder explique:
L'Utilitaire AppleScript permet d'activer le Menu des scripts :
Ouvrez l'Utilitaire AppleScript situé dans le dossier Applications/AppleScript.
Cochez la case "Afficher le menu des scripts dans la barre de menus".
--=====
Save the script as a Script: average&stdev_CSV.scpt
Move the newly created file into the folder:
<startup Volume>:Users:<yourAccount>:Library:Scripts:Applications:Numbers:
Maybe you would have to create the folder Numbers and even the folder Applications by yourself.
go to the Scripts Menu, choose Numbers, then choose "average&stdev_CSV"
Select a folder containing CSV files.
The script open them in Numbers,
calculate the AVERAGE() and STDEV() values
write them with the file name in a text file.
When all files are treated, the text file is opened in Numbers.
You may also save the script as an application package (application under 10.6.x).
Then drag & drop the icon of the folder to treat on the scipt_application icon.
--=====
The Finder's Help explains:
To make the Script menu appear:
Open the AppleScript utility located in Applications/AppleScript.
Select the "Show Script Menu in menu bar" checkbox.
--=====
Yvan KOENIG (VALLAURIS, France)
2010/06/10
2010/06/11 -- now read the csv file only once
--=====
property permis : {"public.comma-separated-values-text", "public.csv"}
property AVERAGE_loc : missing value
property STDEV_loc : missing value
property delim : missing value
property deci : missing value
property altDelim : missing value
property altDeci : missing value
property dossier_temporaire : missing value
property mon_Rapport : missing value
--=====
Entry point used when we double click the script's icon
or when we call it from the Scripts menu .
on run
if my parleAnglais() then
set le_prompt to "Choose a folder containing CSV file …"
else
set le_prompt to "Choisir un dossier contenant des fichiers CSV …"
end if -- parleAnglais
my main(choose folder with prompt le_prompt without invisibles)
end run
--=====
Entry point used when we drag & drop a folder icon on the application_script's icon
on open sel
my main(sel)
end open
--=====
on main(selected_item) -- it's an alias
local isFolder, nomdurapport, les_fichiers, type_ID, un_fichier, nombrededocuments, nomdutableur
the selected folder's pathname
set selected_item to selected_item as text
tell application "System Events"
set isFolder to class of disk item selected_item is folder
end tell -- System Events
if not isFolder then
if my parleAnglais() then
error "You must select a folder containing CSV file …"
else
error "Vous devez choisir un dossier contenant des fichiers CSV …"
end if -- parleAnglais
end if -- not isFolder
Init once variables which will be used later.
set dossier_temporaire to (path to temporary items) as text (* property *)
set AVERAGE_loc to my getLocalizedFunctionName("Numbers", "AVERAGE") & "(" (* property *)
set STDEV_loc to my getLocalizedFunctionName("Numbers", "STDEV") & "(" (* property *)
set {delim, deci, altDelim, altDeci} to my getLocalized_Delimiters() (* properties *)
set nomdurapport to "resume" & my dateTimeStamp() & ".txt" (* locale *)
set mon_Rapport to dossier_temporaire & nomdurapport (* property *)
Extract the list of items stored in the selected folder.
Create the resumeyyyymmddhhmmss.txt temporary file.
tell application "System Events"
set les_fichiers to every disk item of folder selected_item
make new file at end of folder dossier_temporaire with properties {name:nomdurapport}
end tell
Scan the items stored in the folder.
Skip the folders and documents which aren't csv ones.
repeat with un_fichier in les_fichiers
tell application "System Events"
if class of un_fichier is folder then
set type_ID to "Oops, I‘m a folder" (* so it will not be deciphered *)
else
set type_ID to type identifier of un_fichier
set un_fichier to path of un_fichier
end if
end tell -- System Events
if type_ID is in my permis then my traiteun_fichier(unfichier)
end repeat -- with un_fichier
The scan is done, open the resumeyyyymmddhhmmss.txt temporary file in Numbers.
tell application "Numbers"
set nombrededocuments to count of documents
open mon_Rapport
repeat while (count of documents) = nombrededocuments
delay 0.2
end repeat
set nomdutableur to name of document 1
Set cells format to the Scientific one but I can't define the number of decimals !
tell document 1 to tell sheet 1 to tell table 1
set format of range ("B1 : " & name of last cell) to scientific
end tell
save document 1 in (selected_item & nomdutableur)
end tell -- Numbers
Delete the temporary text file
tell application "System Events" to delete disk item mon_Rapport
Reset properties so there content will not be stored in the script
set AVERAGE_loc to missing value
set STDEV_loc to missing value
set delim to missing value
set altDelim to missing value
set deci to missing value
set altDeci to missing value
set dossier_temporaire to missing value
set mon_Rapport to missing value
end main
--=====
on traiteunfichier(unFichier) (* text item *)
local dossierdetravail, nomdu_csvtemporaire, csv_temporaire, utile, cnt, part2
local |dernière|, nombreDeDocuments, nomdutableur, laPlage
tell application "System Events"
tell disk item unFichier
set dossierdetravail to path of container -- of disk item unFichier
set nomdu_csvtemporaire to name -- of disk item unFichier
end tell -- unFichier
if nomdu_csvtemporaire does not end with ".csv" then set nomdu_csvtemporaire to nomdu_csvtemporaire & ".csv"
set csv_temporaire to (dossier_temporaire & nomdu_csvtemporaire)
if exists disk item csv_temporaire then delete disk item csv_temporaire
make new file at end of folder dossier_temporaire with properties {name:nomdu_csvtemporaire}
end tell -- System Events
Read the entire file
set utile to (read file unFichier)
Check that it's matching our requirements. I'm not sure that the second test is always valid.
if (utile contains "#=-=-=-=-=-=-=-=-=") and utile contains "#=-=-=-=-=-=-=-=-=End ./conf/ccia" then
set cnt to offset of "#=-=-=-=-=-=-=-=-=" in utile
Extract the second part which isn't formatted as the first one.
set part2 to text cnt thru -1 of utile
Extract the first part to decipher it.
set utile to (text 1 thru (cnt - 1) of utile) as text
Grabs the index of lower row
set |dernière| to count of (paragraphs of utile)
An ultimate test to be sure that the file is for us.
if (|dernière| > 0) and text -13 thru -2 of (paragraph 6 of utile) is "NFitsAvg'd" then
Normalize the datas according to the local settings
if not ((utile contains delim) and utile contains deci) then
if utile contains altDelim then set utile to my remplace(utile, altDelim, delim)
if utile contains altDeci then set utile to my remplace(utile, altDeci, deci)
end if
Write the normalized beginning and the untouched part2 in a temporary csv file.
write utile to file csv_temporaire
write part2 to file csv_temporaire starting at eof
Open the temporary csv in Numbers
tell application "Numbers"
set nombreDeDocuments to count of documents
open csv_temporaire
repeat while (count of documents) = nombreDeDocuments
delay 0.2
end repeat
set nomdutableur to name of document 1
tell document 1 to tell sheet 1 to tell table 1
Insert a row at top to insert the required formulas.
add row above first row
Insert the formulas calculating the AVERAGEs and the STDEVs .
repeat with c from 2 to 16 by 2
set laPlage to (name of cell 8 of column c) & " : " & name of cell |dernière| of column c
set value of cell 1 of column c to "=" & AVERAGE_loc & laPlage & ")"
set value of cell 1 of column (c + 1) to "=" & STDEV_loc & laPlage & ")"
end repeat -- with c
Extract the calculated values to build a new row in the resume.
set une_ligne to {nomdu_csvtemporaire}
tell row 1
repeat with c from 2 to 17
copy (value of cell c) as text to end of une_ligne
--copy (value of cell c) to end of une_ligne
end repeat
end tell -- row 1
end tell -- document
Save the new Numbers document.
I don't know if it is useful but it's easy to remove or disable the instruction.
save document 1 in (dossierdetravail & nomdutableur)
close document 1 saving no (* So, if you disable the Save instruction, it will close quietly *)
end tell -- Numbers
Write the new row in the resume text file.
write (my recolle(une_ligne, tab) & return) to file mon_Rapport starting at eof
end if -- (|dernière| > 0) or
end if -- read file unFichier…
Delete the temporary csv file
tell application "System Events" to delete file csv_temporaire
end traiteunfichier
--=====
Creates a new iWork document from the Blank template and returns its name.
example:
set myNewDoc to my makeAnIworkDoc(theApp)
on makeAnIworkDoc(theApp)
local t, n
if theApp is "Pages" then
set commun to "iWork '" & my get_iWorkNum("Pages") & ":Pages.app:Contents:Resources:Templates:Blank.template:"
try
set t to ((path to applications folder as text) & commun) as alias
on error
set t to ("Western 2:Applications communes:iWork '" & commun) as alias
end try
else if theApp is "Numbers" then
set commun to "iWork '" & my get_iWorkNum("Numbers") & ":Numbers.app:Contents:Resources:Templates:Blank.nmbtemplate:"
try
set t to ((path to applications folder as text) & commun) as alias
on error
set t to ("Western 2:Applications communes:" & commun) as alias
end try
else
if my parleAnglais(theApp) then
error "The application “" & a & "“ is not accepted !"
else
error "l’application « " & a & " » n’est pas gérée !"
end if
end if
tell application theApp
set n to count of documents
open t
repeat until (count of documents) > n
delay 0.1
end repeat
set n to name of document 1
end tell -- theApp
return n
end makeAnIworkDoc
--=====
on dateTimeStamp()
return (do shell script "date +_%Y%m%d-%H%M%S")
end dateTimeStamp
--=====
Set the parameter delimiters which must be used in Numbers formulas
on getLocalized_Delimiters()
if character 2 of (0.5 as text) is "." then
return {",", ".", ";", ","}
else
return {";", ",", ",", "."}
end if
end getLocalized_Delimiters
--=====
on get_iWorkNum(a)
local verNum
tell application a to set verNum to item 1 of my decoupe(get version, ".")
if (a is "Numbers" and verNum is "2") or (a is "Pages" and verNum is "4") then
return "09"
else
return "11"
end if
end get_iWorkNum
--=====
Useful to get function's localized name if we need to build formulas
examples:
set OFFSET_loc to my getLocalizedFunctionName("Numbers", "OFFSET")
set ADDRESS_loc to my getLocalizedFunctionName(theApp, "ADDRESS")
set INDIRECT_loc to my getLocalizedFunctionName(theApp, "INDIRECT")
on getLocalizedFunctionName(theApp, x)
return my getLocalizedName(theApp, x, (path to application support as text) & "iWork '" & ¬
my get_iWorkNum(theApp) & ":Frameworks:SFTabular.framework:Versions:A:Resources:")
end getLocalizedFunctionName
--=====
on getLocalizedName(a, x, f)
tell application a to return localized string x from table "Localizable" in bundle file f
end getLocalizedName
--=====
on parleAnglais()
local z
try
tell application "Numbers" to set z to localized string "Cancel"
on error
set z to "Cancel"
end try
return (z is not "Annuler")
end parleAnglais
--=====
on decoupe(t, d)
local TIDs, l
set TIDs to AppleScript's text item delimiters
set AppleScript's text item delimiters to d
set l to text items of t
set AppleScript's text item delimiters to TIDs
return l
end decoupe
--=====
on recolle(l, d)
local TIDs, t
set TIDs to AppleScript's text item delimiters
set AppleScript's text item delimiters to d
set t to l as text
set AppleScript's text item delimiters to TIDs
return t
end recolle
--=====
replaces every occurences of d1 by d2 in the text t
on remplace(t, d1, d2)
local TIDs, l
set TIDs to AppleScript's text item delimiters
set AppleScript's text item delimiters to d1
set l to text items of t
set AppleScript's text item delimiters to d2
set t to l as text
set AppleScript's text item delimiters to TIDs
return t
end remplace
--=====
--[/SCRIPT]
Yvan KOENIG (VALLAURIS, France) vendredi 11 juin 2010 19:01:46