AppleScript speed difference between 10.4 & 10.5

Question

This post is related to a question I posed in March 2009 (https://discussions.apple.com/message/9226100) regarding the speed difference between AppleScript text handling in OS X 10.5 compared with 10.4.11.
I have recently written a script to search through a text file which is an xml file exported from WorkGroup Manager. The file, which is approximately 1.8 MB, contains all the user names on the school server (just over 1720 of them), their settings and any other information we have added. After adapting some script ideas (using text delimiters and offsets) which I found on the internet to assist with counting the number of users and searching through the information of each one, one at a time, I have a script which works far in excess of my expectations in terms of speed when run under OS X 10.4, but runs slower than molasses in winter under 10.5.
The script counts the number of accounts, extracts each user's log-in name and also their actual name (which we have entered under a 'general info' or 'comments' field - I can't remember exactly where, but it is stored in the file), makes up a complete list of all account names (with the actual name, if present) and also makes up a subset of this information into a separate list which contains only the log-in names of those whose actual name has not yet been recorded on the server, then saves both lists to separate text files.
My original attempts, using some rather clumsy text searching and comparison techniques, took about 1-2 minutes to go through the whole procedure. After some internet searching, head-scratching and a bit more more work, the finished script, when run on my G5 under OS X 10.4.11, takes less than 3 seconds to do all I have described above, including the writing to text files. To say I was pleased would be an understatement!
Because the school/work environment, where the script would be run, is all OS X 10.5 or higher, I thought I should test it out under at least OS X 10.5. The script was developed on my DP 2.5GHz G5 with 8GB RAM running OS X 10.4.11. Using a fresh, clean installation of 10.5 I ran the script and was extremely disappointed. After letting it run for more than 20 minutes and still not finishing, I force quit AppleScript and reduced the number of accounts in the file. 40 accounts took 3 seconds, 60 accounts took 6 seconds, 80 took 11 seconds, 100 took 19 seconds and 120 took 27 seconds. I eventually let the script run right through - total time taken to process 1720 users was 65 minutes, compared to 3 seconds under 10.4.11!
Given that it was tested on the same Mac, with the same amount of RAM, using the same script and the same original text file, the only variable left (that I can think of) is the change of OS version - 10.4.11 vs 10.5.
As was pointed out to me in my previous post, AppleScript in OS X 10.5 handles all text as Unicode and has a greater overhead in processing time as a result. I have implemented the various bits of advice offered by respondents to my original post and changed the way I handled lists, etc. Obviously, if the script works so speedily under 10.4.11, I must have done something right in terms of code optimization/efficiency. It's when that exact same script is run under OS X 10.5 that it slows down incredibly. I even tried it on a 2.66GHz Intel Core 2 Duo iMac running 10.5.8 - it still took 40 minutes and Activity Monitor showed CPU usage by Script Editor constantly above 80% (often well above 90%). Saving the script as an application made no difference in the time taken.
Does anybody have any knowledge about what makes AppleScript Unicode-only text handling so slow? In this case, it is 1300 times slower! Is there any way of coercing/restricting the text handling/parsing to ASCII? If I really have to, I will set up a humble eMac at school with 10.4 on it just to handle large text files quickly with AppleScript, but I would prefer to be able to do it on the normal work Macs which have 10.5 or later on them. As mentioned, under 10.4.11 the script processes the 1.8MB text file with 1720 users and writes results to two files - all within 3 seconds, so I'm not really looking for coding suggestions unless they are directly related to what has changed in AppleScript under OS X 10.5. Without sounding too smug, the script works properly and speedily (at least in 10.4). I really would like to learn about the changes in AppleScript in OS X 10.5 and how to cope with or work around those changes.

Answer 1

Hi,
text 3 thru 14 in largeText
character 3 thru 14 in largeText
text 3  in largeText
character 3  in largeText
These lines will be slow on Leopard
Getting some text in a variable that contains more than 60000 characters will be slower on Leopard,
but I don't know why it's slower
Here a test script.
script o
    property t_text : ""
end script
set a to "abcdefghij"
set tResult to ""
repeat with i from 1 to 3
    set o's t_text to a
    repeat (item i of {17, 15, 14}) times
        set o's t_text to o's t_text & o's t_text -- add  characters in the variable
    end repeat
    set StartTime to current date
    -- test
    repeat 20 times
        set b to text 3 thru 14 in o's t_text -- get a text in the variable
    end repeat
    -- end test
    set EndTime to current date
    set TimeTaken to EndTime - StartTime
    set tResult to tResult & " Getting text in the variable which contains " & length of o's t_text & ", (20 times) = " & TimeTaken & " seconds." & return
end repeat
tResult
Here the result on my old G5, 2 x 1.8 GHZ
Getting text in the variable which contains 1310720 characters, (20 times)  = 50 seconds.
Getting text in the variable which contains   327680 characters, (20 times)  = 12 seconds.
Getting text in the variable which contains   163840 characters, (20 times)  =   6 seconds.
The result on the same machine on Tiger is always less of one second.
Also, I try with 20 millions characters on Tiger, the result : getting text in the variable which contains 20971520 characters, (20 times)  = 0 seconds.
The solution ( text item in a list)
Here the script
script o
    property my_List : {}
end script
set OldDelims to text item delimiters
set RecordDelimiter to "::::::::::::::"
set LengthOfRecordDelimiter to length of RecordDelimiter
set o's my_List to findAll(read (choose file), RecordDelimiter)
on findAll(str, findString)
    set Oldtid to text item delimiters
    try
        set text item delimiters to findString
        if str does not contain findString then return {"Nothing found"}
        set t to str's text items
        set text item delimiters to Oldtid
        return t
    on error eMsg number eNum
        set text item delimiters to Oldtid
        error "Can't findAll: " & eMsg number eNum
    end try
end findAll
set NumberOfrecords to (count o's my_List)
display dialog "There are " & NumberOfrecords & " accounts."
set StartTime to current date
set text item delimiters to ":"
set FullUserList to {}
set ListOfUnnamedUsers to {}
-- first user needs to be done separately as it is not preceded by RecordDelimiter
set EndOfHeader to "Standard:URL"
set LengthOfEndOfHeader to length of EndOfHeader
set EndOfHeaderOffset to the offset of EndOfHeader in (item 1 of o's my_List)
set OffsetToApply to EndOfHeaderOffset
set TextBeingChecked to text (OffsetToApply + LengthOfEndOfHeader + 1) thru -1 of (item 1 of o's my_List)
tell TextBeingChecked to set {UserName, NameForInfo} to {text item 1, text item 7}
if NameForInfo = "" then set end of ListOfUnnamedUsers to UserName & return
set end of FullUserList to (UserName & tab & NameForInfo & return)
-- now do all the others
repeat with CounterG from 2 to (NumberOfrecords - 1)
    set TextBeingChecked to item CounterG of o's my_List
    tell TextBeingChecked to set {UserName, NameForInfo} to {text item 1, text item 7}
    if NameForInfo = "" then set end of ListOfUnnamedUsers to UserName & return
    set end of FullUserList to (UserName & tab & NameForInfo & return)
end repeat
set o's my_List to {}
set text item delimiters to OldDelims
-- write results to file
-- 1). full user list
set TargetFile1 to (path to desktop folder as string) & "FullUserList1.txt"
try
    open for access file TargetFile1 with write permission
on error
    close access file TargetFile1
    open for access file TargetFile1 with write permission
end try
set EndTime to current date
beep
set InfoToBeWrittenToFile to FullUserList as text
write InfoToBeWrittenToFile to file TargetFile1
close access file TargetFile1
-- 2). list of users without NameForInfo
set TargetFile2 to (path to desktop folder as string) & "UnnamedUsersList1.txt"
try
    open for access file TargetFile2 with write permission
on error
    close access file TargetFile2
    open for access file TargetFile2 with write permission
end try
set InfoToBeWrittenToFile to ListOfUnnamedUsers as text
write InfoToBeWrittenToFile to file TargetFile2
close access file TargetFile2
beep 3
set TimeTaken to EndTime - StartTime
set TimePerAccount to TimeTaken / NumberOfrecords
display dialog (NumberOfrecords & " accounts took " & TimeTaken & " seconds." & return & return & "That equals " & TimePerAccount & " seconds per account.") as string