Custom Styles in TextEdit - Background Highlighting

Question

Hi, I defined a custom/'favorite' style for use in TextEdit which applies yellow background highlighting to selected text. (The style was added to the GlobalPreferences.plist file, as I'm on Snow Leopard). The issue is that when I highlight text and apply this style, it overrides any prior formatting like bold, italic, or underline, and reverts the selection to plain text when applying the background highlighting.
I understand this is mechanically correct, because my user-defined Highlight style induces [plain text + yellow background color]. My question is this: is there any way to modify my defined style to preserve the original formatting of the text, adding only the background color without modifying the pre-existing text attributes? Basically I would like my bold, italic, underline, strikethroughs, font size, etc. to be preserved when applying a background color to selected text.
For reference, I defined the style by electing to 'add to favorite styles' a selection from one of the similarly pre-formatted rich text files that's available, and did NOT include the font or the ruler as part of the style. I understand that I could create separate favorite styles for highlight+bold, highlight+italic, highlight+underline, highlight+strikethrough, highlight+[each font size], and all combinations thereof, but applying each different style to each subsection of text would be as cumbersome as reformatting each subsection as +bold, +italics, etc. after the highlight it applied and text style is reverted to default.
I have extracted the NSFavoriteStyles section from the ~/Library/Preferences/GlobalPreferences.plist file and located the style I defined, it appears like this:
Highlighter =     {
        NSBackgroundColor = <62706c69 73743030 d4010203 04050615 16582476 65727369 6f6e5824 6f626a65 63747359 24617263 68697665 72542474 6f701200 0186a0a3 07080f55 246e756c 6cd3090a 0b0c0d0e 5624636c 6173735c 4e53436f 6c6f7253 70616365 554e5352 47428002 10014831 20312030 2e3200d2 10111213 5a24636c 6173736e 616d6558 24636c61 73736573 574e5343 6f6c6f72 a2121458 4e534f62 6a656374 5f100f4e 534b6579 65644172 63686976 6572d117 1854726f 6f748001 08111a23 2d32373b 41484f5c 6264666f 747f8890 939caeb1 b6000000 00000001 01000000 00000000 19000000 00000000 00000000 00000000 b8>;
What would I need to add to this string to retain existing text formatting when applying the style? It should be possible, as applying the 'Underline' style does not override an existing 'Bold' style, for example, but I don't see any commands in those styles that supplement the information that appears in my defined 'Highlighter' style.
I also understand this may be pre-Lion specific, as I've heard the TextEdit program is now sandboxed in such a way as to prevent the definition of new styles. I'll have to try on my Mountain Lion machine when I have a chance. I appreciate any help, thanks!

Answer 1

Hello
It's been a while and I'm not sure you're still listenting to or interested in this.
Anyway I finally had some time to play with and come up with an implementation via system services.
There are three files to make the service, whose codes are listed below.
• main.m (wrapper executable)
• main.rb (service body)
• make (bash script to make .service from source)
# Recipe
1) Create the three plain text files (in utf8) by copy-pasting codes listed below and place them loose in a new directory, e.g., ~/Desktop/work.
2) Issue the following commands in Terminal, which yield a package named TextHighlightServices.service in ~/Desktop/work.
* You need to have gcc installed, which comes with Xcode.
cd ~/Desktop/work
chmod ug+x make
./make
3) Move the TextHighlightServices.service to ~/Library/Serivecs
4) Log out and log in to let the new service be recognised.
(This may not be necessary for .service, but I'd play for safety. Also you may use the following command in Terminal instead of re-login, if necessary:
# in 10.5
/System/Library/CoreServices/pbs
# in 10.6
/System/Library/CoreServices/pbs -flush
5) Done.
# Usage
If everything goes well, you'll see two services named "Text Highlight - Add" and "Text Highlight - Remove" in Services menu in TextEdit when you select some text in rich text mode. (In 10.6, services only appear in its applicable context, while in 10.5, they may yet appear in non-applicable context and be grayed-out.)
Invoke the serivces on selected rich text via menu or keyboard shortcuts and see if it works. Keyboard shortcuts are currently defined as Command-9 for highlight and Command-0 for unhighlight.
You may change the menu item names, keyboard shortcuts and highlight colour by changing the values in the #properties section at the beginning of make file. (Or you may directly edit Info.plist file in the package. But 'make' is instant and it'll be faster to re-make the package than to edit it manually)
# Notes
• Highlight and unhighlight are implemented as two services and you cannot assign one keyboard shortcut, e.g., Command-Y to both to toggle highlight state. (One 'toggle highlight service' is possible but you need to re-write the code as such)
• Keyboard shortcut is case-sensitive. E.g., J means Command-Shift-j.
• If keyboard shortcut is ignored, it is most likely that the key is already used for something else.
• This service is written to terminate itself if idle for ca. 20 seconds. It will be re-launched on demand. To keep the service running is not a problem, usually, but I noticed this rubycocoa code constantly uses ca 1.1% CPU and decided not to let it waste energy. If you rewrite the code in Objective-C proper, it would be 0.0% CPU usage in idle.
• Tested with 10.5.8 and 10.6.5.
# Files
main.m
// file
//     main.m
// function
//     wrapper executable to run Contents/Resources/main.rb
// compile
//     gcc -framework RubyCocoa -o PROGNAME main.m
#import <RubyCocoa/RBRuntime.h>
int main(int argc, const char *argv[])
    return RBApplicationMain("main.rb", argc, argv);
main.rb
# file
#     main.rb
# function
#     to provide text highlight services
require 'osx/cocoa'
include OSX
class Services < NSObject
    attr_reader :last_invoked
    def init()
        @last_invoked = NSDate.date
        self
    end
    # utility method
    def highlight_pboard_colour(option, pboard, colr)
        # int option : 0 = remove bg colour, 1 = add bg colour
        # NSPastedBoard *pboard : pasteboard passed via service
        # NSColor colr : background colour for highlight
        raise ArgumentError, "invalid option: #{option}" unless [0,1].include? option
        @last_invoked = NSDate.date
        # read rtf data from clipboard
        rtf = pboard.dataForType('public.rtf')
        # make mutable attributed string from rtf data
        docattr = OCObject.new
        mas = NSMutableAttributedString.alloc.objc_send(
            :initWithRTF, rtf,
            :documentAttributes, docattr)
        rase ArgumentError, "zero-length rtf is given" if mas.length == 0
        # add or remove background colour attribute
        r = NSMakeRange(0, mas.length)
        if option == 1
            mas.objc_send(
                :addAttribute, NSBackgroundColorAttributeName,
                :value, colr,
                :range, r)
        elsif option == 0
            mas.objc_send(
                :removeAttribute, NSBackgroundColorAttributeName,
                :range, r)
        end
        mas.fixAttributesInRange(r)
        # make rtf data from mutable attributed string
        rtf = mas.objc_send(
            :RTFFromRange, r,
            :documentAttributes, docattr)
        # write rtf data to the clipboard
        pboard.objc_send(
            :declareTypes, ['public.rtf'],
            :owner, nil)
        pboard.objc_send(
            :setData, rtf,
            :forType, 'public.rtf')
    end
    # service method 1 : highlight rtf
    def highlight_userData_error(pboard, udata, error)
        # NSPastedBoard *pboard : pasteboard passed via service
        # NSString *udata : space delimited string of red, green, blue, alpha components to define background colour
        # NSString **error
        # set background colour from given user data
        r, g, b, a = udata.to_s.split(/\s+/).map {|x| x.to_f}
        colr = NSColor.objc_send(
            :colorWithCalibratedRed, r,
            :green, g,
            :blue, b,
            :alpha, a)
        # add background colour to rtf
        self.highlight_pboard_colour(1, pboard, colr)
    end
    # register ruby method as objc method
    objc_method(:highlight_userData_error, 'v@:@@^@')
    # service method 2 : un-highlight rtf
    def unhighlight_userData_error(pboard, udata, error)
        # NSPastedBoard *pboard : pasteboard passed via service
        # NSString *udata : space delimited string of red, green, blue, alpha components to define background colour
        # NSString **error
        self.highlight_pboard_colour(0, pboard, nil)
    end
    # register ruby method as objc method
    objc_method(:unhighlight_userData_error, 'v@:@@^@')
end       
class AppDelegate < NSObject
    def init()
        @services = Services.alloc.init
        @timer = NSTimer.objc_send(
            :timerWithTimeInterval, 5.0,  # periodic check interval [sec]
            :target, self,
            :selector, :idle,
            :userInfo, nil,
            :repeats, true)
        @TTL = 20.0  # time to live in idle [sec]
        self
    end
    def applicationDidFinishLaunching(notif)
        NSApp.setServicesProvider(@services)
        NSRunLoop.currentRunLoop.objc_send(
            :addTimer, @timer,
            :forMode, NSDefaultRunLoopMode)
    end
    def applicationShouldTerminate(sender)
        @timer.invalidate
        NSTerminateNow
    end
    def idle(timer)
        if @services.last_invoked.timeIntervalSinceNow < -@TTL
            NSApp.terminate(self)
        end
    end
end
def main
    app = NSApplication.sharedApplication
    delegate = AppDelegate.alloc.init
    app.setDelegate(delegate)
    app.run
end
if __FILE__ == $PROGRAM_NAME    # = if this file is the primary ruby program currently executed
    main
end
make
#!/bin/bash
# file
#    make
# function
#    to make PROGNAME.service package from main.m and main.rb
export LC_ALL=en_GB.UTF-8
# properties
PROGNAME="TextHighlightServices"
EXECPATH="${PROGNAME}.service/Contents/MacOS/${PROGNAME}"
BNDL_TYPE="APPL"
BNDL_SIGNATURE="????"
BNDL_VERSION="1.0"
BUILD_VERSION="1"
SRC_VERSION="10000"
HIGHLIGHT_MENU_ITEM="Text Highlight - Add"
HIGHLIGHT_KEY="9"
UNHIGHLIGHT_MENU_ITEM="Text Highlight - Remove"
UNHIGHLIGHT_KEY="0"
HIGHLIGHT_COLOUR="1.0 1.0 0.6 1.0" # R B G A in [0.0, 1.0]
# make package directories
rm -rf "${PROGNAME}".service
mkdir -p "${PROGNAME}".service/Contents/{MacOS,Resources}
# make PkgInfo
cat <<EOF > "${PROGNAME}.service/Contents/PkgInfo"
${BNDL_TYPE}${BNDL_SIGNATURE}
EOF
# make Info.plist
cat <<EOF > "${PROGNAME}".service/Contents/Info.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>CFBundleExecutable</key>
    <string>${PROGNAME}</string>
    <key>CFBundleIdentifier</key>
    <string>bubo-bubo.${PROGNAME}</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>${PROGNAME}</string>
    <key>CFBundlePackageType</key>
    <string>${BNDL_TYPE}</string>
    <key>CFBundleShortVersionString</key>
    <string>${BNDL_VERSION}</string>
    <key>CFBundleSignature</key>
    <string>${BNDL_SIGNATURE}</string>
    <key>CFBundleVersion</key>
    <string>${BNDL_VERSION}</string>
    <key>NSPrincipalClass</key>
    <string>NSApplication</string>
    <key>NSServices</key>
    <array>
        <dict>
            <key>NSPortName</key>
            <string>${PROGNAME}</string>
            <key>NSMessage</key>
            <string>highlight</string>
            <key>NSMenuItem</key>
            <dict>
                <key>default</key>
                <string>${HIGHLIGHT_MENU_ITEM}</string>
            </dict>
            <key>NSKeyEquivalent</key>
            <dict>
                <key>default</key>
                <string>${HIGHLIGHT_KEY}</string>
            </dict>
            <key>NSSendTypes</key>
            <array>
                <string>NSRTFPboardType</string>
            </array>
            <key>NSReturnTypes</key>
            <array>
                <string>NSRTFPboardType</string>
            </array>
            <key>NSTimeout</key>
            <string>10000</string>
            <key>NSUserData</key>
            <string>${HIGHLIGHT_COLOUR}</string>
            <key>NSRequiredContext</key>
            <dict>
                <key>NSApplicationIdentifier</key>
                <array>
                    <string>com.apple.TextEdit</string>
                </array>
            </dict>
        </dict>
        <dict>
            <key>NSPortName</key>
            <string>${PROGNAME}</string>
            <key>NSMessage</key>
            <string>unhighlight</string>
            <key>NSMenuItem</key>
            <dict>
                <key>default</key>
                <string>${UNHIGHLIGHT_MENU_ITEM}</string>
            </dict>
            <key>NSKeyEquivalent</key>
            <dict>
                <key>default</key>
                <string>${UNHIGHLIGHT_KEY}</string>
            </dict>
            <key>NSSendTypes</key>
            <array>
                <string>NSRTFPboardType</string>
            </array>
            <key>NSReturnTypes</key>
            <array>
                <string>NSRTFPboardType</string>
            </array>
            <key>NSTimeout</key>
            <string>10000</string>
            <key>NSRequiredContext</key>
            <dict>
                <key>NSApplicationIdentifier</key>
                <array>
                    <string>com.apple.TextEdit</string>
                </array>
            </dict>
        </dict>
    </array>
    <key>NSUIElement</key>
    <string>1</string>
</dict>
</plist>
EOF
# make version.plist
cat <<EOF > "${PROGNAME}".service/Contents/version.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>BuildVersion</key>
    <string>${BUILD_VERSION}</string>
    <key>CFBundleShortVersionString</key>
    <string>${BNDL_VERSION}</string>
    <key>CFBundleVersion</key>
    <string>${BNDL_VERSION}</string>
    <key>ProjectName</key>
    <string>${PROGNAME}</string>
    <key>SourceVersion</key>
    <string>${SRC_VERSION}</string>
</dict>
</plist>
EOF
# make ServicesMenu.strings
mkdir -p "${PROGNAME}".service/Contents/Resources/English.lproj
cat <<EOF > "${PROGNAME}".service/Contents/Resources/English.lproj/ServicesMenu.strings
/* Services menu item to highlight or unhighlight the selected text */
"${HIGHLIGHT_MENU_ITEM}" = "${HIGHLIGHT_MENU_ITEM}";
"${UNHIGHLIGHT_MENU_ITEM}" = "${UNHIGHLIGHT_MENU_ITEM}";
EOF
# copy *.rb in Contents/Resoures
ditto *.rb "${PROGNAME}".service/Contents/Resources/
# compile wrapper executable
gcc -framework RubyCocoa -o "${EXECPATH}" main.m
It is fun to play with rubycocoa.
And glad if this helps.
Good luck,
H
cf.
http://developer.apple.com/DOCUMENTATION/Cocoa/Conceptual/SysServices/SysService s.pdf
Message was edited by: Hiroto ( fixed 'make' to './make' in Recipe 2, sorry)