Functions for multi-phase catalog access, with retries...

Question

See Catalog:action & Catalog:privateAction below.
These functions support iterative catalog access, to divide catalog updates into chunks...
Also, they support retries to deal with contention in case another plugin task is already accessing the catalog.
-- private method:
function Catalog:_isAccessContention( qual )
    local found = qual:find( "LrCatalog:with", 1, true ) or 0 -- return position or zero, instead of position or nil.
    if found == 1 then -- problem reported by with-catalog-do method.
        local found2 = qual:find( "already inside", 15, true )
        if found2 == nil then
            found2 = qual:find( "was blocked", 15, true ) -- Lr4b
        end
        if found2 then
            -- problem is due to catalog access contention.
            Debug.logn( 'catalog contention:', str:to( qual ) )
            return true
        else
            return false
        end
    else
        return false
    end
end
-- private method:
function Catalog:_action( catFunc, tmo, name, func, ... )
    local t = { ... }
    local tries = math.max( math.ceil( tmo * 2 ), 1 )
    local sts, msg
    local function _func( context )
        sts, msg = func( unpack( t ) )
    end
    local count = 10000000 -- ten million max, for sanity.
    repeat
        local s, other
        if name then
            s, other = LrTasks.pcall( catFunc, catalog, name, _func )
        else
            s, other = LrTasks.pcall( catFunc, catalog, _func )
        end
        if s then -- no error thrown
            if sts then
                return true -- all done.
            elseif str:is( msg ) then
                return false, "Unable to complete catalog update - " .. msg
            else
                -- continue
                count = count - 1                       
            end
        else
            if self:_isAccessContention( other ) then
                tries = tries - 1
                if tries == 0 then
                    return nil, "Unable to access catalog for " .. str:to( tmo ) .. " seconds."
                else
                    LrTasks.sleep( math.random( .1, 1 ) ) -- sleep for half second, plus or minus.
                end
            else
                return nil, "Catalog access function error: " .. str:to( other )
            end
        end
    until count == 0
    return nil, "Program failure"
end
--- Wrapper for named/undoable catalog:withWriteAccessDo method - divide to conquor func.
--  @param tmo (number) max seconds to get in.
--  @param name (string) undo title.
--  @param func (function) divided catalog writing function: returns sts, msg = true when done; false if to be continued; nil, error message if trouble.
--  @param ... (any) passed to func - often a table containing photo start index.
--  @usage example:<br>
--           local function catalogAction( t )<br>
--               if t.i > #photos then<br>
--                   return true -- done, no errors (although should pre-check for photos to process).<br>
--               else<br>
--                   local k = math.min( t.i + 1000, #photos )<br>
--                   while ( t.i <= k ) do<br>
--                       -- do something to photos[t.i]<br>
--                       -- if trouble, return nil, msg.<br>
--                       t.i = t.i + 1
--                   end<br>
--                   if t.i > #photos then<br>
--                       return true -- done, no errors.<br>
--                   else<br>
--                       return false -- continue, no errors.<br>
--                   end<br>
--               end<br>
--           end<br>
--           local sts, msg = cat:action( 10, "Test", catalogAction, { i=1 } )<br>
--           if sts then<br>
--               -- log successful message.<br>
--           else<br>
--               -- print error message...<br>
--           end<br>
function Catalog:action( tmo, name, func, ... )
    return self:_action( catalog.withWriteAccessDo, tmo, name, func, ... )
end
--- Wrapper for un-named catalog:withPrivateWriteAccessDo method - divide to conquor func.
--  @param tmo (number) max seconds to get in.
--  @param func (function) divided catalog writing function: returns sts, msg = true when done; false if to be continued; nil, error message if trouble.
--  @param ... (any) passed to func.
function Catalog:privateAction( tmo, func, ... )
    return self:_action( catalog.withPrivateWriteAccessDo, tmo, nil, func, ... ) -- no name.
end
Note: These functions were changed a bit after further testing and integration - consider them for example purposes only. The released version of these functions will be available shortly as part of the Elare Plugin Framework:
https://www.assembla.com/spaces/lrdevplugin/
PS - After some mods, the final version allows me to write code like this which I find convenient (still not released yet at assembla.com):
local coll
local s, m = cat:update( 10, "Assure Folder Collection", function( context, phase )
    if phase == 1 then
        coll = catalog:createCollection( folder:getName(), parent, true )
        return false -- keep going.
    elseif phase == 2 then
        coll:removeAllPhotos()
        return false -- keep going.
    elseif phase == 3 then
        coll:addPhotos( folder:getPhotos() )
        return true -- done (same as returning nil).
    else
        app:error( "bad phase" )
    end
end )
if s then
    app:logVerbose( "Assured collection: ^1", coll:getName() )
    return coll
else
    return nil, m
end
or
local photos = catalog:getAllPhotos()
local s, m = cat:update( 10, "Update Photos", function( context, phase )
     local i1 = (phase - 1) * 1000 + 1
     local i2 = math.min( phase * 1000, #photos )
     for i = i1, i2 do
        local photo = photos[i]
        -- do something with photo that writes catalog.
    end
    if i2 < #photos then
        return false -- keep going
    end
end )
if s then
    app:logVerbose( "Photos updated..." )
    return true
else
    return false, m
end

Answer 1

Hi Vladimir,
You can see the outside & inside of DevMeta for complete details, but in a nutshell , there are few things the "background" task does, including:
1. Check if the develop settings of the currently viewed photo have changed, if so update metadata for the library panel view (and Library Filter, and Smart Collections...) - this is what I call: Auto-Update.
2. Check if user has edited the metadata - in which case it updates the develop settings of the active photo (called Live-Edit).
The posted function makes sure none of my other plugins complain if they can't get at the catalog while DevMeta has it (which usually does not happen, but sometimes does). However, other peoples plugins will occasionally fail if they assume the catalog to be free for them all the time (and they have DevMeta running). There is at least one other fellow doing something similar, so I'm not the only plugin developer with the "bright idea" of doing things in the background that require catalog access.
Rob