Fixed Video CreationDate Sorting After Converting PSE Catalog

Question

My experience with the problems of sorting videos properly by creating date nearly caused me to give up and go back to PSE Organizer...but I dug in, hacked out a solution, and now I'm sharing it in case it helps anyone else.  This worked for me...but please realize that I could have destroyed my own catalogs if things weren't all as expected.  My Python script below is a down & dirty, no frills way to convert and push all the "CORRECT" timestamps in my old PSE10 catalog to the horribly handled and mis-dated VIDEO files in LR4.  Here's the script for all those who dare to take the life of their catalogs into their own hands.
#  WARNING: Use at your own risk.  This worked for me, but could completely    #
#           DESTROY your catalogs for both Lightroom (LR) and Photoshop        #
#           Express (PSE).  Copy the {YourFilename}.lrcat from LR and the      #
#           catalog.pse??db from PSE to your local python directory and make   #
#           BACKUP copies of those files before you try to move the modified   #
#           ones back to their proper locations.                               #
# Auth...erm, Hacker: Sam Birch                                                #
#  Purpose: Fix Adobe's repeated inability to apply proper dates to video      #
#           files when importing a PSE catalog.  This script gets the list of  #
#           all video files in a Lightroom 4.0 catalog (have not verified this #
#           for any other version), looks for the correct creation date/time   #
#           for those files in the PSE catalog, converts the PSE UTC stamp to  #
#           MY LR local timezone (US/Central), properly formats the output for #
#           LR's captureTime, and saves them in the LR catalog.                #
#    Notes: 1) I moved from PSE 10 to LR 4.  Those are all I wrote this for,   #
#              so do not expect this to work for anything else.                #
#           2) I am in the US/Central timezone, so anyone not in that timezone #
#              will need to edit the script to make it work properly for you.  #
#           3) This is a Python script, and I used 2.7.2 on Windows.  I also   #
#              needed to add the pytz package to handle the timezone           #
#              conversion properly.                                            #
#           4) I am not a Python expert.  I wanted to learn Python, and it let #
#              me accomplish what I wanted.  I would appreciate any construct- #
#              ive criticism with proposed improved code, but please don't ask #
#              for modifications or customizations from me.  I don't have the  #
#              time or knowledge to do so...sorry.  As this was simply about   #
#              taking the least time to get LR functional for me, all config   #
#              items are hard-coded and require modification of the script...  #
#           5) ALL my video files had unique names.  If this is not the case   #
#              for you, THIS WILL NOT WORK RIGHT FOR YOU!!!!                   #
import sqlite3 as lite
import sys
import os, time
from datetime import datetime, timedelta
from pytz import timezone
import pytz
# Your Lightroom catalog (COPIED to the local dir)
lrcon = lite.connect('./test-fresh.lrcat')
# Your Photoshop Express Organizer catalog (COPIED to the local dir)
psecon = lite.connect('./catalog.pse10db')
# Get all the video files in the LR catalog...
lrcur = lrcon.cursor()   
lrcur.execute("SELECT Adobe_images.CaptureTime, AgLibraryRootFolder.absolutePath, AgLibraryFolder.pathFromRoot, AgLibraryFile.idx_filename, Adobe_images.id_local  FROM Adobe_images, AgLibraryFile, AgLibraryFolder,AgLibraryRootFolder WHERE (Adobe_images.fileFormat=\"VIDEO\") AND (Adobe_images.rootFile=AgLibraryFile.id_local) AND (AgLibraryFile.folder=AgLibraryFolder.id_local) AND (AgLibraryFolder.rootFolder=AgLibraryRootFolder.id_local);")
psecur = psecon.cursor()
rows = lrcur.fetchall()
# Variable setup for stats and timezone conversion...
failcount = 0
successcount = 0
utc=pytz.utc
central=timezone('US/Central')
fmt = "%Y-%m-%dT%H:%M:%S" #LR captureTime format string
for row in rows:
    # Attempt to fix through local modified time...but useless since I used
    # PSE backup/restore a few years ago, and that wiped out all original
    # timestamp information.  If you are in Windows and your modified timestamp
    # on the file is correct, this should work...since it worked (but didnt' fix
    # my problem due to PSE's terrible version of backup/restor), I thought I
    # would leave the code as inspiration for others it still may work for...
    #myroot = row[1].encode('utf8')
    #mypath = row[2].encode('utf8')
    #myfile = row[3].encode('utf8')
    #myidx = row[4]
    #photo = myroot+mypath+myfile
    #mytimeLR = time.strftime("%Y-%m-%dT%H:%M",time.localtime(os.path.getmtime(photo)))
    #print photo," last modified: ",mytimeLR
    #updateTmp = (mytimeLR,myidx)
    #lrcur.execute('UPDATE Adobe_images SET captureTime=? WHERE id_local=?',updateTmp)
    #lrcon.commit()
    # Get the filename (ALL MY VIDEOS HAD UNIQUE FILENAMES, SO THIS LEVEL OF MATCHING SUFFICED FOR MY PURPOSES)
    myfile = row[3].encode('utf8')
    # Set up the parameter for the query to the PSE catalog
    matchParm = []
    matchParm.append(myfile)
    # Get the UTC timestamp and file name from the PSE catalog
    psecur.execute('SELECT search_date_begin, filepath_search_index FROM media_table WHERE lower(filename_search_index)=lower(?)',matchParm)
    pserow = psecur.fetchone() # I could do this because I KNOW all my video files have unique names
    if pserow != None:
        # Go through the hassle of decoding and converting the UTC time to US/Central...as well
        # as from the PSE format to LR format similar but different enought to be annoying...
        psedate = pserow[0].encode('utf8')
        pseUTC = datetime(int(psedate[0:4]),int(psedate[4:6]),int(psedate[6:8]),int(psedate[9:11]),int(pse date[11:13]),int(psedate[13:15]),0,tzinfo=utc)
        lrLocal = pseUTC.astimezone(central)
        lrDate = lrLocal.strftime(fmt)
        # I like seeing progress...
        print myfile, ": ", pserow[0]," -> ", lrDate
        # push the corrected time back into the LR catalog...the only WRITE activity so far...
        updateParm = (lrDate,myidx)
        lrcur.execute('UPDATE Adobe_images SET captureTime=? WHERE id_local=?',updateParm)
        lrcon.commit()
        successcount += 1
    else:
        # I got here because I added some video files to LR after I imported, but before
        # I hacked out this script...
        failcount += 1
print "SUMMARY: success count = ", successcount, "  - fail count = ", failcount
if lrcon:
    lrcon.close()
if psecon:
    psecon.close()
Sam

Answer 1

Also, since my camera does not create sidecar files and the video format does not allow for metadata directly...I have problems with date sorting.  LR likes the "Creation Date"; while I understand that Windows' NTFS is one of the few operating systems that actually store a "Modified Date", I cannot understand why Adobe would choose to use a data that changes with every copy (Creation Date), rather than the original capture date (Modified Date) when it is clearly available.
My solution is to move my videos to their final destination outside of LR, run the below PowerShell script to set Creation and Accessed dates both to the Modified date for the file, and then import with LR.  This is a blatant waste of time, but it seems the least of evils I'm aware of (up to the LR 4.0 I'm currently running).  If anyone knows of a better solution, please point me that direction.  Otherwise, I'm hoping this may help others in my boat.
If you ask "Why PowerShell when your last script was Python?"...it boiled down to being able to do the most work with the least amount of coding since Powershell has all that's necessary.
Just create the below as FixDates.ps1, then use command.exe in the root of the directory (and sub directories, since this works recursively) you want to do this for.  I use this on the directory for the batch I just imported.
OH, AND OF COURSE...USE FOR INSPIRATION/UTILITY AT YOUR OWN RISK.  IT WORKS FOR ME, BUT I GUARANTEE NOTHING...
$Videos = Get-ChildItem . -Recurse | where{$_.Extension -match "avi|mov|3gs"}
foreach ($Video in $Videos) {
    Write-Host $Video.fullname
    $Video.CreationTime = $Video.LastWriteTime
    $Video.LastAccessTime = $Video.LastWriteTime
Sam