Metadata identifying a person in a photo

Question

In the olden days, family photos went in a shoe box, and if you were lucky, someone wrote on the back of it to tell you who was in the photo. If nobody did that, chances are that a priceless old family photo would eventually become a worthless photo because nobody knows who it's of.
I went on a slide-scanning rampage a couple years ago and now have 15,000 images taken from slides, along with hundreds of really old family photos, and hundreds more of my own digital camera photos. I would like to identify who is in each photo, but I'm still waiting for the right metadata definition to come along so that doing so would not be a waste of time.
JPEG, TIFF and other image file formats typically allow metadata to be stored inside the file. This allows information about the image such as the date it was taken, keywords, a caption, a title, where it was taken, etc., to be stored inside the image file itself, so when you send someone a copy of the file, they're getting information about the the image at the same time.
Unfortunately, although you can add keywords and captions, there is not yet a well-established standard for identifying people in photographs. I have tried putting everyone's name in a caption, using a comma-separated list of names from left to right, and a semicolon between rows, back to front. But there is often some ambiguity with this ("was the baby on the lady's lap part of row 1 or 2?"), and it is just my own convention. Also, often there are only one or two people I know in a group shot, so I have to resort to "3rd from the left, second row" type descriptions, which is annoying. It really gets old after doing too many pictures with similar people in them, too, and uses up the space in the caption you'd rather use for a description sometimes.
Flickr allows you to draw a rectangle in an image and add a note that is associated with that rectangle. That's a great idea, and apparently they got it from "fotonotes", which does the same thing.
Fotonotes stores a chunk of XML text in "the 8th JPEG header", which is a good start, but I don't know whether such metadata can also be stored in TIFF files or not, and it doesn't seem likely that such headers would survive through utilities.
IPTC is the most common metadata format used in JPEG and TIFF files, but Adobe has been pushing its extensible "XMP" format for storing metadata in image files. Adobe's various programs (like Photoshop) recognize XMP metadata and preserve it whenever you manipulate or convert the files, so it may well be the best place to store the kind of metadata I'm talking about. Microsoft Vista's image browser apparently supports XMP as well. Since XMP is "extensible" (and is an open format), it should be possible to define tags in XMP that would support a rectangle with a note on it.
Does anyone know how to do that? If we could get that figured out, then the next step would be to get several big players (e.g., Adobe, Google, Flickr, Vista, iPhoto, etc.) to support those tags so that tagging your photos that way really does preserve the information for posterity instead of just being a proprietary tag that no software recognizes.
Resolution Changes
It is quite common to change the resolution of a photo for various uses, and one would not want the rectangles to break when that happens. The way Flickr handles it is that they treat the longest dimension as though it was 500 pixels, and then specify their coordinates accordingly. So if you draw a rectangle from 100,200 to 500,800 in an image that is 1500x3000, then they would take the longest dimension (3000) treat it as though it was 500, and scale the coordinates accordingly, in this case, multiply each coordinate by 500/3000=1/6. That way, if you reduce (or enlarge) the image, the coordinates can stay the same. If you crop, you're still hosed, although you would likely be able to manually "repair" the rectangles without knowing who's who in the picture by looking at how they are rela

Answer 1

Note the last paragraph in the original post which explains how Flickr handles resolution changes without breaking the rectangles. Perhaps a more straightforward approach would be to store both the rectangle coordinates for each rectangle AND the image's width and height at the time the rectangles were created. Then if the image is scaled, the rectangles can be scaled accordingly. For example, if a rectangle is at x=100,y=200,w=50,h=70, the "original dimensions" are 1000x2000, and the image's current dimensions are 500x1000, then an application could assume the image has been scaled down by (500/1000=) 0.50, and so the x,y,w,h of each rectangle could also be multiplied by 0.50 (x=50,y=100,w=25,h=35). I'd recommend storing floating-point values so that multiple scalings don't make the rectangles "creep".
Of course, it is possible that really the image was cropped rather than scaled, but fairly unlikely you would happen to crop it the same percent in both directions. Even if an image is cropped, someone could manually repair the rectangles by sliding and scaling them until the fit back on the faces (unless most of the faces were cropped off).
A rectangle-aware application could scale and translate the rectangles during a crop or scaling operation, but other applications could still survive scaling if they at least preserve the XMP tags as-is.