Batch RAW conversion: no sidecar XMP files

When I open and save a RAW file as a JPEG by hand, I always get a XMP file that tells me the RAW settings. Actually the XMP is generated as soon as I "open" the RAW file in PS. When I do the exact same steps in a batched action, no XMP files result. I just painfully batch converted 4000 RAW files, no XMP files showed up. Ran some test cases, manually doing the steps result in a XMP file, batching them doesn't. Why doesn't PS generate the XMP files?
One thing I worry about is what RAW settings were applied to my files. Without the XMP files it's very hard to check. Some settings, like noise reduction and saturation, are pretty subtle and hard to determine w/o a "with" and "without" comparision. In theory the XMP files would let me detemine exactly what settings were applied.
This is for Windows XP, CS3, latest RAW update downloaded a few days ago.

For the case of DNG files with both internal XMP and a sidecar file, Camera Raw picks the file with the newer OS modification date/time.
Adobe does not support sidecar XMP files for JPEG or TIFF files.
It is fairly trivial to write computer programs to exact XMP from files, so yes, it is "possible". You can save the XMP out of any opened file in Photoshop CS3's get info dialog in advanced mode (see the save button at the bottom).

Similar Messages

Sidecar XMP file question.

Greetings all, I have some questions regarding XMP sidecar files; I am creating a complete API to access and create metadata, including XMP (I'm not basing myself on any SDK). 1. A sidecar xmp file must have an extension of .xmp, right, but does it replace an existing extension, or simply add to it: For example, the sidecar filename for testit.exe should be testit.xmp or testit.exe.xmp? If it is the latter case, would it not cause problems with CDFS ISO 9660 Level 1 file systems (8+3 limit)? 2. Does this require the xpacket block? It would seem to be illogical to me, but still, it is not quite clear in the specification. My sidecar XMP generator immediately starts with the x:xmpmeta block (after the <?xml .. standard xml header). Is this wrong or correct? Thanks!

1. The convention so far has been to replace the extension to .xmp - ex. testit.xmp 2. Here is a sample of a side car file that was created for a RAW file: <x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 3.0-28, framework 1.6'> <rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/'> <rdf:Description rdf:about='' xmlns:crs='http://ns.adobe.com/camera-raw-settings/1.0/'> <crs:Version>2.0</crs:Version> ..... I believe it is done in this way so that when the user selects the RAW file in the Photoshop CS filebrowser the xmp info can be seen in the metadata pane and in the file info dialog.

Sidecar .xmp files ignored when importing all files from a card

Hello,
I've noticed what I think is a bug. This is my workflow:
* I shoot pictures as Raw (in .nef format)
* I run a program on them to add sidecar .xmp files with some GPS metadata
* When I import the .net into lightroom I click on "import" on the left panel.
* The import dialog box shows up, with 2 choices : import all pictures from card / select which images to import
1) If I choose to manually select the files to import, the XMP metadata are successfuly taken into account (I choose to convert the NEF to DNG and the GPS data are well imported)
2) But if I choose "import all pictures from card", the GPS metadata are ignored.
Can you advise, please?
Thanks!

Are the XMP sidecars actually on the card?

4.1.1 SDK Problems with missing xpacket tags in sidecar XMP files

The current 4.1.1 SDK has problems with sidecar XMP files that don't have the xpacket headers and trailers, i.e:
<?xpacket begin='ï»¿' id='W5M0MpCehiHzreSzNTczkc9d'?>
<?xpacket end='w'?>
is missing. Now, unfortunately Adobe Bridge CS2/CS3 does not export these xpackets in sidecar XMP files.
The standard, http://www.aiim.org/documents/standards/xmpspecification.pdf, is also very vague about it all:
● Write external metadata as though it were embedded and then had the XMP Packets
extracted and catenated by a postprocessor.
The grammar is strange(past tense had) and the spec implies that the xpacket should be extracted and again catenated... Someone should review this document and clearly state if xpacket statements should be in sidecar files or not. I suspect myself that they should be there, but the standard is very vague.
Anyway, there are two places in the SDK code where changes might be needed:
XMPFiles::Initialize has XMP_Asserts in case the xpacket header/trailer is missing, but the underlying assert is only active in debug builds.
XMPScanner::PacketMachine::FindNextPacket () also has in its truth table the assumption that the xpackets exist.
There could be even other places in the code that assumes that the xpacket tags are present in all files, which includes text XMP sidecar files.
Anyway.
a) Shouldn't bridge export the xpacket tags? Same with any other application?
b) If the spec is vague, then the SDK should not assume that the xpacket tags are present.
Any comments? Has someone already fixed this issue as I suspect a lot of apps using the the XMP SDK would break concerning reading XMP sidecar files? Thx, Kent

I was able to work around the problem by creating a mapped view of the .xmp file (this creates an array in memory backed by the file on disk, so there's no need to read the file into a separate internal buffer), and constructing the SXMPMeta object directly from the buffer. (The ctor for that class calls ParseFromBuffer, so this is the same thing as was suggested by other messages in this thread.)
It seems that Adobe needs to do one of these things:
(1) say that Bridge CS3 has a bug, and agree that Bridge CS3 should include a proper xpacket header when writing xmp sidecar files
(2) say that the XMP Toolkit has a bug, and that the SDK should be able to parse sidecar files without an xpacket header, and agree to fix the toolkit
(3) say that Bridge CS3 and the XMP Toolkit behave as expected, but then provide a sequence of steps by which users of the XMP Toolkit are expected to read xmp sidecar files written by Bridge CS3
Does Bridge CS4 write an xpacket header to the xmp sidecar files?
Maybe what I could do is create a custom file handler for .xmp sidecar files, so I could use the SXMPFiles for everything, instead of having to special-case .xmp files.
My needs are pretty modest though, and it might be just as simple to use the MS DOM-based XML parser for load the xmp sidecar file. I bet I could get the data I need (only the "Rating" for now) using a simple XPath expression.
-Matt

RAW + old photoshop generated .XMP file conversion to DNG

I have a set of camera raw files (.mos) and .xmp side car files that were generated in photoshop several years ago. I am thinking about converting all of the files and side cars to DNGs because Leaf Capture software is no longer being updated.
Before I do I want to confirm that Lightroom can read both the raw AND the corresponding side car files and convert them into single dngs. I also want to know if the .xmp information is extractable after the dng is created in the same way that the raw/original file is extractable.
I've been trying to test lightrooms ability to read .xmp files on my own. I added a .mos file in a folder with a corresponding .xmp file and a .mos file in a folder without a corresponding .xmp file. Lightroom reads similar adjustment information for both images but only generates a preview that more closely resembles a finished retouched .PSD of the added .mos file that does not have the .xmp file in the folder it was added from. I'm at a bit of a loss regarding how to proceed.

nonovsco wrote:
Upate on situation: I followed up with the individuals that created the files and found out that the side car files were in fact generated by leaf and not photoshop. i tried another test and it looks like the folder with a .mos +.xmp file is now reading differently than a .mos file by itself. both suggest that changes were made to the files with modulations in the "develop" section of lightroom.
You can view the XMP file data in Notepad or any other text editor, just select 'Open' and select 'All File Types' so you can see the XMP files. It should be possible to store both C1 and LR develop settings in the same XMP file, but there may be some naming conflcts. Review of an XMP file with both C1 and LR develop settings saved to it should show the two different develop data sets in Notepad. Conversion to DNG should save ALL XMP data, but you may lose some of the (edit) Leaf proprietary data that may be in the raw file's metadata. For now I'd hold onto the raws + XMPs, even if you do convert to DNG.
You can use EXIFtoolGUI to examine the metadata in the new DNG files and compare it to the original XMP file's data. My Canon DSLRs appear to have all of the manufacturer's 'Makers Notes' copied over from the raw file into the DNG metadata.
http://freeweb.siol.net/hrastni3/foto/exif/exiftoolgui.htm

Fix for Camera Raw 5.7 ignoring .xmp files?

I have a problem, some people seem to have: From a certain date on, every sidecared .xmp gets completely ignored. Camera Raw as well as Bridge seem to ignore them. They get created, whether I open them with Bridge or CR. Someone suggested I could fix this problem by associating the CR2 Format with Bridge or Photoshop (CS4), it worked for a while... if I opened it with the Windows Explorer. Opening a CR2 with Bridge netted in the same result: .xmp gets created, but gets ignored by Bridge and Camera Raw. Now even that fix doesn't seem to work: Neither CR nor Bridge want to acknowledge the files.
Is there a fix for that I'm unaware of?
€:
It seems I have found the problem: Someone mentioned it might have something to do with the timestamp of the files not being correct. I checked the date and time of my camera and realised I haven't changed the time, as Daylight-saving changed. Now the .xmp files read correctly. Perhaps that is some use for other people.
€, Update2:
I checked several things: If the modified time of the file lies in the future (here 1hr.), Bridge and Camera Raw have a problem recognizing the .xmp cause it seemed to have been created before the Photo was shot/modified last time. I took the pictures at 01:05pm (the were timed at 02:05pm). CR and Bridge ignored all .xmp-files created before 2:05pm. After 02:05pm, everything worked fine again.

Good to hear that the matter is acknowledged.
I had such cases. I think it was the renaming of old files. Not sure.
Many things can go wrong. Whatever triggers it. It woulld be great to have the option (checkbox) to enforce the (re-)synchronisation of an existing xmp file with a given Raw and/or dng file.
Best regards, Peter

Update RAW conversion for Canon 70D file to CS6

I've tried several times to update my RAW converter to allow editing of the Canon 70D CR2 file in Photoshop (MAC, CS6), but each time Photoshop gives the message that I should get the latest Camera Raw update. My Bridge Preference menu indicates that I have 7.1.0.354. My last attempts were updating to AdobeCameraRaw-7.1-mul-AdobeUpdate and then DNGConverter_7_3. Both installed without issue, but the problem persists.

The current version of ACR for CS6 is version 8.6 Link to DNG converter contains ARC 8.6 and Converter.
Mac - http://www.adobe.com/support/downloads/detail.jsp?ftpID=5824
Win - http://www.adobe.com/support/downloads/detail.jsp?ftpID=5825

.xmp files showing up in my SD card when accessing RAW files in Finder... Very weird!

Hi,
I'm having a very weird problem... I'm using a 4GB Scandisk SD card and am shooting my RAW photos on a Canon Rebel 2ti. I'm using a Macbook Pro, 2011 2.3Ghz i5, 4gb of Ram, with Lion v. 10.7.2 and am accessing the SD card RAW files through Finder. I'm secondary clicking on the image file or double clicking them as PS is my default opening program for RAW files. When I hit the "Done" key on any of the Photoshop files, a new .xmp file shows up in Finder. When I open it, it's a Microsoft Messenger for Mac 8 License Agreement...
I do have Microsoft Office 11 installed and have never used Messenger. It's not a huge deal, but it's driving me crazy and it's totally weird!
Thanks,
-Dave

It's a lot of separate, unrelated things happening to you in one long chain, but each step is normal and explainable.
You are opening raw files straight from a card into Photoshop. Photoshop will open them in Camera Raw. Camera Raw's preferences are set to save changes in XMP sidecar files. You are clicking Done, which applies the default Camera Raw processing, and that change generates an XMP file to store these new settings. That's why an XMP file is showing up in the same folder. On a computer, that's fine. But writing back out to a camera card with a device other than the camera is not recommended. What is recommended is to use another, faster way to preview raw files on a card without changing them (Adobe Bridge, Lightroom, Apple Aperture, Photo Mechanic, etc.), so that the card's left untouched as it should be.
If you don't want the XMP file to be generated, you have to do just one of the following:
In Camera Raw preferences, change the "Save image settings in" option to "Camera Raw database", not "Sidecar XMP files".
Instead of clicking Done every time, which applies any changes, click Cancel instead, if you were just looking and not editing.
Use something other than Photoshop to preview raw files. Most of the other ways are faster than opening them in Photoshop. The recommended workflow is to use a camera downloading utility to look through all the images on the card at once (instead of one by one as you are doing), and then select only the ones you want to bring into your computer. Bridge (via File > Get Photos from Camera, not by browsing), Lightroom, Aperture, and iPhoto all have that capability.
As to why double-clicking an XMP file opens a Messenger license agreement: As you probably know, the system has to figure out what program should open a file with a given filename extension. It sounds like maybe both Adobe and Microsoft have used the XMP extension, and the Office installer probably associated it with some type of MS Messenger file. So when you double-click an XMP file, Mac OS X opens Messenger. Since you have not used Messenger before, it's showing you the "first run" screen for Messenger, and Microsoft has set it up so that you have to click Agree on the license agreement before using it. If you never click Agree, it will still be the "first run" next time you open Messenger, whether you meant to or not. You can use Get Info in the Mac Finder to reassign XMP to open in another program.
You probably didn't expect so many things to happen from just opening a raw file off a card, but that's the way it worked out.

Camera Raw 6.7 does not save XMP files

I am running Camera Raw 6.7 on a Windows XP machine.
In the Preferences under "General" I have the "Save image settings in" set to "Sidecar ".xmp" files.
And in the DNG File Handling, I do NOT have the "Ignore sidecar ".xmp." file check-box checked in.
But still when I alter a raw file and press Done the DNG file changes date/size and no XMP-file is
created in the directory. Also when Ioad the picture and "reset" it to "Camera Raw Default" and
press Done, the next time I open it is in the "Image settings" state anyway
Anyone have any idea what the problem could be ?
Regards,
Klas

Noel Carboni wrote:
Adobe has taken the decision to ALWAYS write metadata into file types they know the structure of.
Personally I find this approach apalling...
And you were accused of being an Adobe "fan-boy" only hours ago...
+1 vote for sidecar option - all file types...

Export Camera Raw sidecar XMP from DB

Hi
I forgot to change my options for xmp sidecar files in bridge last time I installed it. For a good two months it's been saving xmp data in the camera raw databse instead of in sidecar XMP files.
Now I need to move a whole bunch of raw files to a new computer and I don't want to lose all the camera raw adjustments. Is there a trick to write out the info it has in the databse to XMP files along with the corresponding raw files?
As a last resort, would it be possible to import the existing xmp data in the DB on computer 1 into a already existing DB on computer 2?
thanks

ACR has an Export Settings to XMP menu item in the menu available by clicking the arrow-and-three-lines icon at the right of the title area of the tabs in the right panel of ACR.
On my Windows 7 computer the camera-raw database file appears to be:
C:\users\yourusername\AppData\Roaming\Adobe\CameraRaw\Database

Sidecar .xmp for a DNG file ...

Photoshop CS6 Reference says that "When a DNG file is processed in Camera Raw, the settings are stored in the DNG file itself, but they can be stored in a sidecar XMP file instead." How can I store the changes made to a DNG file in a sidecar file? Thanks.

Complete paragraph: "When a camera raw image file is processed with Camera Raw, the image settings are stored in one of two places: the Camera Raw database file or a sidecar XMP file. When a DNG file is processed in Camera Raw, the settings are stored in the DNG file itself, but they can be stored in a sidecar XMP file instead. Settings for TIFF and JPEG files are always stored in the file itself."
And it adds at the end of the same page: "If you want to store all adjustments to DNG files in the DNG files themselves, select Ignore Sidecar “.XMP” Files in the DNG File Handling section of the Camera Raw Preferences dialog box."

"Unable to save the raw conversion settings. There was a write permission error."

I just built a new workstation for processing photos using PS CS5. I use external drives to store my images as I find it easier for backing up as well as for when I want use my laptop for sorting, etc.. I copied over all my old "Collection" files, and of course, had to "fix" them once on the new machine. Once pointed in the right direction, the collections all fill out correctly.
When I go to process a collection though, I encounter an odd problem I can't seem to sort out. I can delete files, rename files, copy/paste to the external drives, etc.. What I can't seem to do is add Labels, Ratings, or modify RAW settings. I mean, I can use ACR to make adjustments, I just can't seem to save them.
The system just ignores label, rating commands completely. When I try to save an ACR adjustment I get this error. "Unable to save the raw conversion settings. There was a write permission error."
The files are NOT write protected. I'm set up as the owner of my workstation, with complete control of the system. I can't seem to find any useful information about this error because the "write permission error" seems to only be happening on installs, so that's all I can find help for.
My system:
Intel Core i7 970@ 3.20GHZ
24.0 GB RAM
64-bit OS - Windows 7 Pro
DX58S02 Motherboard
NVIDIA GeForce GTX 580

Thanks for replying. I've been going crazy trying to fix this.
To answer your first question; I use sidecar XMP files. If everything from my archive was shot in RAW I'd try converting to DNG and see if embedding the changes directly in a file worked. Unfortunately, a lot of my old stuff was shot in JPG, so sidecars seems to be the best choice.
When I add labels, ratings, as well as change RAW settings on files I place on my workstations HD everything works. It's only when I try to do these things with the same files on removable drives that I run into trouble. The drives I use are Transcend 640GB StoreJet 25ms connected via USB.
I don't think it's a UAC problem. At least, to my understanding, doesn't that control how your machine alerts you to program changes? I looked into how to change ownership, and did so, but that didn't work. Here's a link to what I mean: http://www.addictivetips.com/windows-tips/windows-7-access-denied-permission-ownership/
I'm stumped. I also hate the idea that I'll have to go through various folders to find images from already created collections, copy them to a new folder on my HD and then work on them. Then I'll have to put them all back... ugghhhh

*.xmp files are incomplete.

I am running Bridge CS5.1 on OSX Lion.
I am shooing RAW (*.CR2) on a 600D with FW 1.0.2
My workflow is to copy the files from the card to the internal HDD with finder. Then browse the files in Bridge, opening them in Camera RAW when edits need to be made.
This was working until I reset bridge recently due to some cache corruption (which was resolved by the reset). Now, on my latest import when I open the files in Camera RAW, make changes and press done an *.xmp is created, however this file does not contain the changes I made (to white balance, exposure, sharpness, cropping etc). As a result of this, the icon which indicates changes does not show in bridge and when I open the file in Camera RAW again the changes are missing.
Oddly, this problem is only applicable to my latest import - the files which I had imported previously still work within the same workflow. So it appreas there's possibly something wrong with the permissions on the newly created files.
I understand this thread (http://forums.adobe.com/message/4102902#4102902) is addressing a similar problem, and it remains unresolved.
This is a link to an *.xmp file which is holding the changes - http://dl.dropbox.com/u/14483853/IMG_3825.xmp
And this is a link to one which is not - http://dl.dropbox.com/u/14483853/IMG_3832.xmp
Any help would be greatly appreciated.

The XMP files contain information about the last conversion you did on a particular raw file. If you delete it your conversion parameters will revert to defaults next time you open the raw file, instead of Camera Raw using the last conversion parameters.
Go into Edit - Preferences - Camera Raw... and you'll see that you could choose to save the conversion parameters in a central database if you don't like the sidecar XMP files showing up. If you DO choose the central database, consider upping the max size from the default of 1 GB.
-Noel

Why is Bridge creating XMP files in CS6?

I'm using the CS6 beta on a new set of photos. All I've done so far is batch rename all the files (all raw, all CR2) and edited the IPTC metadata - to add my name, the project name, etc. Now I've got a side-car file for every photo. CS5 just included this stuff in the original file, why not CS6?

Hi,
The 'Camera Raw Database' is written to a local directory on your machine, and won't travel (without an extra step) if you copy the CR2 file to another machine. If you use DNG then the metadata will be stored in the file.
If you store the camera raw settings in the Camera Raw database and plan to move the files to a different location (CD, DVD, another computer, and so forth), you can use the Export Settings To XMP command to export the settings to sidecar XMP files.
You may have been using, and wanting, sidecar .xmp files, but just had them hidden in previous Bridge versions. To hide the .xmp file from the Content panel's display, go to View> Show Hidden Files and toggle the checkmark OFF, by selecting it.
regards,
steve

DNG and MESZ timestamp / sony A100 RAW conversion

Hello,
i have a question regarding the timestamp within the EXIF metadata after the RAW to DNG conversion for my A100 pictures. Today I have seen that the timestamp betwen the RAW original file and the converted DNG file differs about 3 hours. I can't explain me this difference by myself. Has someone here an idea for this difference? I use the Mac Version of the actual free DNG converter. My timezone is MESZ/CEST what also means UTC+2 at the moment if I'm right. So 2 hours difference I could imagine because of such a time zone missmatch, but not 3 hours. But if i look at the RAW files the timestamp addition also shows MESZ like in the DNG after conversion. The main reason why I want to use DNG format is the linkage with my GPS logger data. But if the timestamps will convert wrong, this would be a no go for me.
Thanks for your help.
Greetings,
Tschubi

I think what you are seeing is a longstanding mis-match between how Exif and XMP record date-time values, and--possibly--a failure on the part of some of your software to understand how to use date-times read from XMP for the Exif date-time properties. When reading an Exif date-time from XMP, the UTC offset should always be ignored when interpeting the date-time--it has no meaning.
Three date-time metadata properties are involved in this issue, in XMP they are stored as the following properties:
exif:DateTimeDigitized
exif:DateTimeOrigional
xmp:ModifyDate
When Adobe software reads Exif metadata from files, it maps the values in the Exif tag(s) to XMP properties. The XMP is stored inside the file when the software saves or updates a file (or sidecar .xmp file for RAW files).
For exmaple, the exif:DateTimeDigitized the value is taken from two Exif tags, 0x9003 ("DateTimeDigized") and 0x9291 ("SubSecTimeDigizied"). See Part 2 of the XMP Specification for more details about how Exif is mapped to XMP.
The Exif specification defines no way to record a time zone or UTC offset for its date-time values. If you are trying to figure out what time the picture was taken reading Exif, it's impossible to know what time zone is correct. If cameras are recording the time zone, they are storing it in propritary metadata in the file, and not in a standard Exif tag.
Unfortunetly until recently the XMP Specification defined a date-time string format that requires a UTC offset. That's the mismatch. So, what's the software do when it's supposed to write a UTC offset, but doesn't have one to go with the time? Traditionally Adobe software uses whatever time zone your computer's clock has at the time the XMP property is created (that is, when the Exif is read).
So, if your camera's clock was set to record 2009:08:28 11:26:07 in the Exif, and you convert your raw file to a DNG with a computer in Europe, you're probably going to get something like 2009-08-28T11:26:07+02:00 in the XMP. But if I convert the same RAW file to a DNG here in Seattle, I'm going to get 2009-08-28T11:26:07-07:00.
Of course the right thing to do would be to just write 2009-08-28T11:26:07 in the XMP, leaving out the time zone designator (TZD). Expect that in the future. The XMP Specification now says that the TZD is optional and that "software should not assume anything about the missing time zone," but even the most recent release of the DNG Converter (5.5 as I write this) has not yet caught up with this update to the XMP Specification, and it still writes the bogus TZD. Also, any XMP written by older Adobe software will have the TZDs.
What Adobe software like Bridge and Lightroom have done tradtionally is to ignore the TZD in the XMP when using these dates. You'll notice that browsing DNGs with the dates 2009-08-28T11:26:07+02:00 and 2009-08-28T11:26:07-07:00, and running Bridge anywhere in the world (that is regardless of the the clock's Time Zone setting on your computer), the metadata panel will still show you that you took the picture at 11:26 am on August 28, 2009. If other software is trying to shift dates to local times based on the UTC offsets recorded in XMP for these dates, that's a bug.
Note that unlike other date-time values in Exif, the Exif specification says that the GPSTimeStamp is UTC, so this problem does not affect GPS metadata.
I mentioned the XMP specification a few times, you can download it here:
http://www.adobe.com/devnet/xmp/
-David

Batch RAW conversion: no sidecar XMP files

Similar Messages

Maybe you are looking for