Archeive Lightroom Files with metadata

How can you backup your original lightroom 1.0 files to an external drive so you can completely restore data if computer dies? or want to transfer to 2nd computer?
Is the only way-to export DNG format with metadata (these are not original raw format)?
Aperture has a one click vault feature-copying all to external drive.

LR can be set-up yo back-up the dB on a regular basis. The images you should be already backing up. That is all you need at a minimum.
Look in Prefs under Gneral/Default Library for the back-up pulldown.
Don
Don Ricklin, MacBook 1.83Ghz Duo 2 Core, Pentax *ist D
http://donricklin.blogspot.com/

Similar Messages

Ftp upload via Lightroom mobile with metadata?

I've been asked to do a job where I take photos on my mobile and upload directly to an ftp with metadata. Is that something I can do with Lightroom mobile? Or would I have to revert to my laptop after the shoot and do it that way?

OK, let's assume you have the Dropbox app installed on your mobile device. Then in LrMobile , you go to the image and tap the Share icon. The second item is Open In, and you should see Dropbox listed. Choose it and Dropbox's app then gives you a Save button which uploads the adjusted picture.
I've not checked if this includes metadata, and it seems to be one image at a time. I only used it for getting adjusted screenshots from my iPad to someone, so whether it works for you is something you will have to assess, and maybe something else will be more suitable. My guess is that if you load an iOS FTP app (lots of them around) you would go through a similar workflow, Share button and then Share or Open In.

Copy file with metadata to an external system webservice

I am trying to create a custom workflow in visual studio 2013. As a sharepoint user
I want to be able to select a document in a document library and push a button to send this file with meta-data to other system webservice. Is there Any good tutorial i can use ?

You can start here:
https://msdn.microsoft.com/en-us/library/office/dn567558.aspx?f=255&MSPPError=-2147217396
Blog | SharePoint Field Notes Dev Tools |
SPFastDeploy | SPRemoteAPIExplorer

Powershell: How do I upload files with metadata from a manifest file?

I am using the script from this blog to try to upload files into SharePoint, using a manifest file to specify the metadata associated with each file. Right now, the script works, but is not populating
the metadata from the manifest (xml) file. When I write the $metadataManifest variable out to the console, I get the contents of my xml file, so the code is reading the file when it should. However when it gets to the line in bold ($metadataManifest.Columns.Column),
it does not go into that block...it just skips on to checkin the file. Again the $metadataManifest variable shows all the content in my xml manifest file. Here is what the script looks like...
#Get web and document library objects
$web = Get-SPWeb "http://companySite"
$Library = "My Library"
$docLibrary = $web.Lists[$Library]
$ManifestFilePath = "C:\PowerShellScripts\Manifest.xml"
$LocalPath = "C:\Upload\Test Upload\My Upload Directory\"
if ($ManifestFilePath)
$metadataManifest = (Get-Content ($ManifestFilePath))
write-host "Manifest file: " $metadataManifest
else
write-host "Manifest file not specified for categorising uploaded documents"
#Check for the LibraryStartFolder parameter to specify a root folder
if ($PSBoundParameters.ContainsKey('LibraryStartFolder')) {
$folder = $web.GetFolder($docLibrary.Title + $LibraryStartFolder)
else
$folder = $docLibrary.RootFolder
#Attach to local folder and enumerate through all files
if($IncludeSubFolders) {
$files = Get-ChildItem $LocalPath -Recurse
else
$files = Get-ChildItem $LocalPath
$files | ForEach-Object {
#Check if the object is a folder - if so, create it in doc library
if ($_.PSIsContainer) {
if (($IncludeSubFolders) -and (!$FlattenStructure)) {
#Generate folder path for creation in SharePoint
#by looking at the parent folder on the local path
$spFolderPath = ($_.Parent.FullName.Replace($LocalPath,"")).Replace("\","/")
#Get the folder into which the new folder will be created
#by adding the folder path generated above, if one existed
if ($spFolderPath -eq "") {
$currentFolder = $web.GetFolder($folder.Url)
else
$currentFolder = $web.GetFolder($folder.Url + $spFolderPath)
#Check to see if subfolder already exists
#and create it if not
$testFolder = $currentFolder.SubFolders[$_.Name]
if ($testFolder -eq $null) {
write-host "`nAdding folder" $_.Name "to" $docLibrary.Title "in" $web.Title "..." -foregroundcolor Green
$newFolder = $currentFolder.SubFolders.Add($_.Name)
else
write-host "`nFolder" $_.Name "already exists in" $docLibrary.Title "and shall not be created" -foregroundcolor Red
else
#Generate file path for upload into SharePoint
if ($FlattenStructure) {
$spFilePath = ("/" + $_.Name)
else
$spFilePath = ($_.FullName.Replace($LocalPath,"")).Replace("\","/")
$spFullPath = $folder.Url + "/" + $spFilePath
#Check if the file exists and the overwrite option is selected before adding the file
if ((!$web.GetFile($spFullPath).Exists) -or ($Overwrite)) {
#Add file
write-host "`nCopying" $_.Name "to" $spFullPath.Replace("/" + $_.Name,"") "in" $web.Title "..." -foregroundcolor Green
$spFile = $folder.Files.Add($spFullPath, $_.OpenRead(), $true)
#$spFile = $folder.Files.Add($folder.Url + "/" + $file.Name, [System.IO.Stream]$fileStream, $true)
$spItem = $spFile.Item
#Walk through manifest XML file and configure column values on the file
$metadataManifest.Columns.Column | ForEach-Object {
#Single value text columns
try
if (($_.Type -eq "Text") -or
($_.Type -eq "Choice") -or
($_.Type -eq "Boolean") -or
($_.Type -eq "Number") -or
($_.Type -eq "Currency")) {
$columnName = $_.Name
write-host "Setting value on column"$columnName "..." -foregroundcolor Blue
$_.Values.Value | ForEach-Object {
$spItem[$columnName] = $_
write-host "Value set to"$_
catch {}
#Multiple line text column
try
catch {}
#Multiple choice columns
try
catch {}
#Hyperlink columns
try
catch {}
#Single User column
try
catch {}
#Multiple User column
try
catch {}
#Single value Managed Metadata column
try
catch {}
#Multi value Managed Metadata column
try
catch {}
#Update document with new column values
$spItem.SystemUpdate($false)
#Check in file to document library (if required)
#MinorCheckIn=0, MajorCheckIn=1, OverwriteCheckIn=2
if ($CheckIn) {
if ($spFile.CheckOutStatus -ne "None") {
$spFile.CheckIn("File copied from " + $filePath, 1)
write-host $spfile.Name"checked in"
#Approve file (if required)
if ($Approve) {
if ($spItem.ListItems.List.EnableModeration -eq $true) {
$spFile.Approve("File automatically approved after copying from " + $filePath)
if ($spItem["Approval Status"] -eq 0) { write-host $spfile.Name"approved" }
else
write-host "`nFile"$_.Name "already exists in" $spFullPath.Replace("/" + $_.Name,"") "and shall not be uploaded" -foregroundcolor Red
$web.Dispose()
And here is what the manifest file looks like...
<?xml version="1.0" encoding="utf-8"?>
<Columns>
<Column Name="Column1" Type="Text">
<Values>
<Value>First File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
<Columns>
<Column Name="Column1" Type="Text">
<Values>
<Value>Second File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>9876</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data2</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More more Data</Value>
</Values>
</Column>
</Columns>
I can't figure out what am doing wrong...why my script is not iterating through the manifest file to upload the document along with the metadata. I would really appreciate any help. Thanks

Hi Spawn,
Name should not be a problem.. I will probably post you the entire code (I modified it to meet your goals for having multiple column xml nodes as well)
For the code snippet below:
a. In powershell all my changes will have ## prefixed comments
b. in XML I have changed one column name to "Name" to replicate your case and also added a parent node for making the xml well formed.
and the SharePoint output
Please see the code below
$web = Get-SPWeb "http://intranet/sites/test"
$Library = "My Library"
$docLibrary = $web.Lists[$Library]
$ManifestFilePath = "D:\Manifest.xml"
$LocalPath = "D:\UploadPath\"
if ($ManifestFilePath)
##read the xml file as an xml object
[xml]$metadataManifest = (Get-Content ($ManifestFilePath))
write-host "Manifest file: " $metadataManifest
else
write-host "Manifest file not specified for categorising uploaded documents"
#Check for the LibraryStartFolder parameter to specify a root folder
if ($PSBoundParameters.ContainsKey('LibraryStartFolder')) {
$folder = $web.GetFolder($docLibrary.Title + $LibraryStartFolder)
else
$folder = $docLibrary.RootFolder
#Attach to local folder and enumerate through all files
if($IncludeSubFolders) {
$files = Get-ChildItem $LocalPath -Recurse
else
$files = Get-ChildItem $LocalPath
$counter=0
$files | ForEach-Object {
#Check if the object is a folder - if so, create it in doc library
if ($_.PSIsContainer) {
if (($IncludeSubFolders) -and (!$FlattenStructure)) {
#Generate folder path for creation in SharePoint
#by looking at the parent folder on the local path
$spFolderPath = ($_.Parent.FullName.Replace($LocalPath,"")).Replace("\","/")
#Get the folder into which the new folder will be created
#by adding the folder path generated above, if one existed
if ($spFolderPath -eq "") {
$currentFolder = $web.GetFolder($folder.Url)
else
$currentFolder = $web.GetFolder($folder.Url + $spFolderPath)
#Check to see if subfolder already exists
#and create it if not
$testFolder = $currentFolder.SubFolders[$_.Name]
if ($testFolder -eq $null) {
write-host "`nAdding folder" $_.Name "to" $docLibrary.Title "in" $web.Title "..." -foregroundcolor Green
$newFolder = $currentFolder.SubFolders.Add($_.Name)
else
write-host "`nFolder" $_.Name "already exists in" $docLibrary.Title "and shall not be created" -foregroundcolor Red
else
#Generate file path for upload into SharePoint
if ($FlattenStructure) {
$spFilePath = ("/" + $_.Name)
else
$spFilePath = ($_.FullName.Replace($LocalPath,"")).Replace("\","/")
$spFullPath = $folder.Url + "/" + $spFilePath
#Check if the file exists and the overwrite option is selected before adding the file
if ((!$web.GetFile($spFullPath).Exists) -or ($Overwrite)) {
#Add file
write-host "`nCopying" $_.Name "to" $spFullPath.Replace("/" + $_.Name,"") "in" $web.Title "..." -foregroundcolor Green
$spFile = $folder.Files.Add($spFullPath, $_.OpenRead(), $true)
#$spFile = $folder.Files.Add($folder.Url + "/" + $file.Name, [System.IO.Stream]$fileStream, $true)
$spItem = $spFile.Item
## ensure we pick the corresponding node in the columns xml (first file, first xml node, second file - second node likewise)
#Walk through manifest XML file and configure column values on the file
$metadataManifest.FileData.Columns[$counter].Column | ForEach-Object {
#Single value text columns
try
if (($_.Type -eq "Text") -or
($_.Type -eq "Choice") -or
($_.Type -eq "Boolean") -or
($_.Type -eq "Number") -or
($_.Type -eq "Currency")) {
$columnName = $_.Name
write-host "Setting value on column"$columnName "..." -foregroundcolor Blue
$_.Values.Value | ForEach-Object {
$spItem[$columnName] = $_
write-host "Value set to"$_
catch {}
#Multiple line text column
try
catch {}
#Multiple choice columns
try
catch {}
#Hyperlink columns
try
catch {}
#Single User column
try
catch {}
#Multiple User column
try
catch {}
#Single value Managed Metadata column
try
catch {}
#Multi value Managed Metadata column
try
catch {}
#Update document with new column values
$spItem.SystemUpdate($false)
##increment the counter to read the next xml node
$counter++
#Check in file to document library (if required)
#MinorCheckIn=0, MajorCheckIn=1, OverwriteCheckIn=2
if ($CheckIn) {
if ($spFile.CheckOutStatus -ne "None") {
$spFile.CheckIn("File copied from " + $filePath, 1)
write-host $spfile.Name"checked in"
#Approve file (if required)
if ($Approve) {
if ($spItem.ListItems.List.EnableModeration -eq $true) {
$spFile.Approve("File automatically approved after copying from " + $filePath)
if ($spItem["Approval Status"] -eq 0) { write-host $spfile.Name"approved" }
else
write-host "`nFile"$_.Name "already exists in" $spFullPath.Replace("/" + $_.Name,"") "and shall not be uploaded" -foregroundcolor Red
$web.Dispose()
and the xml file
<?xml version="1.0" encoding="utf-8"?>
<FileData>
<Columns>
<Column Name="Name" Type="Text">
<Values>
<Value>First File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
<Columns>
<Column Name="Name" Type="Text">
<Values>
<Value>Second File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
<Columns>
<Column Name="Name" Type="Text">
<Values>
<Value>Third File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
<Columns>
<Column Name="Name" Type="Text">
<Values>
<Value>Fourth File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
<Columns>
<Column Name="Name" Type="Text">
<Values>
<Value>Fifth File</Value>
</Values>
</Column>
<Column Name="Column2" Type="Text">
<Values>
<Value>12585</Value>
</Values>
</Column>
<Column name="Column3" type="Text">
<Values>
<Value>Some Data</Value>
</Values>
</Column>
<Column name="Column4" type="Text">
<Values>
<Value>More Data</Value>
</Values>
</Column>
</Columns>
</FileData>
Thanks and Regards,
Partha
AvePoint

How to export files with metadata information displayed?

I want to wash my .jpg files through LR 5 in order to display the filenames. Via the Slideshow module I have been able to select and direct the displaying of the desired metadata option. When I export the files the displayed metadata information does not make the trip. Any assistance is appreciated.

I have given my filenames descriptors, for instance, 0000100 - Jerry rock climbing in Custer State Park SD. Importing into Lightroom allows me to display the filename metadata. Exporting the files generated in Lightroom to Photoshop Elements allows me to then create a slideshow in which I have more audio flexibilities.

How to reload image file with metadata

Usually I capture pictures in RAW and import into Aperture library. A lot of them is not very good or very important, but after making rate, adding keywords, I want to output them from RAW to JPEG.
And then, import back those JPEG images into the same library with other "good image" RAW.
But, how to keep all the JPEG in same album with rate and keyword?

try this, but test it first. You could use referenced imaged in your library. Do set you library to automatically create previews. Set the preference for Previews to the quality you prefer for the saved JPEG images. Use the "Update Preview" option to create previews of the image you want only the JPEG version of. Once you have a preview of the images you only want a JPEG of, move the original out of the referenced storage location.

Multiple File Upload With Metadata Using REST

hi all
I want to upload multiple files with metadata to document library using REST API. I am using this msdn article
http://msdn.microsoft.com/en-us/library/office/dn769086(v=office.15).aspx for uploading file. I am able to upload single file to document library but it is not working for multiple file. when I select multiple file it is uploading last selected file. can
anyone help with this. I am using office 365 environment.
Thanks in advance

Hi,
According to your post, my understanding is that you wanted to use the REST to upload multiple files.
Per my knowledge, the REST API is not supported for uploading multiple files via a single call.
You can write your own loop to upload multiple files via an individual call.
http://sharepoint.stackexchange.com/questions/108525/multiple-file-upload-with-metadata-using-rest/108532#108532
More reference:
http://sharepointfieldnotes.blogspot.com/2014/04/uploading-documents-and-setting.html
http://www.shillier.com/archive/2013/03/26/uploading-files-in-sharepoint-2013-using-csom-and-rest.aspx
Thanks & Regards,
Jason
Jason Guo
TechNet Community Support

Photo files with no image

I have 201 Aperture files with metadata but no image. Is there something I can do to recover those images? Thanks.
John Willetts

Jim
Yes that is a problem local to that machine. I downloaded the file. It opened in Preview. It imported to and exported from iPhoto with no issues and the FInder reports the dimensions and file size correctly.
I think the clue is in the tiny dimensions that the iMac reports. That 256 x 192 image is the thumbnail within the dng, and that suggests to me that the Raw support on the iMac is damaged in someway.
Regards
TD

FAQ: Will Lightroom work with my camera, or Why won't my raw files open?

What is a raw file?
A raw file is the unprocessed data that was captured by your camera sensor when you took your picture. The only settings on your camera that apply to this file are aperture, shutter speed, and ISO. This means that any effects you may have applied to the file, such as white balance or black and white, may not appear in Lightroom. Your camera will create a JPEG copy of your raw file with all current settings applied and save that within the raw file for use as a thumbnail by most applications. When you first view a raw file in Lightroom, you will see this thumbnail, then Lightroom will read the raw data and create its own thumbnail, which may have a different appearance.
How does Lightroom work with raw camera files?
Lightroom interprets the raw data and makes image information out of it. But every camera has a unique sensor and thus has a unique form of raw file. Even though camera manufactures may use the same file extension each time (e.g. Nikon raw files are always .NEF), each camera model has a different way of storing information. Because of this, we must continually update Lightroom for new cameras. As we add new cameras to the list (over 300 now) we update our published list of supported cameras.
Will the raw files from my camera work with Lightroom?
To determine whether your camera is supported by Lightroom, check this list:
Supported cameras
Go to Help > Updates to install the latest release. You can also manually download and install Lightroom updates:
Macintosh
Windows
What if I own a previous version of Lightroom?
Supposing you have a camera that came out after the release of a newer version of Lightroom, so that your version of Elements is no longer recieving updates, you have two basic options:
Purchase the latest version of Lightroom that supports your camera.
Use the latest version of the free DNG Converter (for Macintosh or Windows)
You can convert your files to the universaly compatible DNG format (the files will work with any version of the Lightroom). See In depth : Digital Negative (DNG) for more information.
For more assistance on DNG:
Adobe TV - The Advantages of the DNG File Format
Adobe TV - On Using DNG in Lightroom
Adobe TV - DNG Enhancements in Lightroom 4
Can I perform tethered capture from my camera to Lightroom?
Lightroom has the ability to automatically import images from your camera, as you shoot, life. To learn more about setting up and useing tethered capture, please see Import photos from a tethered camera. This list shows all the cameras that Lightroom supports for tethered shoooting:
Tethered camera support | Lightroom 4, 3
Use this document if you run into trouble when performing tethered shooting:
Troubleshoot tethered capture | Lightroom 4, 3
For more assistance on Tethered Shooting:
Adobe TV - Import photos from a tethered camera
Will Lightroom help me automatically compensate for the image distortion caused by my camera lens?
The Develop module of Lightroom includes Lens Correction which can read what lens was used from the metadata. With this information it can pick a profile that will reduce distortion caused by the curvature of the lens. For a list of which lenses are supported by which camera bodies, see this list:
Lens profile support | Lightroom 4, 3
If your lens is not currently supported, you have a few options:
Apply the lens correction manually
Use the Adobe Lens Profile Downloader (Download: Mac, Win) to see if someone else has created a profile your camera and lens
Use the Adobe Lens Profile Creator (Download: Mac, Win) to make your own profile for your camera and lens.
For more assistance on Lens Profile Corrections:
Adobe TV - Lens Correction & Perspective Correction
Adobe TV - Create Stunning Images
Adobe TV - Correcting Perspective
Adobe TV - Apply Lens Corrections
Julieanne Kost's Blog - Lens Correction Profiles
Additional resources
For more information about Adobe Camera Raw, please check these links:
Troubleshoot Camera Raw
Why doesn’t my version of Photoshop or Lightroom support my camera?
In depth: Camera Raw
Lightroom Jourrnal (developers blog) about Camera Raw releases
Julieanne Kost's Camera Raw video tutorials
Adobe Camera Raw turns 10

Yes, junky thumb drives pour like rain. Verify same with another or a good thumb drive.
Whats the format of the USB stick?
Use Sandisk or other reliable drives.

When I import TIF and JPG files with same metadata, only the TIF files are imported. How can I change that?

I have imported and renamed TIF and JPG files with the same metadata re recording date and time. Only the TIF files are imported, the JPG files remain unimproved. Thus I have in the Finder e. g. 13 image files in the same folder in which Lightroom only finds 10 files. Any ideas for resolution welcome!
Thanks

Turn off the option Preferences > General > Treat JPEG Files Next To Raw Files As Separate Photos. Then do an import again. It appears that, in this instance, LR considers a TIFF to be a "raw" file.

Will somebody please post an Aperture-written xmp file *with* custom metadata in it - thanks.

It is my understanding that Adobe's migration tool will NOT support transferal of custom metadata from Aperture to Lightroom.
I have recently enhanced my Custom Metadata plugin to read custom metadata from Expression Media xmp, and it seems like a no-brainer to have a "preset" which can read/import/transfer custom metadata from Aperture xmp too - I just need a sample to work with - thanks.
If you don't want to post here, please send directly to me.
robcole.com - Contact Me
Rob

To attempt a wrap-up for this thread:
Aperture does NOT provide a means to save custom metadata in XMP - the initial premise of this thread was incorrect: a thank you to John Beardsworth for setting me straight. Specific question asked has been answered.
The purpose for the thread however was to learn how to get information out of Aperture so it can be gotten into Lightroom (final destination: custom metadata fields).
Here is one way to get custom metadata in Aperture to custom metadata in Lightroom, recommended if you have enough free IPTC fields to house the custom data temporarily:
* Create an applescript file with the following contents, modified to support your custom metadata fields; note: it must also be modified to specify the IPTC fields to be used:
tell application “Aperture”
    set imageSel to the selection
    repeat with i from 1 to count of imageSel
        tell library 1
            tell item i of imageSel
                set customFieldValue to value of custom tag "myCustomFieldName"
                set value of IPTC tag "myFreeIptcFieldName" to customFieldValue
            end tell
        end tell
    end repeat
end tell
Note: you must adapt script depending on your custom fields, and which IPTC fields you have free.
* Select photos in Aperture.
* Execute the script.
* Once photos are in Lightroom, with IPTC fields populated.., then in Lr's plugin manager, with 'Custom Metadata' plugin selected, click 'Transfer Metadata' button, then "follow the yellow-brick road" (you will be prompted with instructions..). Metadata will be transferred from IPTC fields to custom metadata in Lightroom.
If you do NOT have sufficient IPTC fields to use, or would rather not use IPTC fields for this, then you'd have to modify the script to populate a row in a CSV file for each photo, then use the 'Import from CSV' function in Custom Metadata plugin. I can help if it comes to this..
PS - There may be other ways as well, but maybe this is enough..
Rob

Bulk insert files into sharepoint with metadata

Hello everyone,
I am creating an application which will migrate about 16k documents and their metadata from a legacy system to SharePoint.
I got the extract of the legacy system in a way that I have a folder containing the document (pdf file) and an xml file with the metadata per document.
I created the whole thing to be able to do the upload and it works fine, but on the other hand it is damned too slow...
I splited the operations, first I upload all the files and then I update the metadata of the files. I also splited the files in batches of 1000 items.
The file uploading takes more time as the list is filled. At the beginning I needed 15 mins for 1000 files, now that the list has already 3000 files in there, it takes about an hour...
I do check if the file already exists before uploading it because I need to report an error if the file was already in the list (duplicates detection)
Is there anyway to improve the performance of the system?
I also have another issue which is the fact my tool to migrate the files is taking more RAM as the list grows. After the 5000th file my tool is using over 1GB of RAM. Could it be because I use a single SPSite instance for the whole upload? Should I recreate
it during the upload?
Here is the code I use in order to upload the files:
using (var web = _currentSite.OpenWeb())
var library = web.GetList(libraryName);
var relativeFileUrl = string.Format("{0}/{1}", library.RootFolder, fileName);
if (web.GetFile(relativeFileUrl).Exists)
throw new InvalidOperationException(string.Format("The file '{0}' already exists", fileName));
var file = web.Files.Add(relativeFileUrl, fileStream, false);
Thanks a lot for your help!
With kind regards
Carlos

Hi,
So after a lot of digging the logs and so on, my application was not responsible of the lag on the upload.
Basically the list I was uploading to was using an event receiver which had a memory leak and was not using the most performant methods in order to retrieve data etc...
The lags came from the event receiver. Shame is I was the one who developped the event receiver hahaha :)
Anyway, once the event receiver was fixed, I got much better results for the upload and an upload speed of about 600 files per 10 mins which is totally acceptable for me!
I also learned in the way the two following facts that might interest people in this situation:
Event receiver code is loaded in the context of the console application process doing updates to the list. I thought at first the event receiver would be called in a RPC fashion and would live in the web application process. My mistake :)
My event receiver is responding to the ItemAdded and ItemUpdated events which are assynchronous. Once the console application ends its processing, it shuts down all the threads created by the event receiver even if they did not ended their work!
As there is no way to check if the event receiver ran or not in the code, and after searching on the web for a couple of hours, the only way to solve this is to put a Thread.Sleep at the end of the console application in order to let the event receiver thread
pool threads finish their work and avoid killing them.
Normally the wait time should not be too long except if the event receiver is really doing an heavy job or suffers of a huge memory leak (sounds familiar... :p)
Hope this will help others solving their issues.

SSIS - Import Multiple flat files with different metadata

Hi ,
I have set of flat files with different metadata structure, I would like to load them into staging tables.
1. ) Can we load the flatfiles into the staging tables with out having multiple data flow task.
2.) If possible , can we programmatically select the staging table based on the metadata of the flatfile and load them.
Please advise.
Thanks
Thiya

Nope in SSIS a data flow task needs to have a fixed metadata. So if your file metadata varies then best option would be use OPENROWSET syntax to pull the data and populate into your staging table. You may also use
SELECT .. INTO StagingTable ... FROM OPENROWSET (...)
syntax to create staging table at runtime based on the file metadata
http://sqlmate.wordpress.com/2012/08/09/use-your-text-csv-files-in-your-queries-via-openrowset/
If you want to do this in SSIS you need to create data flow dynamically using script task and build the metadata
see
http://www.selectsifiso.net/?p=288
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

How can I import an Aperture Catalog into Lightroom and retain the RAW file as well as the files with the edits?

I have several catalogs in Aperture that I would like to import to Lightroom 5 and I want to retain the original RAW files as well as the files with the edits. How can I do this?

Well, you can bring in the raw file (without edits), and you can bring in a rendered RGB file (e.g. jpeg or tiff) with edits baked in, but what you CAN'T do is bring in a raw file, with the non-destructive Aperture edits, and have Lr translate those Aperture edits into Lightroom edits.
Put another way: no raw converter/editor can understand the edits of any other raw converter/editor. So, you have to work with a rendered version, and/or re-edit from scratch in a new raw converter/editor.
PS - it would be feasible to write a rough translator which approximated raw edits in one world into edits in another, but such does not exist yet for Aperture -> Lightroom, that I know of.

Smart Collection: How to select files with unassigned/unknown/null metadata?

Is there an easy way to set up a Smart Collection to show all files with unassigned/null data in a specific metadata field?
Example: Smart Collection to show all files where IPTC "City" is unassigned (null/ blank)
I currently have a mickey mouse way of doing it that works, by selecting files NOT containing a,b,c,d,e,f,..... in the chosen metadata field, but can't beleve a more elegant (built-in) solution doesn't exist for unassigned/null values
thks!

"I will release a new plugin, Any Filter, in a day or two, that, among many other things, will allow you to search for fields with no values."
that sounds great, thanks, John!
i know nothing about the LR internal code, but would imagine that having to check each field's text string against 36 separate conditions (a-z, 0-9) as i'm currently having to do is not nearly as efficient performance wise as just testing against a single null flag or indicator - especially in large catalogs (i have about 120,000 files in mine)

Archeive Lightroom Files with metadata

Similar Messages

Maybe you are looking for