Duplicate file detection

No matter what import settings I select when I import files from the card reader the importer re-names the file to "filename +#" and imports it again.
I eliminated all copies of the "new" pictures from the organizer and hard drive. I then imported the files and it never recognizes that they are duplicates. I leave some photo's on my camera. How can I get the "duplicate detection" to work?

I'd like to recommend using this duplicate file finder to deal with unnecessary files on your hard drive. It has many functions, user-friendly interface, safety options and 100% efficiency.

Similar Messages

  • Duplicate file detection and removal across multiple volumes?

    hello there
    i use decloner, for duplicate file removal.. it works great for removing duplicates on the same volume, unfortunately, i have a mac pro with multiple hdds and want to run a duplicate search which includes every hard drive, which decloner doesn't seem to do. (some of my hard drives have identical folders and files)
    can anyone recommend me a piece of software that does duplicate removal across multiple volumes?
    many thanks!

    My boss sez "Whew"
    My backup email admin sez "Whew"
    I sez "Whew and Thank You very much"
    Originally Posted by Dave Parkes
    Yep, it is possible for that to happen, FIDs should only be unique within
    a single post office.
    As for moving a user, if their current fid already exists within the
    target po, then they will get a new unused one as part of the move process.
    So, there is nothing really to fix
    Cheers Dave
    Dave Parkes [NSCS]
    Occasionally resident at http://support-forums.novell.com/

  • How to automate removal of Duplicate files

    Is there a way to automate the removal of duplicate files with LR2?
    In my catalog of 62,000 files I have 7,300 that I believe are duplicates.
    They are proprietary RAW files (CR2). I have the same files as DNG. At least I think I do.
    My intention is to have all my RAW files as DNG only.

    If you believe all of you CR2's are duplicates, just filter All Photographs by file type and delete all Raw's.
    If you have unique CR2's (that have not DNG equivalents) you'll ned a third party utility that is able to detect duplicates ignoring the extension. Delete duplicates from drive, then use Synchronize Folder to remove missing files from catalog.

  • Removing iTunes duplicate files

    I'm not sure exactly how it happened, but most of my songs in iTunes have a duplicate file. For example, when I select "Show song file" the folder where the songs are stored looks like the following. The file ending ..1 is the one that is being used in each case.
    01 How Am I Different 1.mp3
    01 How Am I Different.mp3
    02 Nothing is Good Enough 1.mp3
    02 Nothing is Good Enough.mp3
    ...and so on
    First, any ideas how this might have happened? I do remember consolidating the library sometime ago - might that have done it?
    Secondly, how can I safely remove the duplicate files without corrupting my library or losing music?
    Thanks!

    I've got over 900 dupes like this. I looked at each pair, and the only difference is the modification date. All mp3 tags are the same, as are the compression params and the Fairplay params.
    In my case these dupes were created when I imported large partions of an almost-but-not-quite identical library from another computer, in attempt to do a manual sync between two iTunes libraries on two different machines.
    iTunes seems to be a bit too conservative in it's criteria for detecting differences during an import. Clashing mp3 tags is reasonable, but clashing mod dates when there's no other differences is too much. I'm going to file this as an iTunes bug.

  • "duplicate headers detected from server"

    Hi, Chrome suddenly doesnt show my muse site, saying:"duplicate headers detected from server" and "ERR_RESPONSE_HEADERS_MULTIPLE_CONTENT_LENGTH",( but only in some computers) and in Firefox says "corrupt content" ( but only in some computers). Internet Explorer no problem.
    Help me please
    The site is: www.soyunamilanesa.com.ar
    Thanks a lot
    Pablo

    William,
    In Mail 3, unlike earlier versions, there is one file for all accounts that prevents duplicate downloads -- in earlier versions there was one per POP account. This file most likely has become corrupted. Quit Mail, and in the Finder open Home/Library/Mail and locate the file named MessageUidsAlreadyDownloaded3 and drag it out of the Mail folder. For good measure open each POP account folder, and if you see other files with similar name (ending in 2 or no other character) delete those. Now relaunch Mail, and endure one more round of again downloading duplicates, but this time a new MessageUidsAlreadyDownloaded3 will be created, and future results should be good.
    Ernie

  • Itunes & duplicate files

    I have just deleted over 20GB of duplicate files from my music folder on my PC, these have been created by Itunes, does anyone know why Itunes creates duplicate files and how to stop it?
      Windows XP Pro  

    I've got over 900 dupes like this. I looked at each pair, and the only difference is the modification date. All mp3 tags are the same, as are the compression params and the Fairplay params.
    In my case these dupes were created when I imported large partions of an almost-but-not-quite identical library from another computer, in attempt to do a manual sync between two iTunes libraries on two different machines.
    iTunes seems to be a bit too conservative in it's criteria for detecting differences during an import. Clashing mp3 tags is reasonable, but clashing mod dates when there's no other differences is too much. I'm going to file this as an iTunes bug.

  • Apple script to find all duplicate files

    Is it possible to write an applescript to find all duplicate file names on my hard drive? My hd is almost full, and if I can delete duplicate files, it'll save me from having to buy another external drive.
    Thanks,
    George

    Why would you have found 20GB of duplicate images?  iPhoto has built-in duplicate detection, it can remove any duplicates or prevent them from being imported altogether.  Did you disable this in iPhoto for some reason?  Or were you still using folders and subfolders to store your photos?
    Just because two pictures have the exact same filename does NOT mean they are identical.  If this application you used simply compared their filenames, it's possible you actually lost all those photos and not just some duplicates.  ;(  Because many cameras reuse filenames after a certain point! 

  • Duplicate File Format options in Save As Dialog

    I am getting duplicate file format options in my save as dialog. I am running photoshop 12.0.1 on mac os x 10.6.4.

    Thanks for the answer, I have the same problem, happened after I uninstalled CS4, (using cs5 now) however I can't seem to find where the duplicates are located. Any tips as to where I should look?
    Thanks in advance!

  • Donu00B4t load duplicates File to a cube through DTP

    Hi gurus... i want to know if is possible to avoid load the same data from files to a cube.. doing this trough a DTP.
    I think to do this with a step in a process chain that eliminate dupplicates entry in the cube, but the problem is that i haven´t any selection in DTP... this is a full load.
    Regards..

    Neo,
    do you want to avoid loading duplicate files to a cube using  DTP or avoid loading duplicates from the file to the cube ?
    if it is avoid loading duplicate records from file to cube - then ideally you can use a DSo in between...

  • Insane XML Import, Huge Project, Duplicate file names work around...

    I planned on kicking off my journey attempting to edit a massive multi year documentary on FCPX with a nice introduction of the blog I'm going to keep about the experience, but I've run into a bit of a roadblock, or maybe major speed bump at least before even getting to that point. I wanted to share what is working as a work around for me and you guys can tell me how I'm doing it wrong.
    Ok, I will try to explain this as succinctly as possible. I'll write in somewhat stream of consciousness just to try and get through it quicker... Basically, after discovering the work around below, I am now utterly confused on how FCPX handles the relationship between its own database of where media is stored, and the actual media itself. I have plenty experience on FCPX now, probably done 30-40 pro commercial jobs on it over the last year since XML became doable as I'm also a Resolve Colorist and all the FCPX projects where hardcore coloring product spots. For commercial work, I never needed to worry about splitting up footage up over multiple Events. Everything, all in one, FCPX handled it no problem. (well the occasional beach ball, but that seems to be a thing with FCPX in general)
    This will be my 10th feature documentary as an Editor. Every one before it was either on Avid's many flavors over the last 12 years or FCP Studio. When this new film came along, I made the decision a few months ago to use FCPX for a few reasons, but mostly because I'm insane and I like to try to mix it up for myself in a career that can get stale quick if you aren't willing to be that way. The film was shot over 2+ years, every shoot was multi cam 5D (yes i know, looks great, but please kill me), I haven't done the math on length, but there is over 10,000 clips of video (this is actually medium in size compared to what I've dealt with before). Its 5D, so theres external audio for everything. FCPX's syncing is great, but I've learned that theres an unsaid window of heads and tales clips must fall within to sync properly with the nearby clips, if they are too far apart FCPX gives up. One shoot day could have 3 cams, 50 clips each, and 2 audio files to sync to, FCPX simply cannot handle this, so off to Plural eyes they went, no problems.
    Ok, all this is relevant eventually I swear! Again, in the past, all in one event, no problem. I tried for fun to bring all media into one Event on this film. It worked, but there is a 10+ second spinning beach ball for every single move you make, so thats no good. Ok, I'll separate the Events out to, lets say, each shoot month. Well that's dumb, in verite documentary, any shot could be the first, any shot could be the last, you need a command over all searchable footage at all times. Shift selecting all events to search *****, and it actually takes longer for FCP to reload each event each time than it does to just have the one massive one. So no go there. Next hair brained idea... What if make a new Event that is just Compound Clips of all the other Event's contents and do more with Markers and Favorites in logging that I was planning to parse it all out. That way I'm working with and FCPX is dealing with 50-60 clips instead of 10,000+ Quick test, Cmd-A, Opt-G, boom, boom, boom, move all to dedicated to Event, hide huge Event, BEHOLD, that works! FCPX chokes a little bit on the insane length of some of the clips, but searching, and general performance is back on par!
    So your saying to yourself "Ok *********, sounds like you figured it out, what's the problem." Well to you I say, "Not so fast!" Remember, that was just a quick test using the media I had imported into the massive 10,000+ clip Event. To do this project proper, I am having to import Multicam sync'd XMLs from Plural Eyes. And this is where it all starts to fall apart. A little foreshadowing for your eager eyes. 10,000+ files all shot with multiple 5D's over the course of years. What does that mean? many, many duplicate file names!
    FCPX as well all know irritatingly imports XML's as new Events, not into existing ones. This obviously takes a lot of burden off media management because with a new Event comes a new database referencing its own version of the raw media. All well and good, and I'm betting its starting to click for some if you advanced users where I'm finally going with this. So I have 50 or so XMLs to bring in, all done no problem. Now I want to replicate that singular Event like I did with the Compound Clip test and have it all in one place to be my master as extensive logging begins for easy searching once editing begins. Highlight the Events, click Merge Events. NOPE. I get a new "Kill Yourself Now" error (a term I coined for Out of Memory and General Error messages in FCP Legacy meaning there ain't much you can do about it): "Two or more files have the same name. Change the names and try again, because I don't know what the **** to do with them." Ok I made up that last part but that's basically what it's saying. Just take the variable out of the equation, this happens with every which way you could try to get the clips together. Merge Events, dragging events on top of each other, dragging just the Multicam clip alone, nothing gets passed that message. What's worse is that while Batch Renaming seems like a solution, the renames do not populate inside the created clips and there is no way to Batch Rename those. Renaming everything at the finder level isn't so great because then I'd have to resync and theres an offline/online thing going here where the film has to be reconformed eventually.
    Basically, I've found that FCPX handles media management in completely different ways depending on whether you are importing into one Event yourself or doing essentially what is a new import with FCPX moving or merging Events. If you bring in all the media to one Event, on a macro level FCPX goes through file by file making aliases referencing the master file. If it hits a duplicate, it makes a parenthesis counter, and keeps going. And with the genius of FCPX metadata, that file name doesn't even matter, you can change it at the Finder level and FCPX will keep the link intact. BUT for some reason if you try to do this outside the realm of a single Event and combine files after the fact a different process takes over in creating this database system and can't handle the duplicates. I can't totally figure the reason other than it probably is scared to change the originally referenced alias to something else (which it shouldn't be scared of since Merge Events deletes the original anyway).
    This is where it gets INSANE and where I lose all understanding of what is going on, and believe me it was a delicate understanding to begin with. It comes in to play with the work around I figured out. I make the master Event with the 10,000+ clips. Then I import all the XMLs as dedicated Events. Now, I then drag the Multicam clips into the master Event, it WORKS! just takes it, no "Kill Yourself Now" error. Stranger still, now with the switched referenced Event, it even takes into account which aliased duplicate file name it's referencing in the Original Media folder. Somehow, it's taking into account the original file path and saying "Ok, I see 5 instances of MVI_5834.mov. Based on the original file path or maybe creation date or god knows what, it must be MVI_5834 (fcp3).mov." It connects perfectly. I can even remove the old XML imported Event with no problem. Crazier yet, I can now move those again to the dedicated Event I wanted originally that only contains those Multicam or Compound Clips.
    So instead of going straight from A to C, going from A to B to C works even though that actually seems way more complicated. Why can't FCPX handle Merge Events and dragging clips the same way it handles media imported into a single Event. And weirder still, why can't FCPX handle the (fcp1,2,3...) appending in the same scenario. But if the appended links are already there, No Problem. And for the love of god, it'd be nice to important XML's into existing Events and make the correct referencing happen right from the get go, which is really the source of all the above headache. I'd have no problem helping FCPX with a little manual pointing in the right direction just like any other NLE needs.
    Ok, having said all of that crap above, my question is, have I missed something completely simple I should have done instead? Now that I have everything in place how I originally envisioned, I think I will still play around a little bit more to make sure FCPX is really going to be able to handle this project. I'm at a stage right now where going another direction is still an option, although the dare devil in me wants to make this work. Media management aside, once you start editing on a FCPX timeline, its hard to go back to anything else. Apple is going to have to figure out some way not to access to everything at all times to work fluidly or big projects like this are never going to be practical in FCPX.
    Sorry for the long confusing post....

    I'm having the exact same problem, but I know nothing of ruby scripts. I've exhausted my resources with tech support, and after literally hours of work with them it seems I now am faced with either re-rating each individual song, or pointing iTunes to each song that it can't locate. Is yours a solution that could help me? How can I find out more about Ruby Scripts? Thanks for your help, hellolylo (or anyone else who might also be able to help...)
    Kenn

  • HT1414 the iphone software update server could not be contacted.a duplicate file name was specified , please help as this is driving me nuts,does anyone know which file it could possibly be ?

    Hi could anyone help me out? my iphone 3gs keeps restarting itself every two minutes which i think is some kind of software issue,now there are a number of things i have done starting with putting in a new battery.when i charge the phone it resets itself every two minutes or so and the battery indicator stays at 4% so i decided to restore the phone on itunes (which is the latest edition) and when i connect my phone which i have put into recovery mode using the buttons, i get an error message saying :- the iphone software update server could not be contacted.a duplicate file name was specified. i have tried re-installing i tunes and that doesn't solve the issue and i have trawled through the program files to see if i could spot the duplicate but without success ,has anyone had the same issue and resolved it?

    An iPhone 3G cannot be updated beyond iOS 4.2.1...that's the end of the line for your phone.

  • How can I easily delete duplicate files in old Time Machine archives?

    I have multiple older Time Machine archives from different Macs that I have been going through manually and picking files out and copying them to another drive file by file. I'm running into massive quantities of duplicate files and the biggest problem is muliple iphoto pictures including the same picture in different sizes.
    I wanted to give up and just throw the whole Time Machine archive onto another disk so I could format the one it was on and use it for something else. But I got a message reading, "The backup can’t be copied because the backup volume doesn’t have ownership enabled." So I went to manually copy each "Macintosh HD" folder for each date into it's their own respective folders. It worked but required my password for each folder and each of the 19 folders were all ~75GB. Which is impossible because all of that data was on a 500GB drive.
    What should I do? This has become a two month long nightmare.

    I really don't know exactly, but since no one else has a answer I'll give it a shot.
    I don't know much about TimeMachine at all, however I did assist someone that their TM was corrupted and they needed to salvage what they could off the drive, so I'll start with that.
    There is software called Data Rescue and it's designed to read the 1's and 0's of any data on the drive, deleted or not, regardless of the file structure, and output the results to another drive.
    TimeMachine tries to save space, so it uses what appears to be duplicate files, but contain no data as they are placemarks for files in later states that went unchanged from the previous states.
    If Data Rescue has the option of recovering files independently of the file/folder structure this would result in all your files from the recoved TM drive being on all one level on the second drive.
    From that second drive one could run software called DeCloner, which would flag the actual duplicate files of content, not just by name.
    The next stage perhaps would be to eliminate files with no data, the placeholders, using the Finder "size" as a sorting option.
    Saving all this remaining content to another drive to work on further, perhaps all the remaining image files can be seperated, sorted according to largest size, the largest ones (usually the ones you want to keep as they contain more quality)  and opened into iPhoto or Aperture for further refinement as you add a small batches of smaller image sizes in to compare with the larger versions for duplicates/unwanted sizes.

  • HT1660 i have two independent itunes libraries on my computer right now. how do i consolidate them into one library? how do i get rid of duplicate files?

    i have two independent itunes libraries on my computer right now. how do i consolidate them into one library? how do i get rid of duplicate files?

    Hello rmunsey,
    Thank you for the question.  It sounds like you have two separate iTunes Libraries but would like to add the music from one to the other to have one consolidated library.
    If you know where the files are for each library, I recommend opening one iTunes Library in iTunes and importing the files from the other library using the following steps:
    Open iTunes
    From the File menu, choose one of the following choices:
    MacAdd to Library
    Windows
    Add File to Library
    Add Folder to Library
    Navigate to and select the file or folder that you want to add
    Adding music and other content to iTunes
    http://support.apple.com/kb/ht1473
    Next, you can search for duplicates:
    Choose View > Show Duplicate Items to show duplicate items (matches are based on the song name and artist). If you have multiple versions of the same song (for example, live and studio versions, or versions from different albums) you can hold the Alt or Option key (for Mac OS X) or the Shift key (for Windows) and choose View > Show Exact Duplicate Items. This will show only duplicate songs having the same name, artist, and album.
    How to find and remove duplicate items in your iTunes library
    http://support.apple.com/kb/ht2905
    This should help you identify the duplicates all at one time so that you can delete the extra versions of songs.
    Thank you for using Apple Support Communities.
    Best,
    Sheila M.

  • ITunes and I seem to have created duplicate files.  How do I clean this up?

    Hello experts.  I have a conundrum I'm too inexperienced to reason out.  Hoping to tap your creativity.
    I have a huge iTunes library I've created over the years.  50,000+ songs.  No problem, other than my desire to whittle through it slowly and pare it down to only the 22,000 or so I want to keep as my primary library.  I haven't gone through the effort of creating multiple libraries yet, I simply delete songs or albums from the library as time permits without deleting the original file.  I figure iTunes' database will be my primary and in the long-game, I'll create a new library that holds everything, but for now I'll just keep whittling the dB structure away until I get what I want (this is a slow, multi-year project I'm undertaking -- no rush here).
    Here's my problem:
    Most of these songs are physically saved in the iTunes folder on an external hard drive.  Some of the songs I added to my library from the iTunes store when the external hard drive wasn't connected to the Mac - so they are on the Mac's hard drive under the standard iTunes folder structure.  I have the "Keep iTunes Media folder organized" and "Copy files to iTunes Media folder when adding to library" checkboxes checked.
    As I said, I've deleted songs from the iTunes app, but told it to "Keep Files" so that I could listen to them in the future if I decide to re-add them.  Yesterday, I ran out of space on the startup drive of my mac, so naturally, I figured I could consolidate any files on my hard drive over to my external hard drive and delte them from the mac. 
    In iTunes, I went to File -> Library -> Organize Library.  I checked both boxes for "Consolidate files" and "Reorganize files in the folder..."  I let this run overnight to consolidate my library into one place.  Today, I found that the function was unable to complete due to lack of disk space on my external hard drive.  When I went to check the external hard drive, I found that iTunes had duplicated all of the songs that were on the external hard drive in their existing folders. 
    For example, now I have a file under the structure:  iTunes library -> Music -> Blue Oyster Cult -> Heaven Forbid -> Cold Grey Light of Dawn.mp3
    And I also have a file under the structure: iTunes library -> Music -> Blue Oyster Cult -> Heaven Forbid -> Cold Grey Light of Dawn 1.mp3
    So now the immediate problem...  The consolidation didn't finish, so I don't know exactly how much duplication iTunes created.  I checked my library, and sure enough, the song in the library is pointed at the duplicate file (the one with a 1 at the end).  I don't have the option to just copy the iTunes library to another external drive temporarily (or not easily, at least).  I checked out a bunch of duplicate file cleanup programs, but the ones I found (and I even purchased Gemini's dupe file finder/cleanser) don't seem to inherently choose the correct duplicate file that is linked to iTunes library dB.  I can't realistically pick and choose from the thousands of files that have been duplicated.
    I decided to change the advanced preferences in iTunes to an entirely new folder on the External hard drive.  Then I could just go back into File -> Library -> Organize library and select only the "Reorganize files..." checkbox without selecting the "Consolidate files" checkbox.  I figured this would just move all the files that are in the iTunes dB to the new iTunes folder.  Ummm.... but no.  Apparently this "Reorganize files" checkbox is only available if you select "Consolidate files"...?   To be clear, "Reorganize files" is greyed out.
    So to summarize:
    1.  Why did iTunes create copies of existing files in the exact same location they were in?
    2.  What is the right (and I pray easy) way to get my library back to the right size and consolidated into one place?   I'm still dangerously low on disk space on the mac...
    <sigh>
    Thanks for reading through all this! 
    Tim
    Mac Mini  OSX 10.8.3; iTunes 11.0.2 (26)

    See this thread for background on why iTunes can end up with duplicates and how they can be systematically removed. The script there is for Windows users and the "show duplicates" menus have moved around in iTunes 11,  but the advice on clearing recently made duplicates should still hold.
    As given in my older post "Reorganize library" is meant as a one time command to change the structure of a pre-iTunes 9 library into the newer layout that places music in its own subfolder of the media folder. Once used, if the option becomes available again that suggests that a file called .iTunes Preferences.plist in the media folder has been deleted or editied.
    See also this post.
    tt2

  • Create an iPod or iphone version - is there any way to avoid duplicate files OR automating a new name for the new file?

    hi,
    I have two devices for my videos. An ipod classic and an Ipad. I would prefer to keep the highest resolution possible but also will be travelling and it will be useful to have a large archive with me.
    I have found that many of the videos I have are "incompatible" with my ipod classic and am in the process of using the "create an ipod or iphone version" but this results in a duplicate file in my itunes library that is identical in everyway (except size and this can be bigger or smaller).
    When you buy a HD video off itunes, you get both versions but only one file populates in the itunes libary.
    Is there anyway that when itunes "creates the ipod or iphone version", it can do automatically merge these two files so you only see one file in your library? And then it will automatically sync the appropriate file with the appropriate device?
    Or is there a way to have itunes "create the ipod or iphone version" and have it automatically label it with the name "XXX (ipod version"?
    I am finding it very annoying to manually change all the titles on all the files being created.
    thanks, margaret

    When you open a Word document with Pages you get a translated-to-Pages file & the original Word document is unchanged. There are programs that will save over the original file such as LibreOffice. Pages will not.

Maybe you are looking for