Removing Duplicates in LightRoom

When I migrated from iPhoto to LightRoom I wound up with many hundreds, if not thousands of duplicte images.
I purchased the "Duplicate Finder" plug-in, which indeed finds the duplicates, but ther eis no way to simply delete them. I would imagine many users have been through this at some point and hope there is a relatively simple way to get my catalog under control.
Thanks,
Henry

You'll need to decide which are the ones you want to keep. If you're asking whether you can delete from the Smart Collection, without first finding them in the folders, then yes you can. You can use the splat-delete (Cmd-Shift-Opt-Delete or Ctrl-Shift-Alt-Delete) to delete directly, but be warned it skips all warning dialogs.

Similar Messages

Duplicates in lightroom

Topic
duplicates in lightroom
Leni - 05:54pm Mar 1, 2008 Pacific
Just imported photos into lightroom and found many duplicates. is there an easy way to get rid of duplicates?
I am looking for an automation method of doing this.
Imac 24, 2gigs ram, 750mb h.d., 2.8Ghz intel processor using Photoshop Lightroom 1.31

I just had to restore from a failed Buffalo RAID unit, and was a bit shocked to find that, when I reimported with 'remove suspected duplicates', it hardly worked at all, and I wound up with all the 'missing' entries still there, and duplicates of most everything.
So I tried 'locate' for a missing one, thinking find other missing nearby. Fails, because it won't allow two images with the same name (although there are two), and won't look for others.
I expect better tools from Adobe in Lightroom. I'm using Mac, and as a pro photographer, this will lead me to see if Aperture from Apple is smarter.
Adobe should provide smart search capabilities to help us weed out duplicates. What is there now is not very helpful.

Best app to remove duplicates from iTunes 2014

Hi All,
I've been trying to research the best application to sort and remove duplicates from my iTunes library. I have over 7000 songs and iTunes built in duplicate finder doesn't look at the track fingerprint, which is useful for those songs which are labelled "Track_1" etc.
Has anyone reviewed any recent products? I was looking at TuneUp, but after reading so many negative comments, I've decided not to go down that path. I would prefer a program that did most of the work for me, due to the amount of songs. Happy to pay for a good product...
I do have MusicBrainz Picard, which has done a great job of tagging, but don't remove duplicates.
Thanks in advance :-)

Tune up is a great app. When they moved from version 2 to version 3 is when it went to crap and all heck broke loose. They shut their doors but they have since re opened and went back to developing version 2. I use that version and I am pretty happy with it as being an overall cleanup utility. I also use Musicbrainz and a couple of other utilities but in the end if you have an enormous library 20k plus then you are going to have a few slip through. I would probably go with Tuneup if I were you and a thorough third party duplicate finder. Dupe Guru's music edition seems to do a pretty good job.

Removing duplicate values from selectOneChoice bound to List Iterator

I'm trying to remove duplicate values from a selectOneChoice that i have. The component binds back to a List Iterator on the pageDefinition.
I have a table on a JSF page with 5 columns; the table is bound to a method iterator on the pageDef. Then above the table, there are 5 separate selectOneChoice components each one of which is bound to the result set of the table's iterator. So this means that each selectOneChoice only contains vales corresponding to the columns in the table which it represents.
The selectOneChoice components are part of a search facility and allow the user to select values from them and restrict the results that are returned. The concept is fine and i works. However if i have repeating values in the selectOneChoice (which is inevitable given its bound to the table column result set), then i need to remove them. I can remove null values or empty strings using expression language in the rendered attribute as shown:
<af:forEach var="item"
items="#{bindings.XXXX.items}">
<af:selectItem label="#{item.label}" value="#{item.label}"
rendered="#{item.label != ''}"/>
</af:forEach>
But i dont know how i can remove duplicate values easily. I know i can programatically do it in a backing bean etc.... but i want to know if there is perhaps some EL that might do it or another setting that ADF gives which can overcome this.
Any help would be appreciated.
Kind Regards

Hi,
It'll be little difficult removing duplicates and keeping the context as it is with exixting standard functions. Removing duplicates irrespective of context changes, we can do with available functions. Please try with this UDF code which may help you...
source>sort>UDF-->Target
execution type of UDF is Allvalues of a context.
public void UDF(String[] var1, ResultList result, Container container) throws StreamTransformationException{
ArrayList aList = new ArrayList();
aList.add(var1(0));
result.addValue(var1(0));
for(int i=1; i<var1.length; i++){
if(aList.contains(var1(i)))
continue;
else{
aList.add(var1(i));
result.addValue(var1(i));
Regards,
Priyanka

How to remove duplicate items ?

ok so ive moved my iTunes library from the NAS drive I bought (after finding out that wouldnt work) onto my new laCie external drive, but ive found some of my albums have triple copies?
i know how to show duplicates but im not sure how to safely remove duplicates without deleting all copies?

Oh boy.
I'm sure there are better ways to do it than this and it will take time, but to avoid all possible loss of data, what I would do is first consolidate all of your libraries.
• Open the iTunes Library you think is most correct by holding down the OPTION key when you open the iTunes application, which should bring up a dialogue like this: http://cl.ly/image/2i2Q3o0Z0Y3C
• Then go to preferences and make sure it looks like this: http://cl.ly/image/2C2Z0u0C3T3c
I would recommend keeping your main iTunes library on your main hard drive (for me that's my internal), unless you definitely can't fit it.
• Now go to the File menu >Library > Import Playlist
• Navigate to another of the libraries, and click on the iTunes Library.xml file and import it. Do this for each of your libraries, except the one you are currently using.
• Now that you've got everything imported into the one library, the fun part starts.
Do what i said in my previous post and remove all the duplicates.
• Once that's done, check all the other libraries to make sure you haven't missed anything, and send them on their way to the Trash, and empty it to reclaim all that space.
I really hope you get this sorted, I went through an ordeal like this myself recently, so it's going to take time, but it feels good when it's all cleaned up and finished!
xeni
PS. After writing all this I thought, hmm, why didn't I just Google it instead of figuring it out myself? ;P
I found this, and it might help if my instructions weren't clear enough. https://bitly.com/LpqFPq
Also, if you don't already, I urge you to use Time Machine backup. Read more here: http://www.apple.com/osx/apps/#timemachine and http://pondini.org/TM/FAQ.html

How to remove duplicate POP messages in Mail under Mavericks

Hello
I'm having some significant problems with Apple Mail, and despite having researched widely, I haven't been able to find a solution yet. Here is my tale of woe...
My set up is that I use Mail with 5 separate accounts, downloading all messages using POP. I keep ALL my mail (even the trash) so that I have a permanent record of all conversations, going back more than 10 years - this is about 250,000 emails in around 250 subfolders. I'm running the latest version of Mavericks on a 2010 MBP. I have no intention of moving to IMAP - I need to keep a local archive of all my mail.
A few months ago I was having problems with Mail being very slow to load and search emails. I decided to rebuild the mailboxes, but afterwards discovered that thousands of messages in the inbox of one of my accounts (and some of the sub-folders) had disappeared. Once I realised I tried to go back to a recent Time Machine back up, but, for whatever reason, that part of the back up hadn't worked properly and I was unable to recover the previous status. I left it while I decided what to do, and had time to deal with it...
The last 2 years worth of emails on that account were still archived on gmail, so last week I decided to try redownloading the whole lot (something like 72,000 messages), and then planned to use one of the scripts I was aware of to remove the duplicates to the trash, and then remove any duplicates in the trash forever. That ought to mean I at least had an archive of the last two years of missing messages, even if I had no record of which ones had been flagged or replied to, etc.
The downloading took about 24 hours to complete, and I now have almost 70,000 messages in the inbox of that account. I also rebuilt all the folders again, which doesn't seem to have resulted in any further loss of messages as far as I can tell. However, I hadn't figured on two things:
1. Mail 'hides' duplicate messages, meaning that all of the messages I've downloaded are showing as unread, and I can't differentiate the ones I already had in my Inbox, and the new downloads. When I click on those apparently unread messages I can see if they have been replied to or forwarded, etc, but there's no obvious way for me to remove the ones I have already dealt with to the trash, without going through them all manually, which is clearly impossible.
2. The scripts I'd found for removing duplicates don't work. Andreas Amann's Remove Duplicates script doesn't work under Mavericks, and he has abandoned the project. I've also tried the remove-duplicate-messages.scpt made by Jolly Roger (http://jollyroger.kicks-***.org/software/), and while it *sometimes* works on individual subfolders on my Mac (but as far as I can tell removes the duplicate in that folder, rather than the newly downloaded version), mostly it doesn't work at all - it creates a 'Remove Duplicate Messages' folder on my desktop, a log inside it and a folder for removed messages, but nothing appears in the duplicates folder.
So, I'm left in a position where I have 70,000 apparently unread messages in my Inbox, a massively bloated Mail library (which has pretty much doubled in size, because of the 'hidden' messages), a slow and unresponsive Mail program. I've come to the conclusion that there must be some corrupted email somewhere, which probably caused the original email haemorrhage, and may still be causing the inability to remove duplicates. Mail is so slow as to be almost unuseable.
I figure I have a number of options:
1. I could live with the situation and just archive most of the Mail in my inbox, with the side effect that there will be a bunch of messages I have never replied to that are missed.
2. I could abandon the last week's efforts and revert to the version of Mail I was using a week ago, and then redownload the recent emails from my various accounts. That would still leave me without those thousands of emails I lost on my local machine.
3. I could find another way to deal with this. Can I get the remove duplicates script working? Should I revert to the version of Mail from a week ago, download ALL the messages again, but do it in a way that allows me to find the duplicates and remove them? Should I move to another email program altogether (which would presumably be massively disruptive to my work!)
Anyway, apologies for the essay length of this request, and thanks in advance for any help you may be able to offer!

hi Eric
thanks for the tip, but I don't think that solves my problem, which is that Mail apparently hides the duplicates, so they're still there, you just can't differentiate or remove them. I actually *need* the duplicates there so I know I've got everything, but then I need to be able to move the ones I know I've dealt with from my inbox to an archive/trash, and/or remove them completely so Mail isn't totally bloated

I have more than one photo backup on my computer now itunes has downloaded three of each photo. How do I remove duplicates

I have more than one photo backup on my computer now itunes has downloaded three of each photo. How do I remove duplicates?

Wich version of iPhoto do you have? You need iPhoto '09 or '11 to order prints.
Make an album of all photos you want to order in the same order.
Then select all photos at once, when you use Share > order prints.
You should see a panel like this, where you can mark the quantities. The screenshot is from iPhoto 9.6:

HT2905 how do you remove duplicate libraries from itunes

I recently purchased a new window9.1 computer. I tried to move the library to the new pc by using an external haddrive. The songs and other items all transfered, but the mudic remsined on the externalmedia informaiton files, I had to connect the drive to listen to music.. I worked with tech support and copied themusioc files to ITunes. When I reopened the media files and copied music were on the system. ITunes recognzed there was missing information on the media data file and corrected it, so oe I have 20,000 sonfs in a 10,000 song library.How do you delete massive duplicates without individually deleting10,000 songs?

Apple's official advice is here... HT2905 - How to find and remove duplicate items in your iTunes library. It is a manual process and the article fails to explain some of the potential pitfalls.
Use Shift > View > Show Exact Duplicate Items to display duplicates as this is normally a more useful selection. You need to manually select all but one of each group to remove. Sorting the list by Date Added may make it easier to select the appropriate tracks, however this works best when performed immediately after the dupes have been created. If you have multiple entries in iTunes connected to the same file on the hard drive then don't send to the recycle bin.
Use my DeDuper script if you're not sure, don't want to do it by hand, or want to preserve ratings, play counts and playlist membership. See this thread for background and please take note of the warning to backup your library before deduping.
(If you don't see the menu bar press ALT to show it temporarily or CTRL+B to keep it displayed)
tt2

How to remove duplicate songs from Itunes

I've consolidates my music in Itunes 11.01. Playlists are just getting crazy with duplicates.
I want to eliminate duplicate music files, as well as eliminate the references in the interface.
Then sync my new IPad.
When I manually erase a duplicate in Windows Explorer, I'm left with a "!" indication in Itunes that the file cannot be found... duh.
I want to find either an easier way to eliminate duplicates without Windows Explorer, or
a way to select all the "!" references in Itunes interface and delete.
Help please from a completely organized person who hates Apples "intelligence" at thinking it knows what I want, and not providing
adequate help under "remove duplicates" in its help index.
Thanks.

How to find and remove duplicate items in your iTunes library - http://support.apple.com/kb/HT2905
Posts by turingtest2 about different types of duplicates and techniques - https://discussions.apple.com/thread/3555601 and https://discussions.apple.com/message/16042406 (Note: The DeDuper script is for Windows).
May 2014 post on iCloud duplicates - https://discussions.apple.com/message/25867873
Show exact duplicates (Mac and Windows) - https://discussions.apple.com/message/16951281
http://dougscripts.com/itunes/itinfo/dupin.php (commercial)
iTunes Duplicate Song Manager - http://sourceforge.net/projects/itunesdsm/
http://www.hardcoded.net/dupeguru_me/
http://www.wideanglesoftware.com/tunesweeper/index.php
http://www.araxis.com/find-duplicate-files (commercial, free trial) - finds duplicate files on computer, not specifically iTunes
I would do it manually. As you can see from turingtest2's post, some duplicates are not really duplicates.
I would also review how you add media to iTunes. It should not be producing all that many duplicates.

How to remove duplicates in iphoto 7.1.5 and aperture 2.1.4 on same hard drive

How to remove duplicates from iPhoto 7.1.5 and Aperture 2.1.4 on same hard drive?

For iPhoto duplicate annihalitor is a good solution
For Aperture it is best to ask in the aperture forum
LN

How do I find and remove duplicates on the 11.2 itunes?

How do I find and remove duplicates on the 11.2 itunes?

duplicate annihaitor
But often duplicates are a symptom and you really need to address the cause - more information is needed to help out with that
LN

How to remove duplicate messages from Mail in Mavericks (POP)

Hello
I'm having some significant problems with Apple Mail, and despite having researched widely, I haven't been able to find a solution yet. Here is my tale of woe...
My set up is that I use Mail with 5 separate accounts, downloading all messages using POP. I keep all my mail so that I have a permanent record of all conversations, going back more than 10 years - this is about 250,000 emails. I'm running the latest version of Mavericks on a 2010 MBP. I have no intention of moving to IMAP - I need to keep a local archive of all my mail.
A few months ago I was having problems with Mail being very slow to load and search emails. I decided to rebuild the mailboxes, but afterwards discovered that thousands of messages in the inbox of one of my accounts (and some of the sub-folders) had disappeared. Once I realised I tried to go back to a recent Time Machine back up, but, for whatever reason, that part of the back up hadn't worked properly and I was unable to recover the previous status. Ileft it while I decided what to do, and had time to deal with it.
The last 2 years worth of emails on that account were still archived on gmail, so last week I decided to try redownloading the whole lot (something like 72,000 messages), and then uplanned to use one of the scripts I was aware of to remove the duplicates to the trash, and then remove any duplicates in the trash forever. That ought to mean I at least had an archive of the last two years of missing messages, even if I had no record of which ones had been flagged or replied to, etc.
The downloading took about 24 hours to complete, and I now have almost 70,000 messages in the inbox of that account. However, I hadn't figured on two things:
1. Mail 'hides' duplicate messages, meaning that all of the messages I've downloaded are showing as unread, and I can't differentiate the ones I already had in my Inbox, and the new downloads. When I click on those apparently unread messages I can see if they have been replied to or forwarded, etc, but there's no obvious way for me to remove the ones I have already dealt with to the trash, without going through them all manually, which is clearly impossible.
2. The scripts I'd found for removing duplicates don't work. Andreas Amann's Remove Duplicates script doesn't work under Mavericks, and he has abandoned the project. I've also tried the remove-duplicate-messages.scpt made by Jolly Roger (http://jollyroger.kicks-***.org/software/), and while it *sometimes* works on individual subfolders on my Mac (but as far as I can tell removes the duplicate in that folder, rather than the newly downloaded version), mostly it doesn't work at all - it creates a 'Remove Duplicate Messages' folder on my desktop, a log inside it and a folder for removed messages, but nothing appears in the duplicates folder.
So, I'm left in a position where I have 70,000 apparently unread messages in my Inbox, a massively bloated Mail library (which has pretty much doubled in size), a slow and unresponsive Mail program. I figure I have a number of options. Either I could abandon the last week's efforts and revert to the version of

Apologies, this was posted here in error. Please find updated post here: https://discussions.apple.com/message/27261965#27261965

Search for records in the event viewer after the last run (not the entire event log), remove duplicate - Output Logon type for a specific OU users

Hi,
The following code works perfectly for me and give me a list of users for a specific OU and their respective logon types :-
$logFile = 'c:\test\test.txt'
$_myOU = "OU=ABC,dc=contosso,DC=com"
# LogonType as per technet
$_logontype = @{
2 = "Interactive"
3 = "Network"
4 = "Batch"
5 = "Service"
7 = "Unlock"
8 = "NetworkCleartext"
9 = "NewCredentials"
10 = "RemoteInteractive"
11 = "CachedInteractive"
Get-WinEvent -FilterXml "<QueryList><Query Id=""0"" Path=""Security""><Select Path=""Security"">*[System[(EventID=4624)]]</Select><Suppress Path=""Security"">*[EventData[Data[@Name=""SubjectLogonId""]=""0x0""
or Data[@Name=""TargetDomainName""]=""NT AUTHORITY"" or Data[@Name=""TargetDomainName""]=""Window Manager""]]</Suppress></Query></QueryList>" -ComputerName
"XYZ" | ForEach-Object {
#TargetUserSid
$_cur_OU = ([ADSI]"LDAP://<SID=$(($_.Properties[4]).Value.Value)>").distinguishedName
If ( $_cur_OU -like "*$_myOU" ) {
$_cur_OU
#LogonType
$_logontype[ [int] $_.Properties[8].Value ]
#Time-created
$_.TimeCreated
$_.Properties[18].Value
} >> $logFile
I am able to pipe the results to a file however, I would like to convert it to CSV/HTML When i try "convertto-HTML"
function it converts certain values . Also,
a) I would like to remove duplicate entries when the script runs only for that execution.
b) When the script is run, we may be able to search for records after the last run and not search in the same
records that we have looked into before.
PLEASE HELP !

If you just want to look for the new events since the last run, I suggest to record the EventRecordID of the last event you parsed and use it as a reference in your filter. For example:
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*[System[(EventID=4624 and
EventRecordID>46452302)]]</Select>
<Suppress Path="Security">*[EventData[Data[@Name="SubjectLogonId"]="0x0" or Data[@Name="TargetDomainName"]="NT AUTHORITY" or Data[@Name="TargetDomainName"]="Window Manager"]]</Suppress>
</Query>
</QueryList>
That's this logic that the Server Manager of Windows Serve 2012 is using to save time, CPU and bandwidth. The problem is how to get that number and provide it to your next run. You can store in a file and read it at the beginning. If not found, you
can go through the all event list.
Let's say you store it in a simple text file, ref.txt
1234
At the beginning just read it.
Try {
$_intMyRef = [int] (Get-Content .\ref.txt)
Catch {
Write-Host "The reference EventRecordID cannot be found." -ForegroundColor Red
$_intMyRef = 0
This is very lazy check. You can do a proper parsing etc... That's a quick dirty way. If I can read
it and parse it as an integer, I use it. Else, I just set it to 0 meaning I'll collect all info.
Then include it in your filter. You Get-WinEvent becomes:
Get-WinEvent -FilterXml "<QueryList><Query Id=""0"" Path=""Security""><Select Path=""Security"">*[System[(EventID=4624 and EventRecordID>$_intMyRef)]]</Select><Suppress Path=""Security"">*[EventData[Data[@Name=""SubjectLogonId""]=""0x0"" or Data[@Name=""TargetDomainName""]=""NT AUTHORITY"" or Data[@Name=""TargetDomainName""]=""Window Manager""]]</Suppress></Query></QueryList>"
At the end of your script, store the last value you got into your ref.txt file. So you can for example get that info in the loop. Like:
$Result += $LogonRecord
$_intLastId = $Event.RecordId
And at the end:
Write-Output $_intLastId | Out-File .\ref.txt
Then next time you run it, it is just scanning the delta. Note that I prefer this versus the date filter in case of the machine wasn't active for long or in case of time sync issue which can sometimes mess up with the date based filters.
If you want to go for a date filtering, do it at the Get-WinEvent level, not in the Where-Object. If the query is local, it doesn't change much. But in remote system, it does the filter on the remote side therefore you're saving time and resources on your
side. So for example for the last 30 days, and if you want to use the XMLFilter parameter, you can use:
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*[System[TimeCreated[timediff(@SystemTime) <= 2592000000]]]</Select>
</Query>
</QueryList>
Then you can combine it, etc...
PS, I used the confusing underscores because I like it ;)
Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

Help needed in removing duplicate items of list box in java

How to remove duplicate items of list box while dynamically inserting (on-click event)
It is not identifying duplicate data
Variable name is HP_G1
HP_dmg1 = (DefaultListModel) HP_G1.getModel();
int a = HP_G1.getModel().getSize();
System.out.println("HP list no--------> "+a);
if(a!=0)
for (int j=0; j<a; j++)
String item1 = String.valueOf(HP_List.getModel().getElementAt(j));
System.out.println("HP list added--------> "+item1);
if(HP_dmg1.equals(item1)){
HP_dmg1.remove(j);
else
HP_dmg1.addElement(GPL);
}

Your code is unreadable, so I'll ignore it. In the future please press the message editor's CODE button to format code.
As to your problem, to the point you normally use a Set instead of List when you don't want duplicates in a collection.

HT1349 Due to backing up my itunes on an external 1 terrabite hard drive, I now have duplicates of the same song in my library. Anyone know how to run something to remove duplicates? It's taking up too much space on my hard drive.

Is there something I can run to remove duplicate songs from my itunes library. This occurred when I backed my itunes library up on an external hard drive, now I have multiple duplicates of the same songs. HELP!

I cannot find this 300GB "Backup" in the Finder, only in the Storage info when I check "About This Mac".
You are probably using Time Machine to backup your MacBook Pro, right? Then the additional 300 GB could be local Time Machine snapshots. Time Machine will write the hourly backups to the free space on your hard disk, if the backup drive is temporarily not connected. You do not see these local backups in the Finder, and MacOS will delete them, when you make a regular backup to Time Machine, or when you need the space for other data.
See Pondini's page for more explanation: What are Local Snapshots? http://pondini.org/TM/FAQ.html
I have restarted my computer, but the information remains the same. How do I reclaim the use of the 300GB? Why is it showing up as "Backups" when it used to indicate "Photos"? Are my photos safe on the external drive?
You have tested the library on the external drive, and so your photos are save there.
The local TimeMachine snapshot probably now contains a backup of the moved library. Try, if connecting your Time Machine drive will reduce the size of your local Time Machine snapshots.

Removing Duplicates in LightRoom

Similar Messages

Maybe you are looking for