Script for removing duplicates in Mail

I have thousands of saved messages, of which a great many are dupllicates. I have been able to eliminate many of them using the script provided by Andreas Amann http://homepage.mac.com/aamann/, but large numbers of others resist the script because they are not "quite" identical. For example, if one looks in the .emix files, one may find that one has charset="iso-8859-1" and the other has charset=iso-8859-1 without quotation marks. No simple script is going to catch things like that.
I don't know the cause, but one problem is that I often receive an inbox item from the Exchange server that can't be read: the message appears in the viewer, but won't open. Then, seconds or minutes later, a valid version will pop up and, perhaps, the bad one will disappear. However, I have a rule that automatically copies inbox messages to a local folder. As a result, I have two items in that folder (both of which, by the way, I can usually read). Over time, this means thousands of duplicates.
Another symptom for at least some of the duplicates is that what appear to be duplicates appear in the viewer with variations of my name, such as John Smith in one and Smith, John in the other. I can imagine that this is due to message-handling protocols in my company's Exchange-based system, but actually have no clue. Further, not everyone in the company that uses Mail seems to have this problem.
Ideas?
P.S., why can't Apple have a Remove Duplicates feature, as it does in Address Book?

I've written a script called DeDuper which can help remove unwanted duplicates. See this thread for background. I can't necessarily claim it is the best, but it is certainly free.
I have another script called FindTracks which attempts to do just that.
tt2

Similar Messages

Apple script for Message box when Mail app Compose Window Opens

Hi there
I'm trying to make mail app remind me to choose the right emaill address to send from since I have several email addresses. Is there a way I can make or write an apple script for a message box to pop up whenever mail app's compose window opens up.
I can do for the case when I actally click the compose button when I'm on mail app by using UI scripting. But when it comes to hyperlinks which opens up the compose window, it fails.
I just want a script to see whether the compose window is open and if open pop up a message box.
Thanks
Best regards
Zam Shabeer Thahir

TaoChitpol wrote:
By the way, the problem happens with the account of MS Exchange service, I don't know if this is the cause of the problem.
I think yes. The encoding is being corrupted somewhere, probably Exchange. There is nothing you can change on the iPad other than the OS languagel, to see if that helps. You can also tell Apple at
http://www.apple.com/feedback/

Any tips for removing duplicate photos?

I had over 12,000 photos, with batches of photos in directory structures similar to folders like:
/NewEvent
/NewEvent/OriginalJPGs
/NewEvent/ProcessedForWeb
/NewEvent/ProcessedForPrint
/NewEvent/ProcessedForScreenDisplay
Now, I have multiple photos ... is there any way to easily remove duplicate photos in iPhoto? Other Suggestions? I'm switching from PC to Mac, and have just gotten started with iPhoto.
Thanks,
Richard

Well, they are duplicates, as far as I'm concerned, they mess up my orgami slide show by seeing the same photo plastered to the screen three times ... and they were duplicates before I imported them into iPhoto ... but yes, I processed the Nikon NEF file to a proof first image, then to a print optimized second image, then to a web browser optimized third image ... etc ... all BEFORE I imported them into iPhoto ...
So, now, I was wondering if there was a way to delete duplicte (meaning photos that look alike) eventhough they are different sized ...
Are you saying that if my source folders had multiple copies of the exact same file, then iPhoto would have only imported one of these files, and skipped the second?
thanks for your help
Richard

Script for removing color definitions

I'm trying to create a script that will remove color definitions other than the default plus our corporate colors. I'm thinking some kind of "while" loop to go through the doc's color list, matching it to an array of named colors.
I see that there is a Color object and methods such as Delete(), but it's not coming together for me.
Any help would be appreciated.

Finally able to get back to this project.
Thanks for this snippet which does work great for removing a single "known" color. What I actually need to do is remove all colors except a specified list.
I've made an attempt (I'm sill relatively new to JS programming), but it only deletes the first instance instead of iterating through all the color definitions:
doc = app.ActiveDoc   //getting the active doc
col = doc.FirstColorInDoc   //getting the first color in doc
keepColors = new Array ("Black","White","Red","Green","Blue","Cyan","Magenta","Yellow","Dark Grey","Pale Green","Forest Green","Royal Blue","Olive","Salmon","Mauve","Light Salmon","Custom Color");
while (col.ObjectValid() ) //checking if the color object is valid
    if (col.Name in (keepColors)) //color exists
       break
else {
col.Delete() //delete color
col = col.NextColorInDoc

Executing a powershell script for checking duplicate users while creating a AD user throug ADUC console.

Hi,
I have a text file in which some SamAccountNames are present.I need to check the file while creating a new users through ADUC console.If a username that is going to create through ADUC console is present in the file, then it should prompt a message
that the user is already present in the text file.
Is there any possibility of contacting the powershell script from the ADUC console.If so, then while creating a new user through ADUC console, what is the proceedure for executing that powershell script.
please provide me the approriate solutions.
Thanks
Prasanthi k

Run the below Powershell Script for users are exist or not in AD. Later you can create the users.
#Find Users exist in AD or Not?
#Biswajit Biswas
$users = get-content c:\users.txt
foreach ($user in $users) {
$User = Get-ADUser -Filter {(samaccountname -eq $user)}
If ($user -eq $Null) {"User does not exist in AD ($user)" }
Else {"User found in AD ($user)"}
Active Directory Users attributes-Powershell
http://gallery.technet.microsoft.com/scriptcenter/Getting-Users-ALL-7417b71d
Regards~Biswajit
Disclaimer: This posting is provided & with no warranties or guarantees and confers no rights.
MCP 2003,MCSA 2003, MCSA:M 2003, CCNA, MCTS, Enterprise Admin
MY BLOG
Domain Controllers inventory-Quest Powershell
Generate Report for Bulk Servers-LastBootUpTime,SerialNumber,InstallDate
Generate a Report for installed Hotfix for Bulk Servers

Removing duplicate home mailing addresses?

after merging duplicate contacts, most contacts now have multiple (duplicate) mailing addresses. How can I remove/merge these duplicate addresses?
Thanks,
Mike

From what you're describing... It doesn't sound like your modem/router is in full bridge mode. Sounds like half bridge.
I guess you could disconnect the EA6500 from modem. Reset EA6500 to factory defaults. Change the local IP address of the EA6500 to 192.168.2.1. And then reconnect the EA6500 back to the modem.

Powershell script for removing some users from a particular Site Collection

Hi,
I am looking for a PowerShell script to delete a few users from a particular Site Collection. I am unable to delete them from/_catalogs/Users/simple.aspx page therefore need some other medium to
delete users from the site collection.
My ultimate aim is to have no user profile with "tp_deleted" field's value as 0 in the USERINFO table. Currently there are about 40 odd users with this field's value as 0 and this is affecting my crawling of this content database.

Thanks for the reply Alex & eHaze,
I have a content source of root site which crawls all the site collections under it. Out of the 9 site collections, only 8 are getting crawled and 1 doesn't get crawled at all. The error in the crawl logs is
The SharePoint item being crawled returned an error when requesting data from the web service. ( Error from SharePoint site: Value does not fall within the expected range. )
I tried a lot of things, searched over the net and finally found
this which helped me solve the same issue in my development environment. I deleted these users from userInfo table and ran a full crawl. And the issue was fixed.
Now since I cannot delete the users from userInfo table directly from PROD environment, I used .../_catalogs/Users/simple.aspx list
to delete users from this site collection. While some of the users I could delete, quite a few I could not. Clicking on the profile redirected me to the home page rather than the info page of the profile.
This
is why I have to delete these users from the site collection.
Alex - the link you shared, I guess it is for a web application level.
eHaze - the script you shared throws this error:
Get-SPSite : Cannot find an SPSite object with Id or Url: http://dev-apps/divisions/BT. At C:\PowerShell Scripts\DeleteUserFromSiteCollection1.ps1:4 char:19
+ $site = get-spsite <<<< $siteURL
+ CategoryInfo : InvalidData: (Microsoft.Share...SPCmdletGetSite:
SPCmdletGetSite) [Get-SPSite], SPCmdletPipeBindException
+ FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletGetSite
You cannot call a method on a null-valued expression.
At C:\PowerShell Scripts\DeleteUserFromSiteCollection1.ps1:9 char:27
+ $site.SiteUsers.Remove <<<< ($LoginName)
+ CategoryInfo : InvalidOperation: (Remove:String) [], RuntimeExc
eption
+ FullyQualifiedErrorId : InvokeMethodOnNull
hope this info helps.

Need a Script for Removing User Profiles in Windows 7

I am in need of a script that will allow me to remotely delete user profiles from multiple computers. I am an IT Admin over around 15-20 computers that are publicly used by many different people each day, and at least once a month we have to go into each
computer individually..my computer, properties, profiles and delete them one by one. This is growing very tedious due to manpower and the number of computers we have to do this on.
I have absolutely zero knowledge in the scripting department so could somebody point me in the right direction on this? I would like to have a script that i can just run once a month or so and say "delete all profiles from computers x y and z that are
over a few months old." Is this possible? Thanks in advance!

Try this link - http://superuser.com/questions/643417/delete-user-profile-from-command-line-windows-7
https://social.technet.microsoft.com/Forums/scriptcenter/en-US/7663ea51-9f4e-4fd7-9946-e16e65c3ed9d/remove-windows-7-user-profiles-via-powershell-script-initiated-on-logoff?forum=ITCG
Please 'propose as answer' if it helped you, also 'vote helpful' if you like this reply.

How to remove duplicate POP messages in Mail under Mavericks

Hello
I'm having some significant problems with Apple Mail, and despite having researched widely, I haven't been able to find a solution yet. Here is my tale of woe...
My set up is that I use Mail with 5 separate accounts, downloading all messages using POP. I keep ALL my mail (even the trash) so that I have a permanent record of all conversations, going back more than 10 years - this is about 250,000 emails in around 250 subfolders. I'm running the latest version of Mavericks on a 2010 MBP. I have no intention of moving to IMAP - I need to keep a local archive of all my mail.
A few months ago I was having problems with Mail being very slow to load and search emails. I decided to rebuild the mailboxes, but afterwards discovered that thousands of messages in the inbox of one of my accounts (and some of the sub-folders) had disappeared. Once I realised I tried to go back to a recent Time Machine back up, but, for whatever reason, that part of the back up hadn't worked properly and I was unable to recover the previous status. I left it while I decided what to do, and had time to deal with it...
The last 2 years worth of emails on that account were still archived on gmail, so last week I decided to try redownloading the whole lot (something like 72,000 messages), and then planned to use one of the scripts I was aware of to remove the duplicates to the trash, and then remove any duplicates in the trash forever. That ought to mean I at least had an archive of the last two years of missing messages, even if I had no record of which ones had been flagged or replied to, etc.
The downloading took about 24 hours to complete, and I now have almost 70,000 messages in the inbox of that account. I also rebuilt all the folders again, which doesn't seem to have resulted in any further loss of messages as far as I can tell. However, I hadn't figured on two things:
1. Mail 'hides' duplicate messages, meaning that all of the messages I've downloaded are showing as unread, and I can't differentiate the ones I already had in my Inbox, and the new downloads. When I click on those apparently unread messages I can see if they have been replied to or forwarded, etc, but there's no obvious way for me to remove the ones I have already dealt with to the trash, without going through them all manually, which is clearly impossible.
2. The scripts I'd found for removing duplicates don't work. Andreas Amann's Remove Duplicates script doesn't work under Mavericks, and he has abandoned the project. I've also tried the remove-duplicate-messages.scpt made by Jolly Roger (http://jollyroger.kicks-***.org/software/), and while it *sometimes* works on individual subfolders on my Mac (but as far as I can tell removes the duplicate in that folder, rather than the newly downloaded version), mostly it doesn't work at all - it creates a 'Remove Duplicate Messages' folder on my desktop, a log inside it and a folder for removed messages, but nothing appears in the duplicates folder.
So, I'm left in a position where I have 70,000 apparently unread messages in my Inbox, a massively bloated Mail library (which has pretty much doubled in size, because of the 'hidden' messages), a slow and unresponsive Mail program. I've come to the conclusion that there must be some corrupted email somewhere, which probably caused the original email haemorrhage, and may still be causing the inability to remove duplicates. Mail is so slow as to be almost unuseable.
I figure I have a number of options:
1. I could live with the situation and just archive most of the Mail in my inbox, with the side effect that there will be a bunch of messages I have never replied to that are missed.
2. I could abandon the last week's efforts and revert to the version of Mail I was using a week ago, and then redownload the recent emails from my various accounts. That would still leave me without those thousands of emails I lost on my local machine.
3. I could find another way to deal with this. Can I get the remove duplicates script working? Should I revert to the version of Mail from a week ago, download ALL the messages again, but do it in a way that allows me to find the duplicates and remove them? Should I move to another email program altogether (which would presumably be massively disruptive to my work!)
Anyway, apologies for the essay length of this request, and thanks in advance for any help you may be able to offer!

hi Eric
thanks for the tip, but I don't think that solves my problem, which is that Mail apparently hides the duplicates, so they're still there, you just can't differentiate or remove them. I actually *need* the duplicates there so I know I've got everything, but then I need to be able to move the ones I know I've dealt with from my inbox to an archive/trash, and/or remove them completely so Mail isn't totally bloated

How to remove duplicate messages from Mail in Mavericks (POP)

Hello
I'm having some significant problems with Apple Mail, and despite having researched widely, I haven't been able to find a solution yet. Here is my tale of woe...
My set up is that I use Mail with 5 separate accounts, downloading all messages using POP. I keep all my mail so that I have a permanent record of all conversations, going back more than 10 years - this is about 250,000 emails. I'm running the latest version of Mavericks on a 2010 MBP. I have no intention of moving to IMAP - I need to keep a local archive of all my mail.
A few months ago I was having problems with Mail being very slow to load and search emails. I decided to rebuild the mailboxes, but afterwards discovered that thousands of messages in the inbox of one of my accounts (and some of the sub-folders) had disappeared. Once I realised I tried to go back to a recent Time Machine back up, but, for whatever reason, that part of the back up hadn't worked properly and I was unable to recover the previous status. Ileft it while I decided what to do, and had time to deal with it.
The last 2 years worth of emails on that account were still archived on gmail, so last week I decided to try redownloading the whole lot (something like 72,000 messages), and then uplanned to use one of the scripts I was aware of to remove the duplicates to the trash, and then remove any duplicates in the trash forever. That ought to mean I at least had an archive of the last two years of missing messages, even if I had no record of which ones had been flagged or replied to, etc.
The downloading took about 24 hours to complete, and I now have almost 70,000 messages in the inbox of that account. However, I hadn't figured on two things:
1. Mail 'hides' duplicate messages, meaning that all of the messages I've downloaded are showing as unread, and I can't differentiate the ones I already had in my Inbox, and the new downloads. When I click on those apparently unread messages I can see if they have been replied to or forwarded, etc, but there's no obvious way for me to remove the ones I have already dealt with to the trash, without going through them all manually, which is clearly impossible.
2. The scripts I'd found for removing duplicates don't work. Andreas Amann's Remove Duplicates script doesn't work under Mavericks, and he has abandoned the project. I've also tried the remove-duplicate-messages.scpt made by Jolly Roger (http://jollyroger.kicks-***.org/software/), and while it *sometimes* works on individual subfolders on my Mac (but as far as I can tell removes the duplicate in that folder, rather than the newly downloaded version), mostly it doesn't work at all - it creates a 'Remove Duplicate Messages' folder on my desktop, a log inside it and a folder for removed messages, but nothing appears in the duplicates folder.
So, I'm left in a position where I have 70,000 apparently unread messages in my Inbox, a massively bloated Mail library (which has pretty much doubled in size), a slow and unresponsive Mail program. I figure I have a number of options. Either I could abandon the last week's efforts and revert to the version of

Apologies, this was posted here in error. Please find updated post here: https://discussions.apple.com/message/27261965#27261965

Any apps for deleting duplicate mail?

I was thinking this was built in to Mail, but looking for a script or app that will help me find and eliminate duplicate emails in my mailboxes.

Here is a script to remove duplicate messages from Mail.app

Best tool for removal of duplicate photos

What is the best tool for removing duplicates from iPhoto?

For dealing with duplicates in iPhoto check out iPhoto Library Manager Duplicate Annihilator Decloner or iPhoto Duplicate Cleaner

Remove Duplicate record

Dear All,
I have oracle 10g R2 On windows.
I have table structure like below...
ASSIGNED_TO
USER_ZONE
CREATED
MASTER_FOLIO_NUMBER
NAME
A_B_BROKER_CODE
INTERACTION_ID
INTERACTION_CREATED
INTERACTION_STATE
USER_TEAM_BRANCH
A4_IN_CALL_TYPE
A5_IN_CALL_SUBTYPE
DNT_AGING_IN_DAYS
DNT_PENDING_WITH
DNT_ESCALATION_STAGE_2
DT_UPDATEI use sql loader to load the data from .csv file to oracle table and have assign the value to dt_update sysdate. Everytime i execute the sql loader control file dt_update set as sysdate.
Sometimes problem occures while inserting data through sql loader and half row get insert. after solving the problem again i execute sql loader and hence these duplicate records get inserted.
Now I want to remove all the duplicate records for those dt_update is same.
Please help me to solve the problem
Regards,
Chanchal Wankhade.

Galbarad wrote:
Hi
I think you have two ways
first - if it is first import in your table - you can delete all record from table and run import yet one time
second - you can delete all duplicate records and not running import
try this script
<pre>
delete from YOUR_TABLE
where rowid in (select min(rowid)
from YOUR_TABLE
group by ASSIGNED_TO,
USER_ZONE,
CREATED,
MASTER_FOLIO_NUMBER,
NAME,
A_B_BROKER_CODE,
INTERACTION_ID,
INTERACTION_CREATED,
INTERACTION_STATE,
USER_TEAM_BRANCH,
A4_IN_CALL_TYPE,
A5_IN_CALL_SUBTYPE,
DNT_AGING_IN_DAYS,
DNT_PENDING_WITH,
DNT_ESCALATION_STAGE_2,
DT_UPDATE)
</pre>Have you ever tried that script for deleting duplicates? I think not. If you did you'd find it deleted non-duplicates too. You'd also find that it only deletes the first duplicate where there are duplicates.
XXXX> CREATE TABLE dt_test_dup
2 AS
3 SELECT
4      mod(rownum,3) id
5 FROM
6      dual
7 CONNECT BY
8      level <= 9
9 UNION ALL
10 SELECT
11      rownum + 3 id
12 FROM
13      dual
14 CONNECT BY
15      level <= 3
16 /
Table created.
Elapsed: 00:00:00.10
XXXX> select * from dt_test_dup;
        ID
         1
         2
         0
         1
         2
         0
         1
         2
         0
         4
         5
         6
12 rows selected.
Elapsed: 00:00:00.18
XXXX> delete
2 from
3      dt_test_dup
4 where
5      rowid IN ( SELECT
6                    MIN(rowid)
7                 FROM
8                     dt_test_dup
9                 GROUP BY
10                     id
11                )
12 /
6 rows deleted.
Elapsed: 00:00:00.51
XXXX> select * from dt_test_dup;
        ID
         1
         2
         0
         1
         2
         0
6 rows selected.
Elapsed: 00:00:00.00

Remove duplicate emails and mailboxes

I am moving emails from an old Mac to our new MacBook Pro. My wife has a LOT of mailboxes. It seems to me that when she imported her emails onto our old Mac, from an even older Mac, that mailboxes got duplicated. I think there are mailboxes and folders (with different names) that actually contain the same emails (or mostly the same) as other mailboxes and folders.
I've read some of the threads on the Apple Support Communities. I understand there is a script for finding duplicate emails. But will this only find duplicates within the same mailbox? I don't have this problem. I need to find duplicate mailboxes and duplicate emails within different mailboxes.
I am using Lion and Mail.
Thanks for any help
Mark

Same problem, only one of four accounts (POP) acts this way, and is reloading, and re-reloading, and re-re-re (you get the picture) mail from this account going back to June 07. All marked unread,up to almost 2k of these so far. Running OSX.4.10, dual 2ghz Powermac, 4gb ram, no recently added software.
Thanks!

How do I remove duplicates and triplicates?

Hello.
I've just finished installing iphoto '08 and found that I have 2 to 3 duplicates of each picture.
Is there a setting for removing duplicates or do I have to do them by hand?
Many thanks in advance!

Hi LN,
Thanks again for the info. I did download the Duplicate Annihilator and it works great. Problem is I've discovered 3, 4, 5, and in some cases 6 copies of photos. DA took out some at a time ( tried it three times ) and it seems it can only go so far "downwards" and it's for example, I'm guessing, taking out 1 of the sometimes 6 copies each time. Also left with little grey dotted squares where the copies used to be and they are all mostly out of order date wise.
What do you think of this? I have a fresh copy of my latest working iphoto '04 library on an external hard drive. Should I rebuild iphoto '08 and drag the photos from my '04 into the new iphoto? Maybe uninstall iphoto 08' and install again and drag the '04 library into the new iphoto?
My only concern is that the same thing would happen again.
I have a "iphoto cache" plus other iphoto folders on my back up 04'. This I can't find anywhere in my new iphoto '08. Whenever I open the "iphoto '08 library" ( in applications folder ) on my computer iphoto just opens up.
Very glad I have a recent, duplicate free backup on the EX HD. It would be nice to start from scratch somehow. Maybe I have an unnamed iphoto library floating around somewhere?
Maybe just take your first suggestion and drag my pictures from the back-up.
Many thanks for your help!

Script for removing duplicates in Mail

Similar Messages

Maybe you are looking for