My other “fav-o-rite” mail user agent (MUA) is Mutt. Seems that Mutt has a nice duplicate mail detection scheme already built in. You can see details of it at:
One annoying thing about Imap clients like Thunderbird is you can end up getting a bunch of duplicate messages in various folders from copying messages from your Inbox. For instance, I have one folder with about 17,000 messages in it and a bunch of duplicates that are just taking up space, bandwidth and CPU cycles when I open that folder.
This is where a really nifty Thunderbird plugin comes in handy with the amazing name of “Remove Duplicate Messages“. With a simple download, install and a restart of Thunderbird and I found I had about 1,000 messages that were duplicates in that folder, or almost 6% of the folder. In another folder that has my NANOG mailing list, I found about 10% of the messages were duplicates.
The defaults are pretty sane, where it will keep the first message it finds and delete all duplicates after that message. Just right click on the folder (control-click for Mac OSX) and you will see in the menu “Remove Duplicate Messages…”. It will start to scan your folder and then give you a list of duplicate mail it found. You can scan through what it though where dups to verify and then click “Delete Selected” to have it move to the trash the duplicate messages. Quick and easy.