Remove Duplicate Items

Hi,
I have a input structure like below; I need to produce output structure based on Student_no...
scenario 1: If 5 different student number records there...Need to create 5 records in output structure. If the same student info comes more than one time- create only one record to corresponding student and count the subjects.
scenario 2: sometimes input structure may miss the student no...then I need to create blank record or segment only with the marks info.
Input structure:
DT_Student_Input
      Student_Details --- 1 - Unbounded
           Student_No-- 0 -1
           Section_Name -- 0-1
                 Marks_Info -
1--- Unbounded
                        no.of marks 0-1
Note: Marks_info is sub element of Student_Details
help me out!!!
-S
Edited by: stallin xavier on Jan 16, 2009 7:08 AM
Edited by: stallin xavier on Jan 16, 2009 7:09 AM
Edited by: stallin xavier on Jan 16, 2009 7:20 AM

Hi,
1. Check if the Student_No is blank then if YES then fill it with a Node Index (or some Unique value) and then concatenate with some common string (eg: ###).
Eg :
Rec1 : StudNo = 10001
Rec2 : StudNo =
Rec3 : StudNo = 10002
Rec4 : StudNo =
Rec5 : StudNo = 10001
Output should be
Rec1 : StudNo = 10001
Rec2 : StudNo = 2###
Rec3 : StudNo = 10002
Rec4 : StudNo = 4###
Rec5 : StudNo = 10001
2. Then use removeContext to remove all the context and the SORT
Eg :
<ContextChange>
Rec1 : StudNo = 10001
<ContextChange>
Rec2 : StudNo = 2###
<ContextChange>
Rec3 : StudNo = 10002
<ContextChange>
Rec4 : StudNo = 4###
<ContextChange>
Rec5 : StudNo = 10001
<ContextChange>
Output
<ContextChange>
Rec1 : StudNo = 10001
Rec2 : StudNo = 10001
Rec3 : StudNo = 10002
Rec2 : StudNo = 2###
Rec4 : StudNo = 4###
<ContextChange>
3. Then CollapseContext and then SplitByValue (EachValue)
Output
<ContextChange>
Rec1 : StudNo = 10001
<ContextChange>
Rec3 : StudNo = 10002
<ContextChange>
Rec2 : StudNo = 2###
<ContextChange>
Rec4 : StudNo = 4###
<ContextChange>
4. Then check if a Student_No has ### then blank it out before assigning to the target.
This way you can eliminate duplicates and also take care of the BLANK. But the records will get sorted hence while assigning the other fields to the target you need to take care of this.. either by SORT or any method you prefer.
Thanks,
Prakash

Similar Messages

Help needed in removing duplicate items of list box in java

How to remove duplicate items of list box while dynamically inserting (on-click event)
It is not identifying duplicate data
Variable name is HP_G1
HP_dmg1 = (DefaultListModel) HP_G1.getModel();
int a = HP_G1.getModel().getSize();
System.out.println("HP list no--------> "+a);
if(a!=0)
for (int j=0; j<a; j++)
String item1 = String.valueOf(HP_List.getModel().getElementAt(j));
System.out.println("HP list added--------> "+item1);
if(HP_dmg1.equals(item1)){
HP_dmg1.remove(j);
else
HP_dmg1.addElement(GPL);
}

Your code is unreadable, so I'll ignore it. In the future please press the message editor's CODE button to format code.
As to your problem, to the point you normally use a Set instead of List when you don't want duplicates in a collection.

HT2905 No Display Duplicates under File. How to find and remove duplicate items in your iTunes library

I now have iTunes ver 11.0.4.4 under Windows 7. I lost all iTunes stuff when updating to Windows 7. I have loaded thousands amd thousands of music files from backup disks, but there are many duplicates. I am attemping to re-establish my old library. I used to be able to remove duplicates quickly with the old iTunes. The new iTunes doesn't seem to offer the same service. Is there any way to remove duplicates quickly, or must I do it one by one?

When deduping use Shift > View > Show Exact Duplicate Items as this is normally a more useful selection. You need to manually select all but one of each group to remove. Sorting the list by Date Added may make it easier to select the appropriate tracks. If you have multiple entries in iTunes connected to the same file on the hard drive then don't send to the recycle bin. Use my DeDuper script if you're not sure, don't want to do it by hand, or want to preserve ratings, play counts and playlist membership. See this thread for background and please take note of the warning to backup your library before deduping.
(If you don't see the menu bar press ALT to show it temporarily or CTRL+B to keep it displayed)
See also HT2905: How to find and remove duplicate items in your iTunes library
tt2

HT2905 How to find and remove duplicate items in your iTunes library

I need an easy and quick way to remove duplicate items from Itunes library since it duplicates to the IPOD 4.

I've written a script called DeDuper which can help remove unwanted duplicates.
See this thread for background.
tt2

HT2905 Does anyone know how to find and remove duplicate items in the new itunes 11?

I am having a hard time trying to figure out how to find and remove duplicate items under the New iTunes 11? Its not as user friendly as in previous versions. Does anyone know how to figure this out?

As Jim said View > Show Duplicates
or
Option + View > Show Exact Duplicates

PowerShell script to find and remove duplicate items from document library

Hi Friends,
Please check this below script, It is finding the duplicate items and removing it. This script is not working to find the duplicate items from document
library and remove. Can you please let me know where it is wrong.
Add-PSSnapin microsoft.sharepoint.powershell
$web = Get-SPWeb -Identity "http://zapltvsspdev02:4333/bu/EXCO"
$list = $web.Lists["AECI Documents"]
$AllDuplicates = $list.Items.GetDataTable() | Group-Object INumber | where {$_.count -gt 1}
$count = 1
$max = $AllDuplicates.Count
foreach($duplicate in $AllDuplicates)
$duplicate.group | Select-Object -Skip 1 | % {$list.GetItemById($_.ID).Delete()}
Write-Progress -PercentComplete ($count / $max * 100) -Activity "$count duplicates removed" -Status "In Progress"
$count++
Remove-PsSnapin Microsoft.SharePoint.PowerShell
Note:
In the above code, duplicates are found using the "INumber" column.
valmiki

check this
http://stackoverflow.com/questions/21337158/delete-duplicate-items-in-sharepoint-list

DataProvider.merge() method seems to not remove duplicate items

Using the DataProvider class to instantiate UI Components
data, I need to merge dataProvider data and remove any duplicate
items that may result from the merging process. It is supposed
(according to the manual) that the merge method behaves just as i
desire, but actually the data is being merged with duplicates. Any
hints?

The docs are a little ambiguous. There is no actual "removal"
of items. The
method iterates the new data items and appends those that are
not already in
the provider. Since each item is an Object instance, they
must be the same
object to be equal.
In this example, the commonItem is "removed" during the
merge, but there are
still two instances in the original list
var commonItem:Object = {label:"itemX"};
var data1:Array = [
{label:"item1"},
commonItem,
commonItem,
{label:"item2"}
var data2:Array = [
{label:"item1"},
commonItem , // duplicated item
{label:"item3"}
var dp:DataProvider = new DataProvider(data1);
dp.merge(data2);
var myList:List = new List();
myList.dataProvider = dp;

How to remove duplicate items ?

ok so ive moved my iTunes library from the NAS drive I bought (after finding out that wouldnt work) onto my new laCie external drive, but ive found some of my albums have triple copies?
i know how to show duplicates but im not sure how to safely remove duplicates without deleting all copies?

Oh boy.
I'm sure there are better ways to do it than this and it will take time, but to avoid all possible loss of data, what I would do is first consolidate all of your libraries.
• Open the iTunes Library you think is most correct by holding down the OPTION key when you open the iTunes application, which should bring up a dialogue like this: http://cl.ly/image/2i2Q3o0Z0Y3C
• Then go to preferences and make sure it looks like this: http://cl.ly/image/2C2Z0u0C3T3c
I would recommend keeping your main iTunes library on your main hard drive (for me that's my internal), unless you definitely can't fit it.
• Now go to the File menu >Library > Import Playlist
• Navigate to another of the libraries, and click on the iTunes Library.xml file and import it. Do this for each of your libraries, except the one you are currently using.
• Now that you've got everything imported into the one library, the fun part starts.
Do what i said in my previous post and remove all the duplicates.
• Once that's done, check all the other libraries to make sure you haven't missed anything, and send them on their way to the Trash, and empty it to reclaim all that space.
I really hope you get this sorted, I went through an ordeal like this myself recently, so it's going to take time, but it feels good when it's all cleaned up and finished!
xeni
PS. After writing all this I thought, hmm, why didn't I just Google it instead of figuring it out myself? ;P
I found this, and it might help if my instructions weren't clear enough. https://bitly.com/LpqFPq
Also, if you don't already, I urge you to use Time Machine backup. Read more here: http://www.apple.com/osx/apps/#timemachine and http://pondini.org/TM/FAQ.html

Remove duplicates items in List

I created a WebAPI and its calling an SQL Server to get my data. I created a List<> to populate and get those items back to UI. However, in my output there are duplicates due to the data. Is there a way to remove the duplicate strings before passing
it back to my UI?
I'm reading through the data coming back from SQL here
while (reader.Read())
Jobs jobs= new Jobs();
jobs.FunctionDesc = reader.GetString(0);
jobs.Function = reader.GetString(1);
jobss.Add(jobs);
}return jobs
is there a way I can remove the duplicate FunctionDesc prior to returning it back to the UI?

You could keep populating the List<Jobs> exactly like you are doing now, create a class that implements the IEqualityComparer<Jobs> interface and simply pass an instance of this one to the Distinct method that you call on the list once you have
populated it:
public class JobsComparer : IEqualityComparer<Jobs>
public bool Equals(Jobs x, Jobs y)
return x.FunctionDesc.Equals(y.FunctionDesc);
public int GetHashCode(Jobs obj)
return obj.FunctionDesc.GetHashCode();
while (reader.Read())
Jobs jobs= new Jobs();
jobs.FunctionDesc = reader.GetString(0);
jobs.Function = reader.GetString(1);
jobss.Add(jobs);
jobss = jobss.Distinct(new JobsComparer()).ToList();
return jobss;
This will effectively remove all duplicates with minimal changes to your current code.
Hope that helps.
Please remember to close your threads by marking helpful posts as answer and then start a new thread if you have a new question.

HT2905 How to find and remove duplicate items in your iTunes library for iTunes 11

I found instructions for iTunes 10, not the same as 11. New to iTunes - have LOTS of duplicates. There must be a way to delete them besides one at a time! Please help...

If you don't see a menu bar press Ctrl+B to reveal it or Alt to show it temporarily. To check your library for duplicates use Shift > View > Show Exact Duplicates as this is normally a more useful selection. Keep holding down shift until you have clicked on the text Show Exact Duplicates or it may still use the loser definition.
If you find that you have true duplicates you need to manually select all but one of each group to remove, or use my DeDuper script if you don't want to do it by hand. The script attempts to take account of different types of duplicates which need to be handled differently, merges playcounts and preserves playlist membership. Please take note of the warning to backup your library before deduping. See this thread for background.
tt2

How do i remove duplicate items from my library

After interrupting a download of a season of TV shows from the iStore, I restarted my computer and the download process started again from the beginning, creating duplictes of episodes that had previously been downloaded. Result appears to be duplicate files in my library. I've attempted to delete the "duplicates" from my library with no success. Are these really duplicated or is it just a display issue. If they are duplcautes, is there a simple way to provent this from hapening again? Seems odd that the download process wouldn't recognize that the files were already there.

Do some of the episodes show up with a cloud symbol? If so sign out of the iTunes store and then sign in again.
Highlight one duplicated episode, right-click and click Show in Windows Explorer. Are there multiple copies of the same episode listed? If so you have true duplicates and can delete the redunant copies from iTunes, sending the files to the recycle bin when prompted. If there is only one copy of each episode then delete the spare entries from iTunes but don't recycle the files.
tt2

How do I remove duplicate items from my ipod?

I manually managed my music files to add to my new ipod. Now some songs are on there twice. That is not how it shows up in the iTunes library.

- Try deleting them individually.
- Unsync/delete all music and resync
To delete all music go to Settings>General>Usage>Storage>Music>Tap edit in upper right and then tap the minus sign by All Music
- Restore from backup. See:
iOS: Back up and restore your iOS device with iCloud or iTunes
- Restore to factory settings/new iOS device.

HT2905 how do you remove duplicate libraries from itunes

I recently purchased a new window9.1 computer. I tried to move the library to the new pc by using an external haddrive. The songs and other items all transfered, but the mudic remsined on the externalmedia informaiton files, I had to connect the drive to listen to music.. I worked with tech support and copied themusioc files to ITunes. When I reopened the media files and copied music were on the system. ITunes recognzed there was missing information on the media data file and corrected it, so oe I have 20,000 sonfs in a 10,000 song library.How do you delete massive duplicates without individually deleting10,000 songs?

Apple's official advice is here... HT2905 - How to find and remove duplicate items in your iTunes library. It is a manual process and the article fails to explain some of the potential pitfalls.
Use Shift > View > Show Exact Duplicate Items to display duplicates as this is normally a more useful selection. You need to manually select all but one of each group to remove. Sorting the list by Date Added may make it easier to select the appropriate tracks, however this works best when performed immediately after the dupes have been created. If you have multiple entries in iTunes connected to the same file on the hard drive then don't send to the recycle bin.
Use my DeDuper script if you're not sure, don't want to do it by hand, or want to preserve ratings, play counts and playlist membership. See this thread for background and please take note of the warning to backup your library before deduping.
(If you don't see the menu bar press ALT to show it temporarily or CTRL+B to keep it displayed)
tt2

How to remove duplicate songs from Itunes

I've consolidates my music in Itunes 11.01. Playlists are just getting crazy with duplicates.
I want to eliminate duplicate music files, as well as eliminate the references in the interface.
Then sync my new IPad.
When I manually erase a duplicate in Windows Explorer, I'm left with a "!" indication in Itunes that the file cannot be found... duh.
I want to find either an easier way to eliminate duplicates without Windows Explorer, or
a way to select all the "!" references in Itunes interface and delete.
Help please from a completely organized person who hates Apples "intelligence" at thinking it knows what I want, and not providing
adequate help under "remove duplicates" in its help index.
Thanks.

How to find and remove duplicate items in your iTunes library - http://support.apple.com/kb/HT2905
Posts by turingtest2 about different types of duplicates and techniques - https://discussions.apple.com/thread/3555601 and https://discussions.apple.com/message/16042406 (Note: The DeDuper script is for Windows).
May 2014 post on iCloud duplicates - https://discussions.apple.com/message/25867873
Show exact duplicates (Mac and Windows) - https://discussions.apple.com/message/16951281
http://dougscripts.com/itunes/itinfo/dupin.php (commercial)
iTunes Duplicate Song Manager - http://sourceforge.net/projects/itunesdsm/
http://www.hardcoded.net/dupeguru_me/
http://www.wideanglesoftware.com/tunesweeper/index.php
http://www.araxis.com/find-duplicate-files (commercial, free trial) - finds duplicate files on computer, not specifically iTunes
I would do it manually. As you can see from turingtest2's post, some duplicates are not really duplicates.
I would also review how you add media to iTunes. It should not be producing all that many duplicates.

How do i remove duplicate songs on my ipod and in my library

how do i remove duplicate songs on my ipod and in my library

Apple's official advice on duplicates is here: Find and remove duplicate items in your iTunes library. It is a manual process and the article fails to explain some of the potential pitfalls such as lost ratings and playlist membership, or that sometimes the same file can be represented by multiple entries in the library and that deleting one and recycling the file will break the other.
Use Shift > View > Show Exact Duplicate Items to display duplicates as this is normally a more useful selection. You need to manually select all but one of each group to remove. Sorting the list by Date Added may make it easier to select the appropriate tracks, however this works best when performed immediately after the dupes have been created. If you have multiple entries in iTunes connected to the same file on the hard drive then don't send to the recycle bin.
Use my DeDuper script if you're not sure, don't want to do it by hand, or want to preserve ratings, play counts and playlist membership. See this thread for background, this post for detailed instructions, and please take note of the warning to backup your library before deduping.
(If you don't see the menu bar press ALT to show it temporarily or CTRL+B to keep it displayed.)
The most recent version of the script can tidy dead links as long as there is at least one live duplicate to merge stats and playlist membership to and should cope sensibly when the same file has been added via multiple paths.
Fix the library, then syncing should fix the device.
tt2

Remove Duplicate Items

Similar Messages

Maybe you are looking for