Delete dupliate entries in log or Result table

Hi,
I generated a log window with several entries, some of which are repeated. Is there any way to delete entries which are repeated so I get no repetitions?

An example of the log window:

MC0619_T1_200_1
MC0619_T1_200_1
MC0619_T1_315_1
MC0619_T1_315_1
MC0619_T1_5000_1
MC0619_T1_5000_2
MC0619_T2O_200_1

What I want:

MC0619_T1_200_1
MC0619_T1_315_1
MC0619_T1_5000_1
MC0619_T1_5000_2
MC0619_T2O_200_1

Thanks.

Li.

Hi @insertsk8,

If it’s just a one-off occasion, I’d just paste the result in excel and run “remove duplicates”.

On the other hand, if you want to code a solution, there are a couple of alternatives (more and less efficient):

  1. If the ordering doesn’t matter, get the list from the log, make it into an array, sort the array, loop through it and save the first appeareance of each item (i.e. each item that is different from the last one)
  2. If you want to keep the order:
    2a. The lazy and innefficient way: loop through the array from the bottom and compare each item to all the others above it. Save the items that do not match with any above. Reverse that list once you finish.
    2b. A smarter way: obtain a second array with the sorting order of the first one. Repeat 1) but using this second array as an index for the first one, making a list of the indexes you keep. Sort that list, retrieve the items using that sorted list.

Cheers,
Nico

Here’s 2b):

list = newArray(
"MC0619_T1_315_1",
"MC0619_T1_200_1",
"MC0619_T1_200_1",
"MC0619_T1_315_1",
"MC0619_T1_5000_1",
"MC0619_T1_5000_2",
"MC0619_T2O_200_1"	);

//list = split(getInfo("log"), "\n"); // to get the list from the log window

n = list.length;
order = Array.rankPositions(list);
keep = Array.fill(newArray(n), n);

keep[0] = order[0];
k = 1;
for (i = 1; i < n; i++) {
	if(list[order[i]] != list[order[i-1]]) {
		keep[k] = order[i];
		k++;
		}
	}

keep = Array.sort(keep);

for (i = 0; i < n; i++) {
	if(keep[i]<n) keep[i] = list[keep[i]];
	else break;	
	}
	
keep = Array.slice(keep,0,i);

Array.show(keep); // show the final list

Nico

1 Like

Oh, wow! That’s just it! Do you have any suggestion for completely erasing duplicates (i.e. if two entries match > erase them both)? Thanks again!

Figured it out…
Thanks!

ConcatArray = Array.concat(Log, Log1);
SortedArray = Array.sort(ConcatArray);
Array.show(SortedArray);

if(SortedArray[0] != SortedArray[1]){
		print(SortedArray[0]);
}


for (i = 1; i<SortedArray.length-1; i++) {
	if((SortedArray[i] != SortedArray[i+1]) && (SortedArray[i] != SortedArray[i-1])){
		print(SortedArray[i]);
	} 
}

for (i = i; i <SortedArray.length; i++) {
	if(SortedArray[i] != SortedArray[i-1]){
		print(SortedArray[i]);
	}
}
close("SortedArray");
1 Like

Yes, in that case you’d have to check two things for each entry: that the items before AND after are different from itself (and also catch the cases where the entries are at the begining or the end of the list), like this:

list = newArray(
"MC0619_T1_315_1",
"MC0619_T1_200_1",
"MC0619_T1_200_1",
"MC0619_T1_315_1",
"MC0619_T1_5000_1",
"MC0619_T1_5000_2",
"MC0619_T2O_200_1"	);

//list = split(getInfo("log"), "\n"); // to get the list from the log window

n = list.length;
order = Array.rankPositions(list);
keep = Array.fill(newArray(n), n);

k = 0;
for (i = 0; i < n; i++) {
	candidate=true;
	if(i>0){
		if(list[order[i]] == list[order[i-1]]) candidate = false; // same as previous, discard
		}
	if(i < n - 1){
		if(list[order[i]] == list[order[i+1]]) candidate = false; // same as following, discard
		}
	if(candidate){ // still holds
		keep[k] = order[i];
		k++;
		}
	}

keep = Array.sort(keep);

for (i = 0; i < n; i++) {
	if(keep[i]<n) keep[i] = list[keep[i]];
	else break;	
	}
	
keep = Array.slice(keep,0,i);

Array.show(keep); // show the final list

I didn’t notice you were also typing! That’s exactly it!