Remove Duplicate Values From a PowerShell Array

1. Use Select-Object to Remove Duplicate Values From a PowerShell Array
2. Use Sort-Object to Remove Duplicate Values From a PowerShell Array
3. Use Get-Unique to Remove Duplicates From PowerShell Array

An array is a data structure used to store a collection of multiple items, and the items can be the same or different types. Sometimes when working with arrays in PowerShell, you might need to remove duplicate items from an array.

This tutorial will introduce different methods to remove duplicates from a PowerShell array.

Use Select-Object to Remove Duplicate Values From a PowerShell Array

The Select-Object cmdlet is used to select specified properties of an object or set of objects. You can also select unique objects in a specified position in an array.

The -Unique parameter specifies that only a single member will be selected if identical properties and values are in objects. It is case-sensitive, so strings that only differ in character casing are also considered unique values.

We have created an array $data having multiple same items with different cases.

$data = @("powershell", "arrays", "powershell", "Arrays", "PowerShell")

The following example selects unique values from the array and saves them to the $data variable. As a result, the duplicate elements will be removed from an array.

Command:

$data = $data| Select-Object -Unique
$data

Output:

powershell
arrays
Arrays
PowerShell

It removes the duplicate string "powershell" but does not remove "PowerShell" because there is a capitalized letter in the string.

Use Sort-Object to Remove Duplicate Values From a PowerShell Array

The Sort-Object cmdlet sorts objects in ascending or descending order based on their property values. With Sort-Object also, you can use the -Unique parameter to remove the duplicates from the objects.

But it is case-insensitive, so strings that only differ in character casing are also considered the same. The following example removes the duplicate items from an array and saves them to the variable $a.

Command:

$a = @("powershell", "arrays", "powershell", "Arrays", "PowerShell")
$a = $a | Sort-Object -Unique
$a

Output:

Arrays
PowerShell

It removes "Arrays" and "PowerShell" although they only differ by character cases. If you want to make the Sort-Object case-sensitive, you can use the -CaseSensitive parameter.

Command:

$b = @("powershell", "arrays", "powershell", "Arrays", "PowerShell")
$b = $b| Sort-Object -Unique -CaseSensitive
$b

Output:

arrays
Arrays
powershell
PowerShell

Use Get-Unique to Remove Duplicates From PowerShell Array

The Get-Unique cmdlet gets unique items from a sorted list. It compares each item in a sorted list to the next item, removes duplicates, and returns only one member of each item.

This cmdlet only works if the list is sorted. You can use the Sort-Object cmdlet to sort objects by property values.

The following example uses the Get-Unique cmdlet to remove duplicate items from the array $sort after sorting the array elements with Sort-Object.

Command:

$c = @("powershell", "arrays", "powershell", "Arrays", "PowerShell")
$c = $c | Sort-Object | Get-Unique
$c

Output:

Arrays
arrays
PowerShell
powershell

Get-Unique is case-sensitive, so strings that differ only in character casing are also considered unique values. In this way, you can easily remove duplicate elements from an array in PowerShell.