How to Distinct by Property in Java 8 Stream

To deduplicate Java stream objects by a selected property, decide which object should survive for each key and whether the result needs encounter order. Stream.distinct() compares whole objects through equals() and hashCode(); collectors and key-based filters let you choose a property without changing the object’s equality contract.

How Stream.distinct() Defines Equality

Stream.distinct() retains one element from each group of objects that are equal according to equals(). Hash-based implementations also rely on a matching hashCode(). A Person class that inherits identity equality will treat two different instances as distinct even when both have the same name. If the class defines both methods using name, distinct() can deduplicate by name, but that changes equality everywhere the class is used.

For a local choice of key, extract the property or wrap the object with that key instead. The examples below use people with IDs and names; Alice with ID 1 precedes Alice with ID 3, and Bob with ID 2 precedes Bob with ID 5. The selection policy is explicit in each strategy.

Choose a Strategy for Distinct Values by Property

Use Collectors.toMap() when you need one object per key and can state which duplicate wins. Use Collectors.groupingBy() when you need all objects for each key before selecting. A key wrapper lets distinct() do the comparison while preserving the original class’s equality. A distinctByKey predicate or a Set filter is concise for a sequential stream, but it carries mutable state and is not a general parallel-stream solution.

The following complete Java 8 program applies these choices to the same ordered input. It retains the first object in the map, grouping, wrapper, and sequential-filter examples. This packet contains no observed runtime output, so the example is provided without an output transcript.

Run this Java 8 example:

import java.util.Arrays;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

public class DistinctByPropertyExample {
  static class Person {
    final int id;
    final String name;

    Person(int id, String name) {
      this.id = id;
      this.name = name;
    }

    String getName() { return name; }
    public String toString() { return name + "#" + id; }
  }

  static class Keyed<K, V> {
    final K key;
    final V value;

    Keyed(K key, V value) {
      this.key = key;
      this.value = value;
    }

    V value() { return value; }
    public boolean equals(Object other) {
      if (this == other) return true;
      if (!(other instanceof Keyed)) return false;
      Keyed<?, ?> that = (Keyed<?, ?>) other;
      return Objects.equals(key, that.key);
    }
    public int hashCode() { return Objects.hashCode(key); }
  }

  static <T, K> Predicate<T> distinctByKey(Function<? super T, ? extends K> keyOf) {
    Set<K> seen = new HashSet<K>();
    return item -> seen.add(keyOf.apply(item));
  }

  public static void main(String[] args) {
    List<Person> people = Arrays.asList(
        new Person(1, "Alice"), new Person(2, "Bob"),
        new Person(3, "Alice"), new Person(4, "Charlie"),
        new Person(5, "Bob"));

    Map<String, Person> byName = people.stream().collect(Collectors.toMap(
        Person::getName, Function.identity(), (first, later) -> first,
        LinkedHashMap::new));
    System.out.println("toMap: " + byName.values());

    Map<String, List<Person>> groups = people.stream()
        .collect(Collectors.groupingBy(Person::getName));
    List<Person> firstFromGroups = groups.values().stream()
        .map(group -> group.get(0)).collect(Collectors.toList());
    System.out.println("groupingBy: " + firstFromGroups);

    List<Person> byWrapper = people.stream()
        .map(person -> new Keyed<String, Person>(person.getName(), person))
        .distinct().map(Keyed::value).collect(Collectors.toList());
    System.out.println("wrapper: " + byWrapper);

    List<Person> byPredicate = people.stream()
        .filter(distinctByKey(Person::getName)).collect(Collectors.toList());
    System.out.println("distinctByKey: " + byPredicate);

    Set<String> seen = new HashSet<String>();
    List<Person> bySetFilter = people.stream()
        .filter(person -> seen.add(person.getName()))
        .collect(Collectors.toList());
    System.out.println("Set filter: " + bySetFilter);
  }
}

Use Collectors.toMap() to Keep One Value per Key

Collectors.toMap() builds one map entry for each extracted key. Because this input has repeated names, the three-argument overload supplies a merge function: (first, later) -> first keeps the first person encountered; (first, later) -> later would keep the last. Without a merge function, duplicate keys cause collection to fail.

The example uses LinkedHashMap::new, so the resulting map retains key insertion order for this sequential input. The default map collector does not promise that iteration over values() follows encounter order. If ordering matters, choose and document an ordered map explicitly. Map keys also require suitable equality and hash-code behavior, just like any hash-based collection.

Group Values with Collectors.groupingBy()

Collectors.groupingBy(Person::getName) creates a list of every person for each name. Selecting group.get(0) then keeps the first member of each group for this ordered sequential stream. This is useful when later code needs the full groups; if the goal is only one value per key, toMap() expresses that directly and avoids storing lists of duplicates.

The map returned by the default groupingBy() does not promise key iteration order. The example therefore demonstrates which member is selected, not a portable ordering for the printed groups. Use an explicit map supplier when key order is part of the result contract.

Wrap a Key Before Calling distinct()

The Keyed<K, V> wrapper in the program carries both the extracted key and the original object. Its equals() and hashCode() use only the key, so distinct() compares names and the following map(Keyed::value) recovers the original people. This keeps Person equality unchanged and retains the first matching wrapper in an ordered sequential stream.

A wrapper is useful when a pipeline should remain stream-based or when the key logic belongs in a small reusable type. Ensure both equality methods use the same key; overriding only one breaks the equality contract used by hash-based operations.

Filter with a distinctByKey Predicate

distinctByKey accepts a key extractor and returns a predicate that adds each encountered key to a set. Set.add() returns true only for a key not seen before, so the filter keeps the first person for each name. The helper is convenient when the same rule is reused, but each predicate instance owns mutable state and should be used with a sequential stream.

Do not apply this stateful predicate to a parallel stream: concurrent evaluation can make the result unsafe or dependent on scheduling. For parallel processing, prefer a collector whose merge behavior and ordering requirements are explicitly defined.

Use a Set in a Stateful Filter

The final example shows the same idea directly: a HashSet records names as the filter traverses the stream. This is shorter than a helper when used once, and the resulting list keeps the first matching person in encounter order for the sequential stream. It has the same statefulness and parallel-use limitation as distinctByKey.

Choose the set’s element type to match the property whose uniqueness matters. For a compound key, use a small immutable key type or another value with correct equals() and hashCode() rather than concatenating fields into a string, which can create ambiguous keys.

Choose a Strategy and Preserve the Intended Result

Use toMap() for a direct one-per-key result and specify the merge rule. Use groupingBy() when retaining every member of each group is useful. Use the wrapper when key-based distinct() fits the pipeline without changing the original object’s equality. For a short sequential pipeline, a set-backed filter is compact, but keep its mutable-state restriction visible.

Before choosing, check three details: which duplicate survives, whether the output must preserve encounter order, and whether selected-property equality should apply to the object everywhere or only in this pipeline. These choices determine the right implementation more reliably than a generic claim that one collector is always faster.