How to Remove Duplicate Rows by Column in R

To keep one row for each selected key in an R data frame, use distinct() from dplyr with .keep_all = TRUE when you need the other columns. Each method below keeps the earliest matching row in the data’s current order, so sort first when another record should take priority. The examples use a small data frame with id, group, and score columns.

Use the distinct Function of the dplyr Package to Remove Duplicate Rows by Column in R

Use distinct(df, id, .keep_all = TRUE) when id defines a unique record and you want every original column in the result. distinct() compares the selected key columns and keeps the first row for each key value. With .keep_all = TRUE, other columns are carried into the result but do not affect whether two rows match. Without that argument, the result contains only the selected columns.

For a composite key, list each column: distinct(df, id, group, .keep_all = TRUE). Rows with the same id but different group values then remain separate. The following complete example uses the same sample data and shows both key choices. It requires dplyr to be installed; its results are described below, and this example is not accompanied by runtime output.

df <- data.frame(
  id = c(1, 2, 2, 3, 3),
  group = c("A", "A", "B", "A", "B"),
  score = c(8, 5, 9, 4, 7)
)

if (requireNamespace("dplyr", quietly = TRUE)) {
  unique_by_id <- dplyr::distinct(df, id, .keep_all = TRUE)
  unique_by_pair <- dplyr::distinct(df, id, group, .keep_all = TRUE)
  print(unique_by_id)
  print(unique_by_pair)
} else {
  cat("Install dplyr to run these distinct() examples.\n")
}

The single-key result retains (1, A, 8), (2, A, 5), and (3, A, 4). The composite-key result retains those rows plus (2, B, 9) and (3, B, 7), because each id and group pair is unique. The following separate base R example prints both results.

df <- data.frame(
  id = c(1, 2, 2, 3, 3),
  group = c("A", "A", "B", "A", "B"),
  score = c(8, 5, 9, 4, 7)
)

unique_by_id <- df[!duplicated(df$id), ]
unique_by_pair <- df[!duplicated(df[c("id", "group")]), ]

cat("By id:\n")
cat(paste(unique_by_id$id, unique_by_id$group, unique_by_id$score, sep = ":"), sep = "\n")
cat("\nBy id and group:\n")
cat(paste(unique_by_pair$id, unique_by_pair$group, unique_by_pair$score, sep = ":"), sep = "\n")

By id:
1:A:8
2:A:5
3:A:4

By id and group:
1:A:8
2:A:5
2:B:9
3:A:4
3:B:7

Here, !duplicated(df$id) marks later occurrences of each id; passing a data frame of key columns to duplicated() applies the same first-occurrence rule to their combination. Both selections retain all columns from each chosen row.

Use group_by, filter and duplicated Functions to Remove Duplicate Rows by Column in R

For a single global id key, filter(!duplicated(id)) keeps its first occurrence. If the input may already be grouped, call ungroup() first so filter() evaluates the duplicate check over the full data:

df %>% ungroup() %>% filter(!duplicated(id))

The pipe %>% passes the value on its left as the first argument to the function on its right. Grouping by id before filter(!duplicated(id)) does not remove later rows: inside each id group, that key is constant, so duplicated(id) finds no later occurrence. Grouping by another column is useful when IDs can repeat independently within each group; then duplicated(id) keeps the first occurrence of each ID in every group. For a single global key, the ungrouped expression has the same first-row behavior as the base R selection above.

For example, grouping by group applies the same check separately within each group:

df %>%
  group_by(group) %>%
  filter(!duplicated(id)) %>%
  ungroup()

This pipeline keeps the first row for each id within each group, according to the current row order. In the sample data, every id and group pair occurs only once, so each row is the first occurrence of its id within its group and all five rows are retained; a further row repeating an id already seen in the same group would be dropped.

Use group_by and slice Functions to Remove Duplicate Rows by Column in R

The dplyr pipeline df %>% group_by(id) %>% slice(1) %>% ungroup() selects position 1 from each id group. The current row order determines which record is first; slice(1) does not automatically select the highest score or newest record. Arrange the data by the preference you want before grouping and slicing, then ungroup when later operations should treat the result as an ordinary data frame.

For the sample data, this method retains the same rows as the single-key distinct() and duplicated() approaches: the first row for IDs 1, 2, and 3. Use distinct() when the task is selecting unique key combinations; use slice(1) when selecting a row by its position within each group is part of a broader grouped workflow. These dplyr expressions are illustrative and require the dplyr package; the base R example above runs without that dependency.