Filter Pandas DataFrame Rows by Column Values

Filtering rows means building one Boolean value for every row and keeping the rows whose value is True. In pandas, the clearest general form is df.loc[condition]. Use df.loc[condition, columns] when the result should also contain only selected columns.

This article uses one small order table to show exact matches, multiple conditions, lists, ranges, dates, missing values, text searches, and DataFrame.query(). The examples use current Python 3 syntax and the pandas 3.0 API.

Create the Example DataFrame

Create a DataFrame with text, numeric, date, and missing values so each filtering case is concrete:

import pandas as pd

orders = pd.DataFrame(
    {
        "order_id": [101, 102, 103, 104, 105, 106],
        "region": ["East", "West", "East", "North", "West", "East"],
        "product": ["Desk", "Chair", "Desk lamp", "Desk", None, "Chair"],
        "sales": [450, 275, 125, 700, 320, 410],
        "status": ["paid", "pending", "paid", "paid", "cancelled", "pending"],
        "ordered_at": pd.to_datetime(["2026-08-01", "2026-08-03", "2026-08-08", "2026-08-11", "2026-08-14", "2026-08-18"]),
    }
)

All later snippets assume that orders has already been created. Filtering does not reset the original index, so displayed row labels remain useful for tracing a result back to the source table. Call .reset_index(drop=True) only if a new zero-based index is actually required.

Filter Rows by One Column Value

Compare a Series with the required value to create a Boolean mask. Apply the mask with .loc:

east_orders = orders.loc[orders["region"] == "East"]
print(east_orders[["order_id", "region", "sales"]].to_string(index=False))

Output:

 order_id region  sales
      101   East    450
      103   East    125
      106   East    410

orders[orders["region"] == "East"] selects the same rows. .loc is often more expressive because its second argument can select output columns at the same time:

large_orders = orders.loc[orders["sales"] >= 400, ["order_id", "sales"]]
print(large_orders.to_string(index=False))

Output:

 order_id  sales
      101    450
      104    700
      106    410

Use != for the complement of an exact match. Be careful when the column may contain missing values: a comparison is not a substitute for an explicit missing-value test. The dedicated isna() and notna() methods make that intention clear.

Combine Multiple Conditions

Use & when every condition must be true, | when at least one must be true, and ~ to negate a mask. Put each comparison in parentheses because Python’s operator precedence otherwise changes how the expression is evaluated.

mask = (orders["region"] == "East") & (orders["sales"] >= 400)
result = orders.loc[mask, ["order_id", "region", "sales"]]
print(result.to_string(index=False))

Output:

 order_id region  sales
      101   East    450
      106   East    410

This | example keeps paid orders or orders with sales above 600:

mask = (orders["status"] == "paid") | (orders["sales"] > 600)
result = orders.loc[mask, ["order_id", "status", "sales"]]

Do not replace & and | with Python’s scalar and and or. A Series contains many truth values, so pandas cannot reduce it to the one truth value those operators expect:

# Raises ValueError: The truth value of a Series is ambiguous.
orders.loc[(orders["region"] == "East") and (orders["sales"] >= 400)]

This failure usually identifies a missing elementwise operator or missing parentheses rather than bad data.

Filter for a List of Values

Use Series.isin() instead of writing a chain of equality comparisons. Pass a list, set, tuple, Series, or another list-like collection; a single bare string is not valid input.

allowed = ["East", "West"]
result = orders.loc[orders["region"].isin(allowed), ["order_id", "region"]]
print(result.to_string(index=False))

Output:

 order_id region
      101   East
      102   West
      103   East
      105   West
      106   East

Prefix the membership mask with ~ to exclude values:

excluded = ["cancelled", "pending"]
paid_only = orders.loc[~orders["status"].isin(excluded)]

Negate the complete result of isin(), as shown. Writing ~orders["status"] tries to invert the values themselves rather than the Boolean membership mask.

Filter Ranges and Dates

Series.between(left, right) is concise for an inclusive interval. Its default includes both endpoints; set inclusive explicitly when a boundary must be open.

mid_value = orders.loc[orders["sales"].between(300, 500), ["order_id", "sales"]]
print(mid_value.to_string(index=False))

Output:

 order_id  sales
      101    450
      105    320
      106    410

The same approach works for datetimes. Convert the column once with pd.to_datetime() when loading or cleaning data, then compare it with compatible timestamps:

date_mask = orders["ordered_at"].between("2026-08-03", "2026-08-14")
result = orders.loc[date_mask, ["order_id", "ordered_at"]]

For timezone-aware data, make the boundaries timezone-aware too. Comparing aware and naive timestamps raises an error instead of silently guessing a timezone.

Filter Missing or Non-Missing Values

Use isna() to select None, NaN, or NaT, and notna() to require a present value. Do not compare with None or float("nan") when the task is to identify all pandas missing values.

missing_product = orders.loc[orders["product"].isna(), ["order_id", "product"]]
present_product = orders.loc[orders["product"].notna()]
print(missing_product.to_string(index=False))

Output:

 order_id product
      105    None

An empty string is not automatically a missing value. If the source uses both empty strings and nulls, normalize the column first or combine orders["product"].isna() with orders["product"].eq("").

Filter Text Values

The vectorized string accessor creates a mask without a Python loop. For a literal substring, set regex=False; otherwise characters such as ., +, and [ are interpreted as regular-expression syntax. Set na=False so missing product names do not leave an unusable missing value in the mask.

has_desk = orders["product"].str.contains("desk", case=False, regex=False, na=False)
result = orders.loc[has_desk, ["order_id", "product"]]
print(result.to_string(index=False))

Output:

 order_id   product
      101      Desk
      103 Desk lamp
      104      Desk

Use regex=True only when a regular expression is intentional. Methods such as str.startswith() and str.endswith() are clearer when the match belongs at a specific end of the string.

Use DataFrame.query() for Readable Expressions

DataFrame.query() can make a long column-oriented expression easier to scan. Prefix a local Python variable with @, and surround column names that contain spaces or punctuation with backticks.

minimum_sales = 400
result = orders.query("region == 'East' and sales >= @minimum_sales")
print(result[["order_id", "region", "sales"]].to_string(index=False))

Output:

 order_id region  sales
      101   East    450
      106   East    410

Within pandas’ default query syntax, and and or are supported. That does not make scalar and and or valid between Series masks outside query().

Do not build a query expression by concatenating untrusted user input. DataFrame.query() evaluates an expression and its official documentation warns that malicious input can run arbitrary code. For external values, validate them and use ordinary masks where possible.

Choose .loc, Brackets, or query()

Use .loc[mask] as the general default, especially when selecting rows and output columns together. Plain df[mask] is concise and equivalent for row-only Boolean selection. Use query() when its expression is genuinely easier to read and the expression is controlled by the application.

All three forms produce a filtered object; they do not mean “make a guaranteed, independent working copy.” If the next step mutates the result, state that boundary explicitly:

east_orders = orders.loc[orders["region"] == "East"].copy()
east_orders["sales_with_tax"] = east_orders["sales"] * 1.2

Keeping the mask in a named variable also helps when it must be inspected, reused, or tested. Prefer vectorized comparisons over looping through rows, and make data types consistent before filtering. Those habits keep the condition aligned with the DataFrame index and make selected rows predictable.

Avoid Alignment and Data-Type Surprises

A pandas mask is a labeled Series, not merely an unlabelled sequence of true and
false values. When a Boolean Series is passed to .loc, pandas aligns it with the
DataFrame’s index. That is valuable when both objects describe the same rows, but
it can cause an IndexingError if the mask was created from another DataFrame or
before the source index changed. Build the mask from the DataFrame being filtered,
or deliberately reindex it before use. Do not silence an alignment problem by
converting to a list unless positional selection is genuinely intended.

Data types determine comparison behavior as well. Numeric text such as "450"
does not become the number 450 merely because it looks numeric. Convert a column
with pd.to_numeric() and decide how invalid entries should be handled before
applying a numeric range. Likewise, convert date text with pd.to_datetime()
instead of relying on lexicographic string ordering. A consistent dtype turns a
filter from an accidental text comparison into the intended numeric or temporal
comparison.

Case, whitespace, and categorical spelling matter for exact text comparisons.
If input is not normalized, "East", "east", and "East " are different
values. Clean the Series explicitly with string methods when the business rule
considers them equivalent. Do not normalize automatically when case or whitespace
is meaningful data.

Reuse and Debug Boolean Masks

For a complex filter, name each meaningful condition before combining it. A named
mask can be printed, counted, or checked against known row IDs independently. This
is easier to debug than a single expression containing many operators. The final
mask must have one Boolean result per source row, with the same index.

Use mask.value_counts(dropna=False) to see how many rows pass, fail, or remain
unknown. Use orders.loc[mask.fillna(False)] only after deciding that an unknown
condition should exclude the row; filling missing results is a data-policy choice,
not a universal fix. Methods such as isna() and the na argument of string
operations are usually clearer because they encode that decision where the mask
is created.

When no rows match, pandas returns an empty DataFrame with the expected columns;
it does not raise an exception. Check result.empty when an empty selection needs
special handling. Conversely, a filter may return every row if a condition is too
broad. Assertions against expected IDs or row counts are useful in data pipelines
where an unexpectedly empty or full result indicates a bad input or rule.

Filter Without Changing the Source Data

Row selection and row modification are separate operations. Filtering first is
appropriate for reporting, exporting, aggregation, and other read-only work. If
the intent is to update the original DataFrame, assign through .loc on that
original object instead of modifying an uncertain intermediate result. For
example, the shape is orders.loc[mask, "status"] = "review".

If the intent is to create a new working dataset, use .copy() as shown earlier.
That choice makes ownership explicit and avoids relying on whether an intermediate
selection happens to share memory. It also prevents a later edit to the filtered
table from being mistaken for an edit to orders. Choose between direct .loc
assignment and an independent copy according to the task, rather than depending
on chained indexing behavior.

Finally, DataFrame.filter() is not the method for filtering rows by their cell
values. Despite its name, it selects axis labels by exact name, substring, or
regular expression. Use Boolean masks or query() for the value-based task in
this article. Keeping label selection and value selection distinct avoids a common
API-name trap.

Official references: DataFrame.loc, Series.isin, Series.between, Series.str.contains, and DataFrame.query.