Pandas DataFrame Basics: Selecting, Filtering, and Sorting Data

Once you’ve loaded your data into a Pandas DataFrame, the real fun begins. Pandas Filtering and Sorting are essential techniques at this stage. You rarely need all the data. Usually, you need to find specific answers, like “Show me all sales over £500” or “Who are my top 10 customers?” Indeed, mastering Pandas Filtering and Sorting is key to answering these questions effectively.

This guide covers the three most essential skills for answering those questions: Selecting, Filtering, and Sorting. When using these techniques, Pandas Filtering and Sorting ensure you can tackle complex data challenges effortlessly.

For these examples, assume we have this simple DataFrame named df:

1. Selecting Columns

Sometimes you only want to see a specific part of your data, which you can do using Pandas’ selection techniques as part of filtering and sorting tasks.

Select a Single Column

To get just one column, use square brackets with the column name as a string. This returns a Pandas Series (a single column of data).

# Get just the 'Name' column
names = df['Name']
print(names)

Select Multiple Columns

To get multiple columns, you must pass a list of column names inside the brackets. Notice the double brackets [[]].

# Get 'Name' and 'Salary' columns
subset = df[['Name', 'Salary']]
print(subset)

2. Filtering Rows (Conditional Selection)

Filtering is how you ask questions of your data. It works by creating a “mask”—a list of True/False values based on a condition. Efficiently performing Pandas Filtering and Sorting tasks can significantly enhance your data analysis.

Simple Filtering

Let’s find everyone older than 28.

# Step 1: Create the filter (mask)
# This returns True for rows where Age > 28, False otherwise
mask = df['Age'] > 28

# Step 2: Apply the mask to the DataFrame
filtered_df = df[mask]

print(filtered_df)
# Result: Bob (30) and Charlie (35)

Pro-tip: You often see this written in one line:

filtered_df = df[df['Age'] > 28]

Filtering with Multiple Conditions

Use & for “AND” and | for “OR”. Important: You MUST wrap each condition in parentheses ().

# Find people in 'New York' AND with Salary over 71000
ny_high_salary = df[(df['City'] == 'New York') & (df['Salary'] > 71000)]
print(ny_high_salary)
# Result: Diana only

3. Sorting Data

Sorting helps you find your top or bottom performers when dealing with Pandas data. Use the .sort_values() method for effective results.

# Sort by Age (lowest to highest is default)
youngest_first = df.sort_values(by='Age')

# Sort by Salary (highest to lowest)
# Use 'ascending=False' to reverse the order
highest_paid = df.sort_values(by='Salary', ascending=False)

print(highest_paid)
# Result: Bob will be at the top, Charlie at the bottom.

Summary

• Select Columns: df['ColumnName'] or df[['Col1', 'Col2']]
• Filter Rows: df[df['Column'] > value]
• Sort Rows: df.sort_values(by='Column', ascending=False)

Master these three, and you can answer 80% of basic data questions by effectively using Pandas Filtering and Sorting!