Pandas DataFrames: Tables in Python

Pandas is a third-party library for working with table-shaped data. Its central type is the DataFrame: a table with rows and columns.

import pandas as pd

sessions = [
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
    {"subject": "math", "minutes": 60, "date": "2026-05-12"},
]

df = pd.DataFrame(sessions)
print(df)

The list contains dictionaries, one per row. Each dictionary key becomes a column name. This matches the plain record shape that the study log from Structuring Your Data writes to JSON, and it is also the shape produced by calling to_dict() on a Session object.

Some exercises in this chapter use plain dictionaries directly so you can focus on the pandas operation. In the full class-based project, add the small conversion step from Session objects to dictionaries before building the DataFrame.

A Local CSV for Practice

The exercises in this part are still graded by calling your functions directly. That keeps the tests focused on the pandas operation you are practicing. For your own local testing, though, it is useful to have a slightly larger CSV file.

Create a file named study_sessions.csv in the same folder as your Python file. Copy the CSV text below into that file:

subject,minutes,date
math,45,2026-05-01
history,30,2026-05-02
programming,60,2026-05-02
math,25,2026-05-03
writing,40,2026-05-04
biology,35,2026-05-05
programming,90,2026-05-06
history,50,2026-05-07
math,75,2026-05-08
writing,20,2026-05-09
biology,55,2026-05-10
programming,45,2026-05-11
math,60,2026-05-12
history,35,2026-05-12
writing,50,2026-05-13
programming,30,2026-05-14
biology,40,2026-05-14
math,90,2026-05-15
history,25,2026-05-15
programming,80,2026-05-16
biology,65,2026-05-17
writing,30,2026-05-18
math,50,2026-05-19
programming,70,2026-05-20

Pandas can read the file directly:

import pandas as pd

df = pd.read_csv("study_sessions.csv")

print(df.head())
print(df.shape)
print(df.dtypes)

This file has the same three columns used throughout the study-log examples: subject, minutes, and date. When you use pd.read_csv, pandas often converts columns that look numeric into numbers, so minutes should already be an integer column. Still inspect df.dtypes; checking what pandas read is part of the routine.

For date comparisons, convert the date column after reading:

df["date"] = pd.to_datetime(df["date"])

When an exercise expects a list of session dictionaries instead of a DataFrame, you can make one from the CSV:

sessions = pd.read_csv("study_sessions.csv").to_dict("records")

That gives you records with the same shape as the inline examples:

{"subject": "math", "minutes": 45, "date": "2026-05-01"}

Inspecting a DataFrame

Before summarizing a DataFrame, inspect it:

print(df.head())
print(df.shape)
print(df.columns)

.head() shows the first few rows. .shape gives a pair: number of rows and number of columns. .columns shows the column names.

This is the pandas version of the habit you have practiced all along: inspect the data before trusting code that processes it. If a summary is wrong, the first question is often whether the data has the shape you thought it had.

The first routine to practice is small but important: turn study-log records into the same DataFrame shape every time.

records = [
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
]

Rows, Columns, and the Index

In the study log, each row is one session. Each column is one field of that session:

When pandas prints a DataFrame, you will also see a leftmost set of numbers. That is the index:

   subject  minutes        date
0     math       45  2026-05-10
1  history       30  2026-05-11
2     math       60  2026-05-12

The index is not one of your three data columns. It is pandas’ label for each row. For now, treat it mostly as row numbering. Later pandas work can use the index more deliberately, but this part does not need that.

Worked Table Walkthrough

Here is the same sample again, this time with the printed output included:

import pandas as pd

sessions = [
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
    {"subject": "math", "minutes": 60, "date": "2026-05-12"},
]

df = pd.DataFrame(sessions)
print(df)

Output:

   subject  minutes        date
0     math       45  2026-05-10
1  history       30  2026-05-11
2     math       60  2026-05-12

Read this as a table:

• there are three rows, numbered 0, 1, and 2,
• there are three data columns, subject, minutes, and date,
• the first row says math, 45 minutes, on 2026-05-10.

Now inspect the shape:

print(df.shape)
print(list(df.columns))

Output:

(3, 3)
['subject', 'minutes', 'date']

The shape (3, 3) means three rows and three columns. The column list confirms the names you must use later. Pandas is strict about these names: "minutes" and "minute" are different names.

Columns and Series

Selecting one column gives a Series:

minutes = df["minutes"]
print(minutes)
print(minutes.sum())

A Series is like one labeled column from a DataFrame. It can be summarized:

print(df["minutes"].sum())
print(df["minutes"].mean())
print(df["minutes"].max())

The square brackets are not list indexing in the old sense. With a DataFrame, df["minutes"] means “give me the column named minutes.”

sessions = [
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
    {"subject": "math", "minutes": 60, "date": "2026-05-12"},
]

["math", "history", "math"]

Filtering Rows

Filtering in pandas uses a Boolean condition:

long_mask = df["minutes"] >= 45
long_sessions = df[long_mask]

print(long_sessions)

This is the same mask idea from NumPy. The condition creates one Boolean value per row. Passing that Boolean Series back into the DataFrame selects the matching rows.

Many pandas programmers write this in one line:

long_sessions = df[df["minutes"] >= 45]

That is fine once you recognize the parts. While learning, it is often clearer to keep the mask named.

The next exercise also checks the empty case. When you build a DataFrame from a possibly empty list of session records, give pandas the expected columns explicitly:

df = pd.DataFrame(sessions, columns=["subject", "minutes", "date"])

That way, sessions=[] still produces an empty table with the right column names.

Tracing a Filter by Hand

Take this filter:

long_mask = df["minutes"] >= 45
long_sessions = df[long_mask]

With the three-row sample DataFrame:

   subject  minutes        date
0     math       45  2026-05-10
1  history       30  2026-05-11
2     math       60  2026-05-12

the mask is:

0     True
1    False
2     True
Name: minutes, dtype: bool

The index labels line up. Row 0 stays because its mask value is True. Row 1 is removed because its mask value is False. Row 2 stays.

So long_sessions becomes:

  subject  minutes        date
0    math       45  2026-05-10
2    math       60  2026-05-12

Notice that the index labels are still 0 and 2. Pandas did not renumber the rows just because one row was filtered out. This is normal. The index is a label, not a promise that the visible rows will always be numbered 0, 1, 2.

For this course, the index usually does not matter. But seeing 0 and 2 after filtering can surprise you the first time. The important part is that the two matching rows stayed.

Selecting Multiple Columns

One column uses one set of square brackets:

df["minutes"]

Multiple columns use a list of column names:

df[["subject", "minutes"]]

The double brackets can look odd at first. The outer brackets are DataFrame selection. The inner brackets are the Python list of column names.

This returns a DataFrame, not a Series, because the result is still table-shaped. It has fewer columns than the original, but it can still have many rows and many columns.

Common Mistake: Misspelled Column Names

If you ask for a column that does not exist, pandas raises a KeyError:

print(df["minute"])   # KeyError: 'minute'

This is the pandas version of a dictionary key error. A DataFrame column name is a key. If the DataFrame has minutes, then minute, Minutes, and duration are all different names.

When this happens, inspect the columns:

print(list(df.columns))

Then compare the exact spelling. Do this before changing many lines. A single missing s can make a correct idea look broken.

print(df["minute"])

A Helper for the Study Log

The project checkpoint will use a helper like this. It starts from the Session objects inside the StudyLog from Structuring Your Data, converts them to plain dictionaries, and then builds the table:

import pandas as pd

def sessions_dataframe(sessions):
    records = []
    for session in sessions:
        records.append(session.to_dict())

    df = pd.DataFrame(records, columns=["subject", "minutes", "date"])
    df["date"] = pd.to_datetime(df["date"])
    return df

The loop creates the plain records pandas expects. The pd.DataFrame(...) line creates the table. The final line converts the date column from strings to pandas date values. This is similar to date.fromisoformat, but applied to a whole column.

If the input list contains two sessions whose to_dict() methods produce:

[
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
]

then the DataFrame has three columns: subject, minutes, and date.

sessions = [
    {"subject": "math", "minutes": 45, "date": "2026-05-10"},
    {"subject": "history", "minutes": 30, "date": "2026-05-11"},
    {"subject": "programming", "minutes": 90, "date": "2026-05-12"},
]

long_session_subjects(sessions, 45) -> ["math", "programming"]
long_session_subjects(sessions, 100) -> []

Common Mistake: Numbers Stored as Text

Pandas only gives a numeric minutes total if the minutes column contains numbers. If the values are strings, .sum() may join the strings together, or a mixed column may fail with a type error.

Compare these two records:

good = {"subject": "math", "minutes": 45, "date": "2026-05-10"}
bad = {"subject": "math", "minutes": "45", "date": "2026-05-10"}

The printed table may look almost the same, but the types differ. You can inspect the column types with:

print(df.dtypes)

dtypes means “data types”. It tells you what kind of values pandas believes each column contains.

For the study log, minutes should be an integer column. If the Session objects from Structuring Your Data store minutes as integers, pandas receives good data. If a loader or constructor leaves minutes as text, fix that boundary or convert the column before summarizing.

df["minutes"] = df["minutes"].astype(int)

The .astype(int) call asks pandas to convert the column values to integers. If a value cannot be converted, that is useful information: the data should be fixed before the summary is trusted.

The better design is to validate and convert at the boundary where data is loaded. Then summary functions can trust that minutes are numbers.

A Small Inspection Checklist

Before writing a pandas summary, run a few inspection lines:

print(df.head())
print(df.shape)
print(list(df.columns))
print(df.dtypes)

Together, these answer the practical questions:

• Do the rows look like sessions?
• How many rows and columns are there?
• Are the column names exactly what the code will use?
• Is minutes numeric?
• Has date been converted, or is it still text?

This is the pandas version of printing intermediate variables in a loop. You do not need to leave all of these lines in the final program, but they are extremely useful while building the helper functions. If groupby gives a strange result, inspect first. Do not guess first.

Small Pieces Beat Dense Chains

Pandas makes it possible to write very dense code. Dense code can be satisfying, but it is not the goal here. Prefer named intermediate results while learning:

long_mask = df["minutes"] >= 45
long_sessions = df[long_mask]
subjects = list(long_sessions["subject"])

This is longer than a one-line chain, but each line has a name and can be printed while debugging.

long_mask = df["minutes"] >= 45
long_sessions = df[long_mask]
subjects = list(long_sessions["subject"])

Check Your Understanding

1. What does pd.DataFrame(sessions) do when sessions is a list of dictionaries?
2. What is the difference between a DataFrame and a Series?
3. What does df["minutes"] >= 45 produce?
4. Why is it useful to name a mask before using it?