Marathon Data

Numbers That Look Like Text

We already explored the marathon registrations file in lesson 1. It looked clean, but pandas stored the registered_date column as plain text. You can't sort it or compute differences until you fix the type. pd.to_datetime(), astype(), and pd.to_numeric() fix that.

We explored the registrations file in the first lesson. It looked clean compared to the results. But 'looking clean' is not the same as having the right types. Let's check what pandas actually stored.

Let's check the registered_date column in our marathon registrations. The values look like dates, but check the dtype:

import pandas as pd
 
reg = pd.read_csv('/data/marathon_registrations.csv')
print(reg['registered_date'].dtype)
print(reg['registered_date'].head(3).tolist())

Python

Output

pd.to_datetime() converts a string column to a proper datetime type:

import pandas as pd
 
reg = pd.read_csv('/data/marathon_registrations.csv')
reg['registered_date'] = pd.to_datetime(
    reg['registered_date']
)
print(reg['registered_date'].dtype)

Python

Output

What will be the output?

import pandas as pd
 
s = pd.Series([
    '2024-03-10', '2024-05-22'
])
converted = pd.to_datetime(s)
print(converted.dtype)

Python

Once a column is datetime, the .dt accessor unlocks date and time properties like month, year, day, and weekday.

Now that registered_date is a real datetime, we can pull out parts of it. Let's extract the month and year:

import pandas as pd
 
reg = pd.read_csv('/data/marathon_registrations.csv')
reg['registered_date'] = pd.to_datetime(
    reg['registered_date']
)
reg['reg_month'] = reg['registered_date'].dt.month
reg['reg_year'] = reg['registered_date'].dt.year
print(reg[['reg_month', 'reg_year']].head(3))

Python

Output

What will be the output?

import pandas as pd
 
s = pd.Series(['2024-06-15'])
s = pd.to_datetime(s)
print(s.dt.month[0])

Python

astype() converts a column to any compatible type:

df['col'] = df['col'].astype(str)
df['col'] = df['col'].astype(int)
df['col'] = df['col'].astype(float)

Python

In our marathon data, runner_id is stored as a number. But it's really an identifier: you'd never add two runner IDs together. Let's convert it:

import pandas as pd
 
reg = pd.read_csv('/data/marathon_registrations.csv')
print(reg['runner_id'].dtype)
reg['runner_id'] = reg['runner_id'].astype(str)
print(reg['runner_id'].dtype)

Python

Output

What will be the output?

import pandas as pd
 
s = pd.Series([10, 20, 30])
print(s.dtype)
converted = s.astype(str)
print(converted.dtype)

Python

Sometimes numeric columns contain invalid entries like 'N/A'. astype(int) would raise an error. Use pd.to_numeric(errors='coerce') instead.

errors='coerce' turns any value that cannot be converted into NaN:

import pandas as pd
 
s = pd.Series(['245', '190', 'N/A', '210'])
result = pd.to_numeric(s, errors='coerce')
print(result.tolist())

Python

Output

What will be the output?

import pandas as pd
 
s = pd.Series(['10', 'bad', '30'])
result = pd.to_numeric(s, errors='coerce')
print(result.tolist())

Python

What will be the output?

import pandas as pd
 
s = pd.Series(['1', '2', 'oops', '4'])
result = pd.to_numeric(s, errors='coerce')
print(result.isna().sum())

Python