Data Analysis Fundamentals

Exploring a Dataset

The Pandas series taught how to load and filter DataFrames. This lesson covers what analysts do first when facing an unknown dataset: checking its size, column types, and summary statistics with .shape, .dtypes, and .describe().

The Pandas series covered loading and filtering DataFrames. But when facing an unknown dataset, where does analysis actually start? With a few quick checks that reveal what the data contains.

The first question: how much data is there? .shape answers instantly:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
print(df.shape)

Python

Output

49 rows, 6 columns. That single line tells you the size. The result is a tuple: rows first, columns second.

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'A': [1, 2, 3],
    'B': [4, 5, 6]
})
print(df.shape)

Python

Next: what information does each column hold? .columns lists every column name:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
print(df.columns)

Python

Output

Column names alone don't reveal whether values are text or numbers. .dtypes answers that:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
print(df.dtypes)

Python

Output

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'Name': ['Alice', 'Bob'],
    'Score': [85, 92]
})
print(df['Score'].dtype)

Python

.describe() gives you count, mean, standard deviation, min, and max for every numeric column. All in one call, no loops.

Pull out just the averages by accessing the 'mean' row of the describe() result:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
pd.options.display.float_format = '{:,.2f}'.format
stats = df.describe()
print(stats.loc['mean'])

Python

Output

Notice the formatting: Large numbers display with commas (e.g., 3,959,975.00) instead of scientific notation (e.g., 3.959975e+06). This is because we included the line pd.options.display.float_format = '{:,.2f}'.format.

This is worth adding to every analysis script. It configures how floating-point numbers are displayed across your entire Python session. Without it, large numbers become hard to read.

Without the formatter, the same output looks like this:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
stats = df.describe()
print(stats.loc['mean'])

Python

Output

Scientific notation (3.959975e+06) is compact but hard to interpret at a glance. Formatted numbers with commas are immediately readable: 3,959,975 is clearly about 4 million.

Best practice: Add pd.options.display.float_format = '{:,.2f}'.format to the top of your analysis script, right after importing pandas. It makes all numeric output human-readable for the rest of your session.

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'V': [10, 20, 30]
})
stats = df.describe()
print(stats.loc['mean', 'V'])

Python

Glancing at actual records helps confirm the data loaded correctly. .tail() shows the last few rows:

import pandas as pd
 
df = pd.read_csv('/data/companies.csv')
print(df.tail(3))

Python

Output

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'X': [10, 20, 30, 40, 50]
})
r = df.tail(2)['X'].tolist()
print(r)

Python

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'A': [1], 'B': [2], 'C': [3]
})
rows, cols = df.shape
print(cols)

Python

What will be the output?

import pandas as pd
df = pd.DataFrame({
    'V': [5, 15]
})
s = df.describe()
print(s.loc['max', 'V'])

Python