Module 2 Lab Companion - BAN 6003 (AI pilot)

How to use this companion

Open the Module 2 notebook and this companion at the same time. Work through the companion from top to bottom and run the matching notebook cells as you go. The notebook is the authoritative source for the lab sequence, code and required work. This companion explains what each step means, what to notice in the output and why the step matters.

Read aloud: controls and directions

Before you start: choose a reading mode, speed, and voice below. Select Read Aloud from Beginning to hear the full companion, or use Read Aloud from This Section beside any major section to continue from that point. While audio is playing, a floating control bar remains visible at the bottom of the page. Use Pause to pause temporarily, Resume to continue, and Stop to end the current reading.

Study mode reads explanations, results, tables, and notes while skipping exact Python code. Full mode also reads Python code and verbalizes common programming symbols.

Reading mode: Speed: 1.0× Voice: Jump to section:

Ready to read.

The read-aloud feature uses your browser's speech engine. A screen reader can access the page independently of these controls.

What is in the notebook, and what this companion adds

Notebook: the exact Python code, the Your Turn tasks, the final practice, the offline assignment and the written reflections.
Companion: plain-language explanations, the results you should expect (taken from a real run of the notebook), sample solutions for the Your Turn tasks (folded, so you try first), common mistakes and how to fix them.
If anything ever seems inconsistent: follow the current notebook.

Notebook contentCode, official prompts, required tasks and the lab sequence come from the notebook. Each code block shows its notebook cell number.

Companion-added instructor noteThese clearly labeled notes are explanations, cautions or study advice added by this companion, many of them from the in-person lab session. They are not extra required notebook steps unless the note says so.

This companion is designed for keyboard and screen-reader access, but it is not a separate version for students who need accessibility features. It is the same learning guide for everyone. You can read it visually, use a screen reader, or use the built-in read-aloud controls while you work in the notebook.

Use either course environment

GitHub Codespaces (main path): open your own repository in Codespaces, open the Module 2 notebook and use the default Python kernel.
Local VS Code (optional, for experienced students): clone your repository, use the base conda environment, install requirements.txt, open the repository folder in Visual Studio Code and open the notebook.
In either environment: the notebook and this companion are the same. Keep the repository structure unchanged so the notebook paths and output folders work.

↑ Back to Lab Contents

Module 2 Learning Goals

Last week you asked where data lives. This week you ask what condition the data is in. You load a small employee table with Python and Pandas and profile it: you inspect its structure, content and quality before anyone fixes anything. By the end of the lab you should be able to:

Run basic Python code in a notebook and read the result.
Explain why Pandas is useful for tables of rows and columns.
Load a CSV file with pd.read_csv().
Inspect a table with head(), tail(), shape, columns, info() and dtypes.
Count missing values with isna().sum(), summarize numbers with describe() and categories with value_counts().
Find possible duplicate IDs, and write short observations based on evidence.

The main idea: profile first, clean later. This week you describe problems; you do not fix them.

↑ Back to Lab Contents

Before You Start: Repository and Setup

Module 2 uses the same repository you created in Module 1 (the Weeks 1–2 template). Do not start a new one, and do not work in the instructor template.

Open your own repository in GitHub Codespaces. A new Codespace usually needs 4–5 minutes the first time; reopening it is much faster.
Open notebooks/Week2_lab1_python_pandas_guided_lab.ipynb. Check the file name: the folder also holds Week1_introduction_to_codespaces.ipynb and Week2_assignment.ipynb.
If VS Code asks for a kernel, choose the Python environment it suggests. Then run the cells from the top. If you restart the kernel, run from the top again.

Optional local path: in GitHub Desktop, select your repository and choose Open in Visual Studio Code. Trust the folder when VS Code asks, and choose the base conda environment as the kernel. The screen looks almost the same as in Codespaces, because Codespaces also uses VS Code.

↑ Back to Lab Contents

Part 1: Python Basics You Need for This Course

This is not a full Python course. You need to run cells, read code carefully, change small pieces of example code and explain the output.

1.1 Python as a calculator

# If you run a 10 kilometer race in 43 minutes and 30 seconds,
# what is your average seconds per mile?
# 1 mile = 1.61 kilometers

seconds = 43 * 60 + 30
miles = 10 / 1.61
seconds_per_mile = seconds / miles

seconds_per_mile

Result

420.21000000000004

About 420 seconds, or 7 minutes per mile. The long tail of digits is normal: computers store most decimals with a tiny rounding error. The cell creates three variables: seconds, miles and seconds_per_mile.

1.2 Store values in variables

name = "Alice"
age = 28
salary = 95000.0
is_manager = False

print(name)
print(age)
print(salary)
print(is_manager)

print(type(name), type(age), type(salary), type(is_manager))

Result

Alice
28
95000.0
False
<class 'str'> <class 'int'> <class 'float'> <class 'bool'>

Four types: str (text), int (whole number), float (number with a decimal point) and bool (True or False).

1.3 Comments

# This is a comment. It explains what the code does.

message1 = "It's not a bug; it's a feature."
print(message1)

Result

It's not a bug; it's a feature.

Python ignores everything after # on that line. A # works for one line only; for several comment lines, start each line with #.

1.4 Lists

departments = ["Engineering", "Marketing", "Sales"]

print(departments)
print(departments[0])  # first element
print(departments[1])  # second element

Result

['Engineering', 'Marketing', 'Sales']
Engineering
Marketing

A list uses square brackets and commas. Positions start at 0: departments[0] is the first element. In a list of 100 elements, the last one is at position 99.

Change a list element

vector1 = [1, 2, 3]
vector1[0] = 100
print(vector1)

Result

[100, 2, 3]

Multiplying a list repeats it

vector1 = [100, 2, 3]
print(vector1 * 2)

Result

[100, 2, 3, 100, 2, 3]

* 2 repeats the list. It does not double each number. (If you know R, this is a key difference.)

Double each element

# Loop version
new_vector = []
for x in vector1:
    new_vector.append(x * 2)
print(new_vector)

# Optional: list comprehension version
[x * 2 for x in vector1]

Result

[200, 4, 6]
[200, 4, 6]

Both versions give the same result. You do not need to master loops today.

1.5 Dictionaries

employee = {
    "name": "Bob",
    "department": "Marketing",
    "salary": 78000
}

print(employee["name"])
print(employee["department"])
print(employee["salary"])

Result

Bob
Marketing
78000

A dictionary uses curly brackets and holds key: value pairs. You look up a value with its key in square brackets: employee["name"].

1.6 Errors and help()

help(print)

↑ Back to Lab Contents

Part 2: What Is Pandas?

Pandas is the main Python tool for tables. You will use it to load, inspect, filter, summarize, group, clean and transform data. Modeling comes later, with scikit-learn.

2.1 Import Pandas

import pandas as pd
print("Pandas imported successfully!")

Result

Pandas imported successfully!

2.2 A DataFrame and a Series

sample = pd.DataFrame({
    "employee_id": [1, 2, 3],
    "department": ["Sales", "Marketing", "Engineering"],
    "salary": [60000, 75000, 90000]
})

sample

Result

   employee_id   department  salary
0            1        Sales   60000
1            2    Marketing   75000
2            3  Engineering   90000

The dictionary keys become column names and each list becomes a column.

Select one column

sample["department"]

Result

0          Sales
1      Marketing
2    Engineering
Name: department, dtype: object

One column is a Series. It also shows the index. Typing a name on the last line of a cell displays it, without print(). Depending on your Pandas version, the last line may say dtype: object or dtype: str; both mean text.

2.3 Methods and attributes

print("DataFrame shape attribute:")
print(sample.shape)

print("DataFrame head method:")
sample.head()

Result

DataFrame shape attribute:
(3, 3)
DataFrame head method:

Then the table is shown. shape has no parentheses: it is an attribute, stored information (3 rows, 3 columns). head() has parentheses: it is a method, an action. sample.shape() would give an error.

↑ Back to Lab Contents

Part 3: Data Import

The file is data/employee_data.csv. A CSV file stores a table as plain text, with commas between values.

3.1 Reading a CSV without Pandas

import csv

csv_rows = []
with open("../data/employee_data.csv", "r") as file:
    csv_reader = csv.reader(file)
    for row in csv_reader:
        csv_rows.append(row)

# Show only the first 3 rows so the output is not too long
csv_rows[:3]

Result

[['\ufeffemployee_id',
  'first_name',
  'last_name',
  'hire_date',
  'department',
  'salary',
  'age',
  'full_time'],
 ['101', 'John', 'Smith', '4/15/22', 'Marketing', '85000', '34', 'TRUE'],
 ['102', 'Jane', 'Doe', '8/20/21', 'Sales', '92000', '45', 'TRUE']]

Each row arrives as a list of text values, which is awkward to analyze. You do not need the csv module in this course. The odd \ufeff before employee_id is an invisible marker that some programs put at the start of a file; read_csv() removes it for you.

Reading the same file with Pandas

df = pd.read_csv("../data/employee_data.csv")
df.head()

3.2 Load the employee dataset

df = pd.read_csv("../data/employee_data.csv")
print("Data loaded successfully!")

Result

Data loaded successfully!

↑ Back to Lab Contents

Part 4: First Look at the Data

4.1 First rows

df.head()

First five rows of df
	employee_id	first_name	last_name	hire_date	department	salary	age	full_time
0	101	John	Smith	4/15/22	Marketing	85000	34.0	True
1	102	Jane	Doe	8/20/21	Sales	92000	45.0	True
2	103	Peter	Jones	1/2/22	HR	71000	29.0	True
3	104	Mary	Johnson	11/10/23	Engineering	115000	38.0	True
4	105	Mike	Brown	5/30/20	sales	89000	NaN	False

4.2 Last rows

df.tail()

Last five rows of df
	employee_id	first_name	last_name	hire_date	department	salary	age	full_time
8	102	Emily	White	1/15/24	Sales	91000	31.0	True
9	110	David	Miller	7/22/19	HR	68000	58.0	True
10	111	Sarah	Moore	3/12/23	Marketing	$91,000	28.0	True
11	error	Kevin	Clark	2/5/24	Sales	88000	35.0	True
12	113	Brian	Hall	6/18/22	Engineering	5900000	999.0	True

4.3 Dataset size

df.shape

Result

(13, 8)

A readable size message

num_rows, num_columns = df.shape
print(f"This dataset has {num_rows} rows and {num_columns} columns.")

Result

This dataset has 13 rows and 8 columns.

shape holds two numbers, so one line can store them in two variables.

4.4 Column names

df.columns

Result

Index(['employee_id', 'first_name', 'last_name', 'hire_date', 'department',
       'salary', 'age', 'full_time'],
      dtype='object')

The names come wrapped in an Index object. You can treat it like a list for now, for example df.columns[0] is 'employee_id'.

4.5 Data types and non-missing counts

df.info()

Result

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 13 entries, 0 to 12
Data columns (total 8 columns):
 #   Column       Non-Null Count  Dtype
---  ------       --------------  -----
 0   employee_id  13 non-null     object
 1   first_name   13 non-null     object
 2   last_name    13 non-null     object
 3   hire_date    12 non-null     object
 4   department   13 non-null     object
 5   salary       12 non-null     object
 6   age          12 non-null     float64
 7   full_time    12 non-null     object
dtypes: float64(1), object(7)
memory usage: 964.0+ bytes

13 entries means 13 rows. Four columns have 12 non-null values out of 13, so each is missing one value. Only age is numeric (float64).

4.6 Data types only

df.dtypes

↑ Back to Lab Contents

Part 5: Data Profiling Checklist

Profiling asks six questions: How many rows and columns? What types? Any missing values? What range of numbers? Are categories consistent? Any duplicate IDs or rows? You are still diagnosing, not fixing.

5.1 Missing values

df.isna().sum()

Result

employee_id    0
first_name     0
last_name      0
hire_date      1
department     0
salary         1
age            1
full_time      1
dtype: int64

hire_date, salary, age and full_time each miss one value, which matches the info() counts.

Missing counts as percentages

missing_summary = pd.DataFrame({
    "missing_count": df.isna().sum(),
    "missing_percent": df.isna().mean() * 100
})

missing_summary

Missing-value summary (13 rows)
column	missing_count	missing_percent
employee_id	0	0.000000
first_name	0	0.000000
last_name	0	0.000000
hire_date	1	7.692308
department	0	0.000000
salary	1	7.692308
age	1	7.692308
full_time	1	7.692308

5.2 Numerical summary with describe()

df.describe()

Result

              age
count   12.000000
mean   118.583333
std    277.411889
min     28.000000
25%     32.500000
50%     36.500000
75%     46.750000
max    999.000000

Only age appears. The mean age is 118.6, which is impossible for employees. The median (50%) is 36.5, which looks normal; a single value of 999 pulls the mean up.

5.3 One statistic at a time

# Example: summary statistics for age
print("Count:", df["age"].count())
print("Mean:", df["age"].mean())
print("Median:", df["age"].median())
print("Minimum:", df["age"].min())
print("Maximum:", df["age"].max())
print("Standard deviation:", df["age"].std())

Result

Count: 12
Mean: 118.58333333333333
Median: 36.5
Minimum: 28.0
Maximum: 999.0
Standard deviation: 277.41188882347143

The same numbers as describe(), one at a time. This form is useful when you need one number for a sentence in a report.

5.4 Categorical summary with value_counts()

df["department"].value_counts()

Result

department
Sales          4
Marketing      3
Engineering    3
HR             2
sales          1
Name: count, dtype: int64

Sales and sales are counted separately. To Python they are different text. This is a consistency problem to note now and fix in Week 3. Mean or median make no sense for a category like department; counts do.

Include missing values in the count

df["department"].value_counts(dropna=False)

Result

department
Sales          4
Marketing      3
Engineering    3
HR             2
sales          1
Name: count, dtype: int64

The output does not change, because department has no missing values. Try dropna=False on full_time to see the difference (Part 7).

5.5 Unique values

df["department"].nunique()

Result

5 labels, although there are really 4 departments. nunique() adds nothing new here, but it helps when a column has hundreds of categories that you cannot count by eye.

5.6 Full duplicate rows

# Count fully duplicated rows
full_duplicate_rows = df.duplicated().sum()
print(f"Number of fully duplicated rows: {full_duplicate_rows}")

Result

Number of fully duplicated rows: 0

Duplicate employee IDs

# Count duplicate employee IDs
employee_id_duplicates = df["employee_id"].duplicated().sum()
print(f"Number of duplicated employee_id values after the first occurrence: {employee_id_duplicates}")

Result

Number of duplicated employee_id values after the first occurrence: 1

Which ID repeats?

df["employee_id"].value_counts().head(10)

Result

employee_id
102    2
101    1
103    1
104    1
105    1
106    1
107    1
108    1
110    1
111    1
Name: count, dtype: int64

ID 102 appears twice: once for Jane Doe (row 1) and once for Emily White (row 8). These are two different people with the same ID, so it is not a copied row. Someone needs to find out which ID is correct.

↑ Back to Lab Contents

Part 6: Writing Data Quality Observations

Profiling is not finished until you write down what you found. A good observation is specific (it names the column and the number) and based on evidence (an output you can point to). Clear notes matter when you hand work to a teammate, a manager or a client.

Complete independently: Your Turn

Write 3–5 observations in the notebook cell Your Turn: Write 3–5 Observations. This is graded, so the companion does not write them for you.

What to check: each observation names a column, gives a number from an output above (a count, a percentage, a minimum or maximum), and says why it might matter later. Describe the problem; do not propose a fix yet. Do not copy the notebook's examples, which describe a different, 500-row dataset. You may also run one more check of your own, for example value_counts() on another column, and write about it.

↑ Back to Lab Contents

Part 7: Optional Challenge

These cells are optional and not graded. Run them if you have time.

Challenge 1: another categorical column

# Optional challenge 1: Choose another categorical column and run value_counts().
# Replace "department" with another column name if appropriate.

df["department"].value_counts(dropna=False)

Try first

Change "department" to another categorical column. Which column has a missing value that dropna=False will show?

Count the values of full_time, including missing values.

Show sample solution for Challenge 1 (optional)

Use this only after you have made your own attempt. Compare the structure and logic, not just the final line of code. This sample solution is written by the companion; it is not part of the notebook.

df["full_time"].value_counts(dropna=False)

What it prints:

full_time
True     11
False     1
NaN       1
Name: count, dtype: int64

Without dropna=False, the NaN row disappears and the counts add up to 12, not 13.

Challenge 2: rows with any missing value

# Optional challenge 2: Show rows with at least one missing value.

df[df.isna().any(axis=1)].head()

Rows with at least one missing value
	employee_id	first_name	hire_date	salary	age	full_time
4	105	Mike	5/30/20	89000	NaN	False
5	106	Susan	NaN	95000	41.0	True
6	107	Chris	2023-FEB-21	NaN	33.0	True
7	108	Laura	9/1/21	120000	52.0	NaN

Challenge 3: sort by salary

# Optional challenge 3: Sort by a numerical column.
# If salary is stored as text in this dataset, this may not sort the way you expect.
# That is okay! We will learn how to fix data types in Week 3.

df.sort_values("salary", ascending=False).head()

Result

   employee_id first_name last_name hire_date department salary   age full_time
5          106      Susan     Davis       NaN      Sales  95000  41.0      True
1          102       Jane       Doe   8/20/21      Sales  92000  45.0      True
8          102      Emily     White   1/15/24      Sales  91000  31.0      True
4          105       Mike     Brown   5/30/20      sales  89000   NaN     False
11       error      Kevin     Clark    2/5/24      Sales  88000  35.0      True

Look at the order of the salaries.

Try first

Optional extra: show both rows that share a duplicated ID, so you can compare them side by side.

Use duplicated(keep=False), which marks every copy, not just the later ones.

Show sample solution for the optional extra

df[df["employee_id"].duplicated(keep=False)]

What it prints:

  employee_id first_name last_name hire_date department salary   age full_time
1         102       Jane       Doe   8/20/21      Sales  92000  45.0      True
8         102      Emily     White   1/15/24      Sales  91000  31.0      True

In a large table, sorting the result by ID with .sort_values("employee_id") puts each pair next to each other.

↑ Back to Lab Contents

Week 2 Assignment Notebook

Part 8 of the guided lab looks ahead to Week 3: correcting data types, handling duplicates and missing values, and documenting each decision. Before that, there is a second notebook to complete.

Complete independently: Week2_assignment.ipynb

This notebook is graded and intentionally not solved in the companion. It uses the same employee_data.csv, so the guided lab is your reference: find the matching function there and apply it yourself.

Task 0: run the cell that imports Pandas and loads the data.
Task 1: show the shape, the column names, the first five rows and the last five rows.
Task 2: inspect types and non-missing counts with info() and dtypes.
Task 3: count missing values with isna().sum() (percentages optional).
Task 4: run describe(), then at least two single statistics for one numeric column.
Task 5: value_counts() for department and one more categorical column.
Task 6: check full duplicate rows, duplicate employee_id values, and the most common IDs.
Task 7: write 3–5 overall observations. Do not clean the data.

What to check: each # Your code here cell is filled and runs without errors, and its output is visible. Each written response uses numbers from your own outputs. A cell can show only its last result automatically, so use print() or separate cells when one task asks for several outputs.

↑ Back to Lab Contents

Common Problems

If a cell fails, read the last line of the error message first. Then find it below.

Still stuck? Post the error message and the cell number in the course Q&A discussion. Do not post your full answers.

↑ Back to Lab Contents

Save and Submit

Complete the guided lab, including your observations, and the assignment notebook.
Run each notebook from top to bottom and confirm there are no errors. Save both notebooks.
Commit and push. In Codespaces: open Source Control, write a short message such as Complete Module 2 data profiling lab, select Commit, then Sync Changes. Or in the terminal: git add ., git commit -m "Complete Module 2 data profiling lab", git push. On the local path, use GitHub Desktop: Commit to main, then Push origin.
Open your repository on GitHub in the browser, refresh it, and confirm your latest commit is there. The instructor grades the online repository, not the copy on your computer.
Submit your repository link in Canvas. If the repository is private, invite zzz1990771 first.

↑ Back to Lab Contents

Lab Wrap-Up

You now have a minimum profiling toolkit:

head(), tail(), shape, columns: see the structure.
info(), dtypes: check types and non-missing counts.
isna().sum(): count missing values.
describe() and single statistics: check numeric ranges and impossible values.
value_counts(), nunique(): check category labels.
duplicated(): check repeated rows and repeated IDs.

These are the essentials, not every function you will ever need. When you need something new, check the Pandas documentation, use help(), or ask. In Week 3 you start fixing what you found today: data types, duplicates and missing values, with each decision documented.

↑ Back to Lab Contents

Notebook and accessibility note

The current Module 2 notebook is authoritative for Python code, sequence, prompts and required work. This companion adds explanation, sample solutions for the Your Turn tasks and accessible presentation without changing those requirements. Expected results come from running the notebook on the nycflights13 data.

The page is designed for keyboard navigation, screen readers, browser text-to-speech and ordinary visual reading. Important instructional information is presented as text rather than depending on a visual demonstration.