File I/O Deep Dive

Layer 7 begins. We go deeper on file handling than Session 29's introduction — proper resource management, different file modes, and safely reading large files.

1. Learning Objective

2. Pre-Coding Quiz

Answer these before reading the concept explanation or writing any code. It is fine to get these wrong — that is the point. You need 4/5 to proceed to the lab.

3. The Concept — Robust File Handling

with — guaranteed cleanup, even on error

The with statement (a "context manager") guarantees a file is properly closed when the block ends, whether it ends normally or because of an exception — directly connecting to Session 11's finally block concept, but automated and less error-prone.

# Risky — if something raises an exception between open() and close(), the file leaks open
f = open("countries.json")
data = f.read()
f.close()  # this line is skipped entirely if the read() line raised an exception!

# Safe — the file is guaranteed to close, no matter what happens inside the block
with open("countries.json") as f:
    data = f.read()
# f is already closed here, even if an exception occurred inside the block

File modes

The second argument to open() controls how the file is accessed: "r" (read, default), "w" (write, overwrites), "a" (append, preserves existing content).

# Read (default) — file must already exist
with open("countries.json", "r") as f:
    content = f.read()

# Write — creates the file if missing, OVERWRITES if it exists
with open("log.txt", "w") as f:
    f.write("First line\n")

# Append — creates the file if missing, adds to the END if it exists
with open("log.txt", "a") as f:
    f.write("Another line, added without erasing what was there\n")

Reading large files incrementally

Iterating a file object directly reads it one line at a time, using a small, constant amount of memory — essential for files too large to comfortably fit in memory all at once.

# Loads the ENTIRE file into memory at once — risky for very large files
with open("huge_log.txt") as f:
    content = f.read()

# Processes one line at a time — memory usage stays small regardless of file size
with open("huge_log.txt") as f:
    for line in f:
        if "ERROR" in line:
            print(line.strip())

Handling encoding issues

Text files can be encoded differently. Specifying the encoding explicitly, or handling a decode error gracefully, prevents a crash on unexpected file content.

try:
    with open("countries.json", encoding="utf-8") as f:
        content = f.read()
except UnicodeDecodeError as e:
    print(f"File is not valid UTF-8: {e}")
    content = None

4. Lab

What you will build

A file called file_io_lab.py, extending the country_explorer package.

Step-by-step instructions

# file_io_lab.py
import json

def load_json_file(path):
    try:
        with open(path, encoding="utf-8") as f:
            return json.load(f)
    except FileNotFoundError:
        print(f"{path} does not exist — using an empty dataset")
        return []
    except json.JSONDecodeError as e:
        print(f"{path} contains invalid JSON: {e}")
        return []
    except UnicodeDecodeError as e:
        print(f"{path} is not valid UTF-8: {e}")
        return []

def log_load_attempt(path, success):
    with open("load_log.txt", "a") as f:
        status = "SUCCESS" if success else "FAILED"
        f.write(f"{status}: {path}\n")

from country_explorer import CountryRepository

def build_repository_from_file(path):
    data = load_json_file(path)
    log_load_attempt(path, success=bool(data))
    return CountryRepository(raw_data=data)

repo = build_repository_from_file("countries.json")
print(len(repo.get_all()))

missing_repo = build_repository_from_file("does_not_exist.json")
print(len(missing_repo.get_all()))  # 0 — graceful, no crash

with open("load_log.txt") as f:
    for line in f:
        if line.startswith("FAILED"):
            print(line.strip())

5. Expected Files Changed

6. Commit Checkpoint

Once the lab is complete and you can explain every line, make this exact commit:

git add file_io_lab.py load_log.txt docs/sessions/session-41/index.html
git commit -m "session-41: robust file I/O with proper resource management and encoding handling"

7. Code Review Checklist

Go through your code line by line and check each item:

8. Post-Coding Quiz

Same 5 questions. Take it again now that you have written and run the code. You need 4/5 to mark this session complete.

9. Reflection Questions

Think through these after the post-quiz. No right answer — they are for discussion.

10. What Breaks If This Knowledge Is Missing?

11. What We Learned