Exceptions and Error Handling

Intermediate⏱75 minutes📖18 min readLesson 3 of 10

Exceptions and Error Handling

Handle failure without hiding it: read tracebacks, catch only what you can repair, chain causes with raise from, design custom exception hierarchies, and retry only the errors that can succeed later

The nightly order import had run without a failure for eleven months. Then a supplier changed one column header, and the importer started dropping rows. The per row handler caught Exception, wrote a line to a log file nobody read, and continued with the next row. By the time finance noticed, twelve thousand paid orders were missing from the warehouse, and the log contained twelve thousand identical lines that said only error.

The code did exactly what it was told. The problem was the instruction. Catching an exception is not the same as handling it, and a broad catch is a promise to repair every possible failure, including the ones the author never imagined.

This lesson builds the judgement that the importer lacked. It explains what an exception object contains, how the four clauses of a try statement run in order, how to raise and chain errors without losing the original cause, and how to decide which failures deserve a retry and which deserve a crash. It builds on the class model from lessons 1 and 2, because a custom exception is a class like any other.

Key terms

Exception

An object that represents a failure and interrupts normal control flow until some code handles it.

Traceback

The ordered list of frames from the call site down to the line that raised the exception.

Raise

The statement that creates an exception and starts the search for a matching handler.

Handler

An except clause whose exception class matches the raised exception or one of its bases.

Else clause

The branch of a try statement that runs only when the try body finished without raising.

Finally clause

The branch that always runs, whether the try body succeeded, raised, or returned.

Chaining

Linking a new exception to the original one through __cause__ or __context__ so both appear in the traceback.

Exception group

A single exception that wraps several independent exceptions, raised with except* handlers.

Bare except

An except clause with no class, which catches BaseException and therefore also catches shutdown signals.

EAFP

Easier to ask forgiveness than permission, the style that tries an operation and handles the resulting exception.

Retryable error

A failure caused by a temporary condition that may succeed if the same operation runs again later.

1. The Exception Hierarchy Is a Contract

Every exception inherits from BaseException, but only a branch of that tree represents ordinary program failure. The other branch holds events that belong to the interpreter or the operating system: SystemExit, KeyboardInterrupt and GeneratorExit.¹

python

print([cls.__name__ for cls in FileNotFoundError.__mro__])
print(issubclass(FileNotFoundError, OSError))
print(issubclass(SystemExit, Exception))

The first line prints ['FileNotFoundError', 'OSError', 'Exception', 'BaseException', 'object']. The second prints True, because a missing file is a kind of operating system error. The third prints False, because a request to exit the process is not a program failure that a library should repair.

The diagram below shows the split that the issubclass calls just tested.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A top down flowchart of the Python exception hierarchy, with BaseException at the top splitting into SystemExit, KeyboardInterrupt, GeneratorExit and Exception, and Exception splitting into ValueError, TypeError, OSError and RuntimeError, with FileNotFoundError below OSError.

The green branch is the one application code should catch. The red branch is reserved for the runtime, and a program that swallows KeyboardInterrupt cannot be stopped with Ctrl+C without killing the process from outside.

Layer	Examples	Who handles it
Interpreter events	`SystemExit`, `KeyboardInterrupt`, `GeneratorExit`	Almost nobody; let them propagate
Programming errors	`TypeError`, `AttributeError`, `NameError`	Fix the code; catch only at a boundary
Expected failures	`ValueError`, `KeyError`, `OSError`	The caller that has a recovery path
Domain failures	Your own `Error` classes	The layer that understands the business rule

The table is a guide, not a law. A ValueError from int("abc") is expected and a ValueError from a broken invariant is a bug, and both use the same class. The distinction comes from the call site, not from the type name.

📌Important

A library should raise, and an application boundary should catch. The boundary is the place that knows what to do next: show a message, return a default, retry, or fail the request. Code in the middle should add context and let the exception continue.

The except Exception clause is the widest catch that still excludes the interpreter events. It is appropriate in a worker loop that must keep running, in a web framework that converts failures into responses, and in a top level main that logs and exits with a status code. It is a poor choice in the middle of a parser, because it hides the bug that the next test would have found.

2. The Four Clauses and Their Order

A try statement has four parts, and each answers a different question. The try block holds the operation that may fail. The except blocks choose which failures to repair. The else block runs only when no exception occurred. The finally block runs no matter what happened.²

python

def divide(a: int, b: int) -> float | None:
    try:
        result = a / b
    except ZeroDivisionError:
        print("except")
        return None
    else:
        print("else")
        return result
    finally:
        print("finally")

Calling divide(10, 2) prints else and then finally, and returns 5.0. Calling divide(10, 0) prints except and then finally, and returns None. The finally block runs in both cases, including the case where the function is about to return a value.

The order is worth memorising because a surprising number of bugs come from putting cleanup in else or logging in finally and then reading the output in the wrong order.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A flowchart of a try statement. The try body runs; a matching exception enters an except handler, success enters the else clause, and unmatched exceptions propagate. Both handler and else paths meet at the finally clause before control continues after the try.

The else clause exists for a specific reason. Code placed there is not protected by the except blocks above it, so an exception raised in else propagates instead of being caught by a handler that was meant for the try body. That sounds like a disadvantage until you consider the alternative.

python

import json


def parse_settings(text: str) -> dict:
    try:
        data = json.loads(text)
    except json.JSONDecodeError:
        print("invalid JSON, using defaults")
        return {}
    else:
        print("parsed", len(data), "keys")
        return data

If the print call were inside the try block, a bug in the formatting code would be caught as if it were invalid JSON. The else block keeps the protected region as small as possible, which is the whole point.

The video below walks through the same four clauses with a live example, so you can watch which branch runs for each input.

Corey SchaferWatch on YouTube

Description

A video lesson that runs a try block with except, else and finally clauses, showing which branch executes when the body succeeds, fails with a handled error, or fails with an unhandled one.

The finally clause has one dangerous interaction with control flow. If a return, break or continue inside finally exits the block, it discards any in flight exception and replaces a pending return value. The behaviour is documented, widely considered a bug magnet, and since Python 3.14 the compiler emits a SyntaxWarning for it.³

python

def dangerous() -> str:
    try:
        raise ValueError("this error disappears")
    finally:
        return "finally wins"

Calling dangerous() returns "finally wins" and the ValueError is never seen. The warning is 'return' in a 'finally' block, and the fix is to move the return after the finally block or to remove it entirely.

⚠️Warning

Never put return, break or continue inside finally. The clause exists for cleanup, and control flow statements there can swallow an exception that the caller was waiting for.

The fill in the blank exercise checks the order of the four clauses.

Fill in the Blanks

How to answer

Fill-in-the-blank exercise with 4 blanks. Type the missing words, then check your answer.

Complete the description of a try statement.

The blank block holds the operation that may fail. A matching blank clause repairs the failure. The blank clause runs only when the try body finished without an exception. The blank clause always runs, whether the body succeeded, raised, or returned.

Word Bank, drag words into the blanks above:

try

except

else

finally

raise

assert

with

match

3. What the Exception Object Carries

An exception is an object, and the message is only one of its attributes. The args tuple holds the constructor arguments, str() renders the message, and __traceback__ holds the frames from the raise site down to the handler. Built in exceptions add their own attributes on top of that.

python

try:
    open("/srv/reports/missing.csv")
except FileNotFoundError as exc:
    print(exc.errno)
    print(exc.strerror)
    print(exc.filename)
    print(exc.args)

The output is 2, No such file or directory, /srv/reports/missing.csv, and (2, 'No such file or directory'). The filename attribute is not part of the message, which is why string matching on str(exc) is fragile. Inspect the attributes instead.

OSError carries errno and strerror, and its subclasses narrow the meaning: FileNotFoundError, PermissionError, IsADirectoryError, NotADirectoryError and TimeoutError all inherit from it.¹ The built in TimeoutError is an operating system timeout, not the subprocess timeout class, and section 7 shows why that distinction matters.

The traceback module turns the frames into text or into structured data. These functions are useful in a handler, a logger and a test, but they are rarely useful in library code that should simply raise.⁴

python

import traceback


def risky() -> None:
    raise ValueError("row 41 has no total")


try:
    risky()
except ValueError as exc:
    print(exc.args)
    print(traceback.format_exc().splitlines()[-1])

The output is ('row 41 has no total',) and ValueError: row 41 has no total. The full format_exc() string includes every frame, which is what a log entry needs and what a user facing message usually does not. In application code, logging.exception records the message and the full traceback at error level from inside a handler, which is the same information without a manual call to the traceback module.⁵

Since Python 3.11, an exception can also carry notes. add_note appends a line that appears in the formatted traceback but not in str(exc), which makes it a good place for context that helps a developer without changing the message that the program matches on.⁶

python

try:
    try:
        raise ValueError("bad row")
    except ValueError as exc:
        exc.add_note("file: orders.csv")
        exc.add_note("line: 41")
        raise
except ValueError as exc:
    print(exc)
    print(exc.__notes__)

The output is bad row and ['file: orders.csv', 'line: 41']. The formatted traceback prints the notes after the message, each on its own line. The notes survive re-raising because they live on the exception object.

💡Tip

Add notes at the layer that knows the context, and keep the message stable. A support engineer can read the file and line from the note, while the caller keeps matching on the original message and type.

Two more attributes describe how the exception relates to earlier ones. __cause__ is the explicit cause set by raise ... from, and __context__ is the implicit exception that was being handled when the new one was raised. The next section uses both.

4. Raising, Re-raising and Chaining

The raise statement has three forms. raise SomeError("message") creates and raises a new exception. A bare raise re-raises the exception currently being handled. raise exc raises an existing exception object again, which is a different operation from a bare raise even though the two look similar.

python

import traceback


def inner() -> None:
    raise ValueError("bad")


def bare() -> None:
    try:
        inner()
    except ValueError:
        raise


def named() -> None:
    try:
        inner()
    except ValueError as exc:
        raise exc


for func in (bare, named):
    try:
        func()
    except ValueError as exc:
        frames = [frame.name for frame in traceback.extract_tb(exc.__traceback__)]
        print(func.__name__, frames)

The output is bare ['<module>', 'bare', 'inner'] and named ['<module>', 'named', 'named', 'inner']. The named form adds the current function to the traceback again, because raise exc is a new raise at that line. With a long chain of handlers, the traceback fills with frames that add no information.

⚠️Warning

Use bare raise to re-raise. raise exc rebuilds the traceback from the current frame, which makes the original raise site harder to find and makes logs longer without adding context.

Chaining is the explicit way to add context. The pattern below converts a parsing failure into a domain failure while preserving the original error as the cause.

python

import json


class ConfigError(Exception):
    """A configuration file could not be loaded."""


def load_config(text: str) -> dict:
    try:
        return json.loads(text)
    except json.JSONDecodeError as exc:
        raise ConfigError("config is not valid JSON") from exc

When load_config("{bad") fails, the traceback shows ConfigError: config is not valid JSON, then a line that says the error was the direct cause of the following exception, then the original JSONDecodeError. The caller can catch ConfigError without knowing that JSON was involved, and a developer can still see the exact parse failure.²

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A sequence diagram of a config loader. The caller calls load_config, which calls json.loads. json.loads raises JSONDecodeError, the loader catches it and raises ConfigError from it, and the caller receives ConfigError whose cause attribute points at JSONDecodeError.

The from exc form sets __cause__ and also sets __suppress_context__ to True. That second flag tells the traceback printer not to show the implicit context separately, because the explicit cause is the useful story. Without from, the same raise would still set __context__, and the traceback would print a "During handling of the above exception" section.

python

try:
    load_config("{bad")
except ConfigError as exc:
    print(type(exc.__cause__).__name__)
    print(exc.__suppress_context__)

The output is JSONDecodeError and True. The cause is available for code that wants to inspect it, and the printed traceback stays readable.

The other direction is raise ... from None, which suppresses the context entirely. Use it when the original exception is noise, for example when a lookup failure is an expected part of a fallback path. Do not use it to hide a failure that an operator will need during an incident.

ℹ️Note

__context__ is set automatically whenever an exception is raised inside an except block. __cause__ is set only by raise ... from. A handler that wants the original error should chain explicitly, not rely on context.

The matching exercise below connects each form to its effect.

Matching Question

How to answer

Matching exercise with 5 pairs. Match each item on the left with its answer on the right.

Match each raise form to the traceback it produces.

raise

Drop here

raise exc

Drop here

raise NewError from exc

Drop here

raise NewError from None

Drop here

raise NewError

Drop here

Drag answers from here:

Re-raises the active exception without adding a frame

Raises the same object again and adds the current frame

Shows NewError as the direct cause of exc

Hides the earlier exception from the traceback

Creates a new error while the old one becomes implicit context

A bare raise outside any handler has no active exception to re-raise, so it fails with RuntimeError: No active exception to reraise. That error is a useful signal that the statement ended up on a path the author did not expect.

5. Designing a Custom Exception Hierarchy

A custom exception earns its place when callers need to distinguish one failure from another, or when the failure carries data that the caller needs. The bank example below has one base class and three specific failures.

python

class BankError(Exception):
    """Base class for account operations."""


class InsufficientFundsError(BankError):
    def __init__(self, balance: float, amount: float) -> None:
        self.balance = balance
        self.amount = amount
        self.shortfall = amount - balance
        super().__init__(f"need {self.shortfall:.2f} more")


class AccountNotFoundError(BankError):
    def __init__(self, account_id: str) -> None:
        self.account_id = account_id
        super().__init__(f"account {account_id} not found")


class FrozenAccountError(BankError):
    pass

Each subclass stores the values a handler might need and passes a readable message to super().__init__. InsufficientFundsError(50.0, 80.0) prints need 30.00 more, and exc.shortfall is 30.0. The handler can decide what to do from the attribute instead of parsing the message.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A class diagram with Exception at the top, BankError below it, and three subclasses of BankError: InsufficientFundsError with balance, amount and shortfall, AccountNotFoundError with account_id, and FrozenAccountError with no extra fields.

The hierarchy stays useful because it is shallow. A caller can catch InsufficientFundsError to offer a top up, catch BankError to log any account problem, or catch Exception at the process boundary. Each level has a meaning that a reader can explain without opening the source.

Order the handlers from specific to general. Python checks them in source order and runs the first match, so a broad handler placed first makes the later ones unreachable.

python

def describe(error: BankError) -> str:
    try:
        raise error
    except InsufficientFundsError as exc:
        return f"short by {exc.shortfall:.2f}"
    except AccountNotFoundError as exc:
        return f"unknown account {exc.account_id}"
    except BankError:
        return "account operation failed"

The drag and drop exercise below orders the same idea as a checklist.

Ordering Question

How to answer

Ordering exercise with 4 items. Move them into the correct order, then check your answer.

Order these except clauses from the one that should be checked first to the one that should be checked last.

except InsufficientFundsError

except AccountNotFoundError

except BankError

except Exception

A custom exception should inherit from Exception, not BaseException, and its name should end in Error. It should not define __init__ unless it has data to store, and it should not override __str__ when a well chosen message is enough. A hierarchy deeper than three levels is usually a sign that the exception is carrying program logic that belongs in a result object.

🚨Danger

Do not use exceptions for ordinary branching. Raising ItemNotFound to end a loop, or ValidationFailed to return a boolean, makes the common path slower and harder to read. Use a return value for an expected outcome and an exception for a broken promise.

Grouping independent failures

Sometimes several operations fail at once and the caller needs all of them, not just the first. Since Python 3.11, ExceptionGroup wraps multiple exceptions and except* handles them by type.⁷

python

try:
    raise ExceptionGroup(
        "batch failed",
        [ValueError("row 1"), TypeError("row 3"), ValueError("row 9")],
    )
except* ValueError as group:
    print("value", len(group.exceptions))
except* TypeError as group:
    print("type", len(group.exceptions))

The output is value 2 and type 1. Each except* clause receives a subgroup containing only the matching exceptions, and unmatched exceptions are re-raised together as a new group. This is the right shape for validating a batch of rows, running a set of independent checks, or reporting every failure from a test suite.

BaseExceptionGroup can wrap BaseException instances, while ExceptionGroup wraps only Exception. A try statement cannot mix except and except* clauses, because the two models of control flow do not compose. A group also offers split, subgroup and derive for code that needs to inspect the parts before deciding.

📌Important

except Exception catches an ExceptionGroup as one object. It does not run the individual exceptions through the handler. When a group is possible, use except* or inspect group.exceptions explicitly.

6. Retrying Only What Can Succeed Later

A retry is a bet that the failure was caused by a temporary condition. That bet is correct for a dropped connection, a rate limit or a lock held for a few milliseconds. It is wrong for a ValueError, a TypeError or a KeyError, because the same input will fail the same way forever.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A flowchart of a retry loop. The operation runs; success returns the result, a transient error checks whether attempts remain and otherwise waits with exponential backoff and retries, and a permanent error or exhausted attempts re-raises the last failure.

The loop below takes the wait function as a parameter, which keeps the policy testable and avoids a real sleep in a unit test.

python

class TransientError(Exception):
    """A failure that may succeed if the operation runs again."""


def fetch_with_retry(operation, wait, attempts: int = 4, base: float = 0.5):
    for attempt in range(1, attempts + 1):
        try:
            return operation()
        except TransientError:
            if attempt == attempts:
                raise
            delay = base * 2 ** (attempt - 1)
            print(f"attempt {attempt} failed, waiting {delay:.1f}s")
            wait(delay)

A demo with a function that fails twice prints attempt 1 failed, waiting 0.5s, wait 0.5, attempt 2 failed, waiting 1.0s, wait 1.0, then ok. The fourth call is never made because the third attempt succeeded.

The chart below shows the delay policy that the loop implements. The numbers are the policy, not a measurement, and production code should add random jitter so that many clients do not retry in lockstep.

Exponential backoff with a 0.5 second base

Description

Bar chart of retry delays for five attempts with a 0.5 second base. Attempt one waits 0.5 seconds, attempt two 1 second, attempt three 2 seconds, attempt four 4 seconds and attempt five 8 seconds.

Three rules keep a retry loop from becoming an outage amplifier. Cap the number of attempts and the total elapsed time. Retry only operations that are safe to repeat, because a retried payment can charge a customer twice. Re-raise the last error with its cause intact when the budget is exhausted, so the caller sees the real failure instead of a generic timeout.

⚠️Warning

A retry loop around a permanent error turns one bug into thousands of log lines and can exhaust a rate limit or a queue. Classify the error before you decide to retry, and never retry ValueError, TypeError or KeyError from the same input.

7. Real-World Case Studies

Two standard library exceptions show how much context a well designed error can carry, and how a handler turns that context into a message a human can act on.

A JSON config loader reports the exact position

json.JSONDecodeError inherits from ValueError and exposes msg, doc, pos, lineno and colno.⁸ A loader can use those attributes to point at the broken character instead of printing a generic parse failure.

python

import json


def load_config(text: str) -> dict:
    try:
        return json.loads(text)
    except json.JSONDecodeError as exc:
        print(f"{exc.msg} at line {exc.lineno}, column {exc.colno}")
        print(exc.doc.splitlines()[exc.lineno - 1])
        print(" " * (exc.colno - 1) + "^")
        raise

For the input {"a": 1,}, the output is Illegal trailing comma before end of object at line 1, column 8, then the original line, then a caret under the offending comma. The lineno and colno values are one based, which is why the caret uses colno - 1 spaces.

The handler re-raises, which is the right choice for a reusable loader. It adds the diagnosis to the console and lets the caller decide whether to fall back to defaults, fail startup or ask the user to fix the file. A loader that returned {} on every parse error would hide a broken deployment configuration.

A subprocess runner separates exit codes from timeouts

subprocess.run(check=True) raises CalledProcessError when the child exits with a non zero status, and TimeoutExpired when the deadline passes. The two carry different attributes: CalledProcessError has returncode, cmd, stdout and stderr, while TimeoutExpired has cmd, timeout, output and stderr.⁹

python

import subprocess
import sys


def run_step(command: list[str], timeout: float = 30.0) -> str:
    try:
        completed = subprocess.run(
            command,
            check=True,
            capture_output=True,
            text=True,
            timeout=timeout,
        )
    except subprocess.TimeoutExpired as exc:
        raise RuntimeError(f"step timed out after {exc.timeout}s") from exc
    except subprocess.CalledProcessError as exc:
        detail = (exc.stderr or "").strip()
        raise RuntimeError(f"step exited {exc.returncode}: {detail}") from exc
    return completed.stdout

A command that writes missing dependency and exits with status 3 produces step exited 3: missing dependency, with CalledProcessError as the cause. A command that sleeps past the deadline produces step timed out after 0.2s, with TimeoutExpired as the cause.

The distinction matters because the recovery differs. A non zero exit code usually means the command ran and rejected its input, so a retry will fail the same way unless something changed. A timeout may be transient, so a retry with a longer deadline is reasonable. subprocess.TimeoutExpired is a subclass of SubprocessError, not of the built in TimeoutError, so catching TimeoutError around this code would not catch it.

💡Tip

Convert library exceptions into domain exceptions at the boundary of your module, and chain the original with from. Callers then depend on your vocabulary, while operators still see the library detail in the traceback.

8. Anti-Patterns and a Review Checklist

Most exception bugs come from a small set of patterns. The table below names each one, the damage it causes, and the smallest fix.

Anti-pattern	Damage	Fix
`except:` with no class	Catches `KeyboardInterrupt` and `SystemExit`	Catch `Exception` or a specific class
`except Exception: pass`	Silently drops every failure	Log, recover, or re-raise
Catching around a whole function	One handler covers unrelated operations	Wrap only the operation that can fail
`raise exc` in a handler	Duplicates frames in the traceback	Use bare `raise`
Catching and returning a default	Turns a failure into corrupt data	Let the caller choose the default
Retrying every error	Amplifies a permanent failure	Classify transient errors first
Using exceptions for branching	Slower and harder to read than a return value	Return a result or a sentinel
Logging and re-raising without a cause	Duplicate log entries with no link	Log once at the boundary, chain with `from`

The checklist below is short enough to use in review. Each question maps to a failure that the importer in the opening story would have caught.

Does the handler name the specific exceptions it can actually repair?
Is the try block as small as possible?
Does the success path live in else when it must not be caught?
Does cleanup live in finally without a return or break?
Does every re-raise use a bare raise or raise ... from?
Does the log include the traceback exactly once?
Does the retry loop cap attempts and classify the error?

ℹ️Note

A handler that logs and continues is making a policy decision on behalf of every caller. If the program cannot continue correctly, the honest response is to raise, not to print and carry on.

9. 2026 Updates (as of November 2026)

Python 3.15.2 is the current stable documentation set.¹⁰
ExceptionGroup, BaseExceptionGroup and the except* syntax have been available since Python 3.11, and they are the documented way to report several independent failures at once.⁷
BaseException.add_note has been available since Python 3.11, and notes are printed by the default traceback formatter.⁶
Since Python 3.14, the compiler emits a SyntaxWarning when return, break or continue would exit a finally block, which turns a silent exception swallowing bug into a visible warning.³
Python 3.15 improved AttributeError messages with suggestions for names on inner objects and for common methods from other languages, which shortens the distance between a traceback and a fix.¹¹
Python 3.15 also reports the process id and exit code when a ProcessPoolExecutor child terminates abruptly, so a worker crash is visible in the traceback instead of appearing as a generic broken pool.¹¹

📚 Did you know?

sys.excepthook is called for an uncaught exception at the top level, and replacing it changes how the interpreter prints the final traceback, not how the exception propagates.
A StopIteration raised inside a generator is converted into a RuntimeError, because the generator protocol uses StopIteration to signal the end of iteration and cannot tell the two apart otherwise.
KeyboardInterrupt inherits from BaseException, which is why a bare except: makes a long running script ignore Ctrl+C while except Exception: leaves the interrupt alone.

📌Important

Comparative Verdict. Validate input at the edge when the rule is cheap and the failure is expected, and raise a specific exception when the rule is expensive or the caller must decide. Catch only where the program can repair the failure or translate it for a boundary. Let programming errors crash in development, and let them become a logged 500 response in production. Retry only transient errors, with a capped budget and a preserved cause.