Performance Optimization, Cython, and Numba

Advanced⏱90 minutes📖16 min readLesson 5 of 10

Performance Optimization, Cython, and Numba

Profile before you optimize, fix the algorithm and the data layout first, then move the hot loop into Cython or Numba when the measurements justify it

A reporting job takes nine minutes and the team decides it needs a faster machine. The job reads a million rows from a database, builds a list of ids, then checks every row against that list with in. The lookup is quadratic, and no amount of CPU changes the shape of the curve. Replacing the list with a set takes the same job under a minute on the same hardware.

The second version is fast enough until the next feature arrives, and then a single scoring loop becomes the bottleneck. That loop needs native code: Cython or Numba compiles it, and a measured comparison decides which one earns a place in the build. The order matters. Algorithm first, data structure second, array layout third, native code last. Native code applied to a quadratic algorithm only makes the wrong answer arrive sooner.

This lesson follows that order. It starts with measurement, moves through algorithmic and vectorized fixes, builds a typed Cython extension and a Numba kernel, and compares them on one honest benchmark. It also covers the failure modes that native code introduces: overflow, disabled bounds checks, missing wheels, and a first call that pays for compilation.

ℹ️Note

The benchmark numbers in this lesson were measured on CPython 3.14.7 with NumPy 2.5.4, Cython 3.3.1 and Numba 0.68.1 on an eight-core Linux machine. Cython and Numba listings are shown as plain text because the browser runner has no C compiler. The pure Python and timeit examples run everywhere.

Key terms

Profiler

A tool that records where a program spends time, either by sampling the stack or by tracing every call.

Deterministic profiling

Tracing every function call and return, which gives exact counts and adds overhead to every call.

Statistical profiling

Sampling the call stack at intervals, which has low overhead and estimates where time goes.

Vectorization

Expressing an operation over a whole array so the loop runs in compiled code instead of the interpreter.

Broadcasting

NumPy's rule for combining arrays of different shapes without copying data explicitly.

Memoryview

A typed view over a buffer that Cython can index without creating Python objects.

GIL release

Temporarily allowing other Python threads to run while a C loop executes.

JIT compilation

Compiling a function to machine code at run time, usually on the first call.

nopython mode

Numba's fast path, which compiles a function without falling back to the Python interpreter.

Overflow

Wrapping or truncating a fixed-width integer result, which Python integers never do.

Bounds check

A safety check on every array index that costs time and prevents out-of-bounds access.

ABI

The binary interface a compiled extension targets, which must match the interpreter and platform.

1. The Order of Optimization

Optimization has a cheapest-first order, and skipping a step is how teams end up with a compiled extension around a bug. The diagram is the whole lesson in one picture.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A flowchart of optimization order starting from a measurement, then asking whether the algorithm is quadratic, whether a better data structure exists, whether the data can be vectorized, and only then whether native code is justified.

The loop back to the measurement is not decoration. Every change invalidates the previous profile. A fix in the database layer can move the bottleneck into serialization, and a vectorized loop can expose the cost of allocating the output array.

2. Measure Before You Change Anything

timeit answers "how long does this snippet take". It runs the snippet many times, discards the loop overhead, and reports a distribution. Use it for a microbenchmark with a fixed input, and use min or the median rather than the mean, because a garbage collection pause inflates the mean.

python

import timeit

setup = "values = list(range(1000))"
statement = "sum(values)"

result = timeit.repeat(statement, setup=setup, number=10_000, repeat=5)
print([round(value, 4) for value in result])
print("best", round(min(result), 4))

cProfile answers "which function consumed the time". It traces every call, so it slows the program down and distorts code that makes many small calls.¹ That is acceptable for attribution and unacceptable for measuring the absolute cost of a fast path.

python

import cProfile
import pstats

def slow_report(rows):
    return [sum(1 for value in rows if value == target) for target in range(50)]

def main():
    rows = list(range(20_000))
    slow_report(rows)

profile = cProfile.Profile()
profile.enable()
main()
profile.disable()
stats = pstats.Stats(profile)
stats.sort_stats("cumulative").print_stats(8)

Sampling profilers answer "where is the process now" without instrumenting the code. py-spy attaches to a running process and reads its stacks, including native frames, which is the only practical way to inspect a production service.² scalene adds line-level CPU and memory attribution and separates Python time from native time.³

Python 3.15 adds a standard profiling package with both approaches under one namespace, and the sampling half is named Tachyon.⁴ It can run a script or attach to a live process.

bash

python -m profiling.sampling run script.py
python -m profiling.sampling attach 12345
python -m profiling.sampling dump 12345
python -m profiling.sampling run --flamegraph -o profile.html script.py

The tracing half, profiling.tracing, replaces the cProfile workflow with exact call counts, and cProfile remains available for compatibility.⁵ For tools that need very low overhead without sampling, sys.monitoring exposes interpreter events and lets a tool enable only the events it needs.⁶

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A sequence diagram of an optimization loop, where the engineer measures a baseline, forms one hypothesis, changes one thing, re-measures, and either keeps the change or reverts it.

The comparison that matters is always before and after on the same input, the same interpreter and the same hardware. A speedup quoted from a blog post is a hypothesis, not a result.

3. Algorithmic and Data Structure Wins

The cheapest speedup is usually a better container. Membership in a list scans every element, while membership in a set or dictionary is expected constant time.⁷ The difference grows with the size of the collection, and it is invisible in a profile that only shows one call.

python

import time

VALUES = list(range(10_000))
PROBES = list(range(10_000))

start = time.perf_counter()
hits = sum(1 for probe in PROBES if probe in VALUES)
print("list", round(time.perf_counter() - start, 4), hits)

lookup = set(VALUES)
start = time.perf_counter()
hits = sum(1 for probe in PROBES if probe in lookup)
print("set", round(time.perf_counter() - start, 6), hits)

On the measured machine the list version took 0.148 seconds and the set version 0.0006 seconds, a factor of roughly 240 for ten thousand probes. The same rule applies to counting: a loop that calls list.count for every distinct value is quadratic, while one pass into a dictionary is linear.

python

import time

VALUES = list(range(10_000))

start = time.perf_counter()
counts = [VALUES.count(value) for value in VALUES]
print("list.count", round(time.perf_counter() - start, 4))

start = time.perf_counter()
counts = {}
for value in VALUES:
    counts[value] = counts.get(value, 0) + 1
print("dict", round(time.perf_counter() - start, 4))

The measured difference was 0.533 seconds against 0.0005 seconds. The dictionary version also uses less memory than a list of counts for sparse values, because it stores only the keys that appear.

Other algorithmic wins that show up constantly:

Pattern	Slow version	Fast version
Membership	`x in list`	`x in set`
Counting	`list.count` per value	one pass into a dict
Sorting by a key	sort the full objects	sort with `key=`
Range lookup	scan for the interval	`bisect.bisect_right`
Repeated parsing	parse on every request	parse once, cache the result
String building	`+=` in a loop	`"".join(parts)`

Matching Question

How to answer

Matching exercise with 5 pairs. Match each item on the left with its answer on the right.

Match each performance tool to the question it answers.

timeit

Drop here

cProfile

Drop here

py-spy

Drop here

scalene

Drop here

tracemalloc

Drop here

Drag answers from here:

How long does this snippet take

Which function consumed the time

Where is this running process now

Which line uses CPU and memory

Where were these objects allocated

4. Vectorization with NumPy

A NumPy array stores homogeneous values in one contiguous buffer, and an operation over the array runs in compiled code. Broadcasting lets arrays of different shapes combine without an explicit Python loop.⁸ The win is real and the memory cost is easy to miss.

python

import numpy as np

values = np.arange(1_000_000, dtype=np.float64)

# One temporary array for the square, then a reduction.
total = (values * values).sum()

# Reuse a buffer to avoid the temporary.
buffer = np.empty_like(values)
np.square(values, out=buffer)
total = buffer.sum()

print(total)

The first version allocates one array of eight megabytes for the square. On a large frame that allocation can dominate the run and push the process into swap. In-place operations and out= parameters keep the memory flat.

Three rules cover most vectorization work:

Keep the dtype numeric. An object array falls back to Python calls and can be slower than a list.
Avoid Python loops over elements. If the loop has data-dependent branches, vectorization needs masks or np.select.
Count the temporaries. Each arithmetic expression can allocate a full array.

python

import numpy as np

prices = np.array([10.0, 20.0, 30.0])
quantities = np.array([1, 2, 3])
discount = 0.1

gross = prices * quantities
net = gross * (1 - discount)
print(net)

Broadcasting works when the trailing dimensions match or one of them is 1, so a column vector and a row vector combine into a matrix without an explicit repeat. That is a convenience and a trap: a shape mistake can allocate a much larger array than intended.

Ordering Question

How to answer

Ordering exercise with 5 items. Move them into the correct order, then check your answer.

Order the optimization steps from cheapest to most expensive.

Fix a quadratic algorithm

Replace a linear scan with a set or dictionary

Express the loop as a NumPy operation

Compile the remaining hot loop with Cython or Numba

Rewrite the extension in C or Rust

5. Cython: Typed Loops and Memoryviews

Cython compiles a Python-like file to C. The first version is often the same code with type declarations, and the compiler removes the interpreter overhead from every loop iteration.⁹ The listing below is the function used in the benchmark later in this lesson.

plaintext

# fastsum.pyx
def total_typed(unsigned long long n):
    cdef unsigned long long i
    cdef unsigned long long acc = 0
    for i in range(n):
        acc = (acc + i * i) % 1000000007
    return acc

cdef declares a C variable. The loop counter and the accumulator live in registers instead of Python objects, and the modulo keeps the result inside a 64-bit integer. Build it with cythonize, which compiles the generated C file and links it into an extension module.

bash

python -m cython fastsum.pyx -3
python setup.py build_ext --inplace

plaintext

# setup.py
from setuptools import setup
from Cython.Build import cythonize

setup(
    name="fastsum",
    ext_modules=cythonize("fastsum.pyx", language_level=3),
)

The build produces a file named fastsum.cpython-314-x86_64-linux-gnu.so on CPython 3.14 for x86-64 Linux. The name encodes the ABI, which is why a wheel built for one interpreter version does not load in another.

A memoryview is the safe way to pass an array into Cython. The declaration double[::1] promises a one-dimensional, C-contiguous buffer, and indexing it inside the function is a pointer operation with no Python object created.¹⁰

plaintext

# scale.pyx
def scale(double[::1] values, double factor):
    cdef Py_ssize_t index
    for index in range(values.shape[0]):
        values[index] *= factor

A non-contiguous array raises ValueError at the call, which is better than silently reading the wrong memory. When the input may be strided, declare double[:] without the ::1 and accept slower indexing.

The NumPy integration guide covers the older cimport numpy style and the typed memoryview style, and recommends memoryviews for new code.¹¹ Cython also accepts a pure Python file annotated with cython.int and related types, which lets the same module run unmodified under CPython.¹²

The video below compiles a small function with Cython and shows the generated C and the speed difference, which is a useful first look before reading the build files in this section.

NeuralNineWatch on YouTube

Description

A video walkthrough that turns a pure Python function into a Cython module, compiles it, and compares the run time against the original Python version.

The generated annotation report is the fastest way to find the lines that still create Python objects.

bash

cython -a fastsum.pyx

Open fastsum.html in a browser. Lines shaded yellow still call into the Python API. A fully typed loop is white, and the number of yellow lines is a good proxy for how much interpreter overhead remains.

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A class diagram of the Cython build inputs and outputs, with the pyx source, the generated C file, the platform specific extension module, and the Python module that imports it.

6. Cython: Directives, Overflow and Parallelism

Cython is safe by default. Every index access checks bounds and every slice handles negative indices, and both cost time. The directives turn the checks off for a module or a function, and the tradeoff is that an out-of-bounds access becomes undefined behaviour instead of an exception.

plaintext

# cython: boundscheck=False, wraparound=False, cdivision=True
# cython: language_level=3

def dot(double[::1] left, double[::1] right):
    cdef Py_ssize_t index
    cdef double total = 0.0
    for index in range(left.shape[0]):
        total += left[index] * right[index]
    return total

Disable the checks only after the loop is correct and tested. A negative index with wraparound=False reads before the buffer, and the process may return a plausible number instead of crashing.

Overflow is the other silent bug. Python integers grow to any size, while a C integer wraps at its width. The benchmark function uses unsigned long long and a modulo to stay inside the range. Without the modulo, the sum of squares for ten million values exceeds 64 bits and the Cython and Numba versions disagree with the Python version.

plaintext

def total_overflow(unsigned long long n):
    cdef unsigned long long i
    cdef unsigned long long acc = 0
    for i in range(n):
        acc += i * i
    return acc  # wraps silently once the sum exceeds 2**64 - 1

Parallel loops use prange from cython.parallel, and the function must release the GIL around the loop.¹³ The example scales an array in place, which has no cross-iteration dependency and needs no reduction.

plaintext

from cython.parallel cimport prange

def scale_parallel(double[::1] values, double factor):
    cdef Py_ssize_t index
    for index in prange(values.shape[0], nogil=True):
        values[index] *= factor

A reduction such as a sum needs care, because two threads writing the same accumulator lose updates. Cython detects simple += reductions in prange, and a manual reduction needs a per-thread buffer or an atomic. Measure the parallel version before trusting it: on a memory-bound loop, adding threads changes little and may make the result slower.

On a free-threaded build, Cython 3.1 and later can mark a module as compatible with the directive # cython: freethreading_compatible = True. The directive is a promise, not a check. Code that still relies on the GIL for object safety must use with gil: or a critical section with cython.critical_section.¹⁴

plaintext

# cython: freethreading_compatible = True

import cython

lock = cython.pymutex()

def update(shared):
    with cython.critical_section(lock):
        shared.append(1)

7. Numba: JIT for Numeric Code

Numba compiles a function to machine code on the first call. Decorate it with @numba.njit to require nopython mode, which refuses to fall back to the interpreter and therefore refuses unsupported Python features.¹⁵ The first call pays for compilation; later calls run at native speed.

plaintext

import numba

@numba.njit(cache=True)
def total(n):
    acc = 0
    for i in range(n):
        acc = (acc + i * i) % 1000000007
    return acc

cache=True writes the compiled code to disk so the next process skips compilation, and it requires a real file rather than a REPL or a notebook cell. Without it, every worker process compiles its own copy, which matters in a process pool.

parallel=True turns prange loops into threaded code, and fastmath=True allows floating-point reassociation.¹⁶ Fast math can change results, especially around NaN and infinity, so it belongs only in code that has tests against a reference implementation.

plaintext

import numba
from numba import prange

@numba.njit(parallel=True, cache=True)
def scale(values, factor):
    for index in prange(values.shape[0]):
        values[index] *= factor
    return values

Numba works best on scalars, NumPy arrays and loops over them. It supports typed containers such as numba.typed.List and numba.typed.Dict, and it does not support most of pandas, arbitrary class hierarchies or dynamic attribute access. A function that uses those features either fails to compile or falls back to object mode, which is slower than plain Python and should be treated as a bug.

Numba feature	Use
`@njit`	Require nopython mode
`cache=True`	Reuse compiled code across processes
`parallel=True` and `prange`	Thread a loop with no dependencies
`fastmath=True`	Allow reassociation, with a numerical risk
`@vectorize`	Build a NumPy ufunc from a scalar function
`@stencil`	Express a neighbourhood operation on an array

💡 Tip: Use Ctrl + Mouse Wheel to zoom, or click the fullscreen icon

Description

A state diagram of a Numba function from first call through type inference, compilation, machine code execution, and the cached path on later calls or later processes.

Fill in the Blanks

How to answer

Fill-in-the-blank exercise with 1 blanks. Type the missing words, then check your answer.

Complete the decorator that requires Numba's fast nopython mode.

import numba @numba.blank def total(n): acc = 0 for i in range(n): acc += i * i return acc

Word Bank, drag words into the blanks above:

njit

8. The Measured Comparison

One workload, four implementations, same machine and same interpreter. Each function computes the same modular sum over ten million integers and returns the same result, so the comparison is honest.

plaintext

import time
import numpy as np
import numba
from fastsum import total_typed

N = 10_000_000
MOD = 1_000_000_007

def pure(n):
    acc = 0
    for i in range(n):
        acc = (acc + i * i) % MOD
    return acc

@numba.njit(cache=False)
def nb(n):
    acc = 0
    for i in range(n):
        acc = (acc + i * i) % MOD
    return acc

def numpy_version(n):
    values = np.arange(n, dtype=np.int64)
    return int(((values * values) % MOD).sum() % MOD)

def bench(fn, arg):
    best = float("inf")
    result = None
    for _ in range(3):
        start = time.perf_counter()
        result = fn(arg)
        best = min(best, time.perf_counter() - start)
    return best, result

print("pure", bench(pure, N))
print("cython", bench(total_typed, N))
print("numba", bench(nb, N))
print("numpy", bench(numpy_version, N))

Implementation	Seconds for 10 million iterations	Speedup over pure Python
Pure Python	0.69	1.0x
Cython, typed loop	0.026	26x
Numba, `njit`	0.037	19x
NumPy, vectorized	0.07	10x

Seconds for ten million modular sum iterations by implementation

Description

Bar chart of seconds for ten million iterations on CPython 3.14.7. Pure Python takes 0.69 seconds, a typed Cython loop takes 0.026 seconds, Numba njit takes 0.037 seconds and a NumPy vectorized version takes 0.07 seconds.

The ordering is instructive. Cython and Numba keep the accumulator in a register and never allocate an array. NumPy allocates an 80 megabyte input array, an 80 megabyte square array and a modulo temporary, so the loop becomes memory-bound. Vectorization is not automatically the fastest option; it is the fastest option when the data already lives in an array and the operation maps cleanly onto whole-array primitives.

Numba's first call needs compilation before it reaches the measured state. On this machine the first call to the trivial function took 0.15 seconds and the second took 0.037 seconds. A real function with more types takes longer, which is why a service should warm the kernel at startup or enable cache=True.

python

def speedup(baseline: float, candidate: float) -> float:
    return baseline / candidate

print(round(speedup(0.69, 0.026), 1))
print(round(speedup(0.69, 0.037), 1))
print(round(speedup(0.69, 0.07), 1))

9. Profiling a Production Process

Production profiling has different constraints from a laptop benchmark. The process is already running, restarting it loses the state that reproduces the problem, and the profiler must not add enough overhead to change the behaviour.

py-spy top shows a live view, py-spy dump prints every thread's stack once, and py-spy record writes a flame graph.² Because it reads process memory from outside, it works on a process started without any profiling flags.

bash

py-spy top --pid 12345
py-spy dump --pid 12345
py-spy record -o profile.svg --pid 12345

Tachyon does the same from the standard library on Python 3.15 and later, with a GIL mode that shows how much time the process spends holding the interpreter lock.⁴ That view is the quickest way to prove that a service is GIL-bound rather than I/O-bound.

bash

python -m profiling.sampling attach 12345
python -m profiling.sampling attach 12345 --gil --flamegraph -o gil.html

A sampling profiler can miss a function that runs for less than one sample period. If the suspicion is a fast function called a million times, use the deterministic tracer on a reduced workload, or a line-level tool such as scalene, and confirm with a targeted timeit microbenchmark.

10. Failure Modes

Symptom	Likely cause	Fix
Benchmark result changes every run	No warm-up, small input, noisy machine	Pin the input, warm up, report the minimum
Cython loop is still slow	Types missing, objects still created	Read the `cython -a` annotation report
Native result differs from Python	Integer overflow or dtype truncation	Use a wider type or a modulo
Segfault in a Cython extension	Bounds checks disabled and index out of range	Re-enable `boundscheck` and add a test
Numba function is slower than Python	Object mode fallback	Use `njit` and remove unsupported features
Every worker recompiles Numba code	`cache=True` missing or unusable	Enable the cache and use a file-based module
Wheel will not import	ABI mismatch or missing platform tag	Build with cibuildwheel for each target
Free-threaded build loses parallelism	An extension re-enabled the GIL	Check `sys._is_gil_enabled()` and the extension metadata

The last row connects this lesson to the concurrency lesson. A native extension that has not declared free-threading support causes CPython to re-enable the GIL when it is imported, which silently removes the parallel speedup the team expected. Cython's freethreading_compatible directive is the declaration for Cython modules.¹⁴

python

import sys
import sysconfig

print(sysconfig.get_config_var("Py_GIL_DISABLED"))
print(sys._is_gil_enabled())

Run that check after the imports, not at the top of the module, because the state can change when an extension loads.

11. Case Studies

Case study: pandas and Cython

pandas implements performance-critical kernels in Cython, and its contributor guide documents the Cython conventions used across the code base.¹⁷ Grouping, window operations and string handling all have typed inner loops that avoid creating a Python object per row.

What to copy: keep the public API in Python and move only the measured hotspot into Cython. What to avoid: compiling a function before the algorithm and the data layout are settled, because every change then requires a rebuild.

Case study: scikit-learn and Cython

scikit-learn uses Cython for tree building, pairwise distances and many preprocessing kernels, and its developer guide specifies how those modules are written and tested.¹⁸ The estimators stay in Python, and the numeric inner loops are compiled with typed memoryviews.

What to copy: the separation between an estimator's Python API and its compiled kernel, which keeps the code testable and the wheel build predictable. What to avoid: exposing the compiled module as public API, because the ABI and the signatures can change between releases.

Case study: STUMPY and Numba

STUMPY computes the matrix profile, a similarity join over time series, and uses Numba to compile the inner loops over NumPy arrays.¹⁹ The algorithms are expressed as loops over arrays, which is exactly the shape Numba handles well, and the package ships as a normal wheel without a C build step.

What to copy: choose Numba when the hot code is a numeric loop over arrays and a pure Python wheel is a deployment requirement. What to avoid: assuming Numba covers dataframe operations, because the unsupported path falls back to slow object mode.

12. 2026 Updates (as of November 2026)

Python 3.15.2 is the current stable release, and 3.15 adds the profiling package with the Tachyon sampling profiler and a tracing module.⁴⁵
Tachyon can attach to a running process by pid, dump a single stack snapshot, and record a flame graph, which removes the need for a third-party profiler in many cases.⁴
The 3.15 release also upgrades the experimental JIT compiler, so interpreter level speedups continue to improve without changes to application code.⁵
Cython 3.3.1 is the current release, with freethreading_compatible and critical sections for free-threaded builds.¹⁴
Numba 0.68.1 supports Python 3.10 through 3.15 and requires NumPy below 2.6, so a NumPy upgrade can force a Numba upgrade.¹⁵
Cython, Numba and NumPy all publish version-specific wheels, and a free-threaded build needs cp315t or abi3t wheels rather than the default ones.

📚 Did you know?

timeit disables the garbage collector during a run by default, which removes collection pauses from the measurement but also hides the cost of allocation.⁷
A Cython memoryview declared as double[::1] rejects a non-contiguous array at the call, while double[:] accepts it and indexes more slowly.¹⁰
Numba compiles a separate specialization for every argument type signature, so a function called with both int32 and int64 arrays pays twice.¹⁵