Threads, Processes, and the GIL in Practice

A script that reads forty URLs takes forty times as long as it should. You add threads, the wall clock drops, and you conclude that Python's threads work. Then you point the same pattern at a number-crunching loop and nothing improves at all. Both results come from the same design decision inside the interpreter, and until you know which one you are looking at, you are guessing.

The guess is expensive. The wrong choice between threads and processes costs you a rewrite, and the wrong choice about shared state costs you a bug that appears once a week in production and never on your laptop.

The GIL protects interpreter state, not your data

CPython has a single lock, the Global Interpreter Lock, that a thread must hold before it can execute Python bytecode. Only one thread runs Python code at a time. Every release of CPython through 3.14 ships this way by default, so the examples here are about the build you almost certainly have: on this machine, sys._is_gil_enabled() returns True.

The lock exists because CPython's memory management is not thread safe. Reference counts go up and down on almost every operation, and two threads adjusting the same count without a lock would corrupt the heap. The GIL is one coarse lock standing in for thousands of fine ones. It is a deliberate trade: single-threaded speed and a simple C extension API in exchange for no parallel Python execution. Free-threaded builds exist from 3.13 onward, and they are getting usable, but they are a separate download with their own packaging story. Do not assume one.

The GIL is not the thing that makes your counter wrong. That is the next point, and it trips up more people than the GIL itself.

The GIL is released every few milliseconds, and during I/O

The interpreter hands the lock around. A thread that has held it for the switch interval, five milliseconds by default, is asked to release it, which is why two threads running nothing but a busy loop both finish roughly on time:

a finished at 58952.740
b finished at 58952.752
two 0.5s loops in threads took 0.52s

Two half-second loops finishing within 12 milliseconds of each other is parallelism of a sort: the interpreter alternates between them. The total work did not shrink, and on a machine with spare cores you would expect two independent loops to finish in 0.5s of wall clock if they ran at the same time. They finish in 0.52s here because the work is trivial and the switching overhead is small, and the number stays near 0.5s whether you use one thread or two.

More importantly, C code inside CPython releases the lock whenever it is about to wait. Reading a socket, writing to a file, sleeping, waiting on a subprocess: in all of those, the interpreter drops the GIL and lets another thread run Python. That single behaviour is the entire reason threads are useful for network code.

Threads win when the work is waiting

Here is a loop over eight jobs that each spend 0.25 seconds asleep, which stands in for any call that waits on something outside the process:

import time
from concurrent.futures import ThreadPoolExecutor


def blocking_work(seconds):
    """Stand-in for any call that waits on something outside the process."""
    time.sleep(seconds)
    return seconds


jobs = [0.25] * 8

if __name__ == "__main__":
    started = time.perf_counter()
    [blocking_work(job) for job in jobs]
    one_at_a_time = time.perf_counter() - started

    for workers in (2, 4, 8):
        with ThreadPoolExecutor(max_workers=workers) as pool:
            started = time.perf_counter()
            list(pool.map(blocking_work, jobs))
            elapsed = time.perf_counter() - started
        print(f"{workers} workers  {elapsed:5.2f}s  speedup {one_at_a_time / elapsed:4.2f}x")

    print(f"sequential  {one_at_a_time:5.2f}s")

2 workers   1.01s  speedup 2.01x
4 workers   0.51s  speedup 4.00x
8 workers   0.25s  speedup 8.02x
sequential   2.03s

The speedup tracks the worker count almost exactly, and it would keep tracking it past eight, because the jobs are not competing for the CPU. They are competing for nothing. The GIL is held for a few microseconds to start each sleep and released for the whole 0.25 seconds.

pool.map preserves input order in its results, so eight workers do not reorder your output. It also raises the first exception it hits only when you consume the results, not when the work happens, which surprises people who expect the pool to stop early.

Threads are useless for busy work, and that is measurable

Now the same shape with real arithmetic. count_primes_below does trial division with no I/O at all:

import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def count_primes_below(limit):
    """Count primes below limit using trial division. Pure CPU work."""
    count = 0
    for candidate in range(2, limit):
        for divisor in range(2, int(candidate**0.5) + 1):
            if candidate % divisor == 0:
                break
        else:
            count += 1
    return count


def best_of(repeat, fn):
    times = []
    for _ in range(repeat):
        started = time.perf_counter()
        fn()
        times.append(time.perf_counter() - started)
    return min(times)


if __name__ == "__main__":
    limits = [150_000] * 4

    def sequential():
        return sum(count_primes_below(n) for n in limits)

    def threaded():
        with ThreadPoolExecutor(max_workers=4) as pool:
            return sum(pool.map(count_primes_below, limits))

    def processed():
        with ProcessPoolExecutor(max_workers=4) as pool:
            return sum(pool.map(count_primes_below, limits))

    sequential()  # warm-up, so import and pool start-up are not measured
    for label, fn in (
        ("sequential", sequential),
        ("ThreadPoolExecutor(4)", threaded),
        ("ProcessPoolExecutor(4)", processed),
    ):
        print(f"{label:<26}{best_of(3, fn):>7.2f}s")

sum(count_primes_below(150_000) for 4 items), best of 3 runs
sequential                   0.47s
ThreadPoolExecutor(4)        0.49s
ProcessPoolExecutor(4)       0.26s

On an M1 Pro with 8 logical cores, threads bought nothing: 0.49 against 0.47 is noise. Processes bought 1.8x, not the 4x the worker count suggests, because the platform has 6 performance cores and 2 efficiency cores, and process start-up plus pickling is charged to the measurement. Reporting this as a clean 3.9x would be a lie. The direction is what matters: to run Python bytecode in parallel you need separate interpreters, and that means processes.

Processes fork work through pickle, which limits what you can send

Every argument to a process worker and every return value travels through pickle, because the interpreter on the other side has its own memory. That rules out anything pickle cannot name by module path:

from concurrent.futures import ProcessPoolExecutor

DATA = [1, 2, 3, 4]


def square(n):
    return n * n


class Recorder:
    def __init__(self, prefix):
        self.prefix = prefix

    def label(self, n):
        return f"{self.prefix}{n}"


if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=2) as pool:
        print("module-level function:", list(pool.map(square, DATA)))

    recorder = Recorder("row-")
    with ProcessPoolExecutor(max_workers=2) as pool:
        print("bound method:", list(pool.map(recorder.label, DATA)))

    try:
        with ProcessPoolExecutor(max_workers=2) as pool:
            print(list(pool.map(lambda n: n * n, DATA)))
    except Exception as error:
        print(f"lambda failed: {type(error).__name__}: {error}")

module-level function: [1, 4, 9, 16]
bound method: ['row-1', 'row-2', 'row-3', 'row-4']
lambda failed: PicklingError: Can't pickle <function <lambda> at 0x10631a610>: it's not found as __main__.<lambda>

A lambda has no importable name, so there is nothing to send. A bound method works because pickle records the instance and the method name. A database connection, an open file, or a client holding a socket does not survive the trip either. The usual answer is to pass the data and let each worker build its own connection, which also means the connection setup cost lands on the worker.

Threads interleave, so shared state needs a lock

The GIL makes a single bytecode instruction atomic. It does not make a sequence of them atomic. Any read, compute, write pattern can be interrupted between the read and the write, and the second thread reads a value that is already stale. The window does not need to be wide, but it has to exist:

import threading

def run(use_lock, per_thread=2_000, threads=8, width=5_000):
    state = {"processed": 0}
    lock = threading.Lock()
    dirty = [True] * width

    def between_read_and_write():
        # stands in for any work done between reading the value and writing it back
        return any(not flag for flag in dirty)

    def worker():
        for _ in range(per_thread):
            if use_lock:
                with lock:
                    current = state["processed"]
                    between_read_and_write()
                    state["processed"] = current + 1
            else:
                current = state["processed"]
                between_read_and_write()
                state["processed"] = current + 1

    threads = [threading.Thread(target=worker) for _ in range(threads)]
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()
    return state["processed"]


if __name__ == "__main__":
    print("expected", 2_000 * 8)
    print("unsafe", [run(False) for _ in range(3)])
    print("lock  ", [run(True) for _ in range(3)])

expected 16000
unsafe [2317, 2878, 2374] seconds [1.26, 1.24, 1.19]
lock   [16000, 16000, 16000] seconds [1.23, 1.2, 1.2]

Roughly 85 out of every 100 increments were lost, and the three runs do not even agree with each other, which is the signature of a race rather than a logic error. The locked version is exact, and the elapsed time is the same to two decimal places. A lock that costs nothing because the critical section is short is the normal case.

Two cautions. threading.Lock is not reentrant: taking it twice in the same thread deadlocks, and if you need that, threading.RLock is the reentrant one. And a lock only helps if every path to the variable takes it. One unlocked writer is enough to keep the bug alive.

Measure the split before you choose

You can answer the threads-or-processes question without guessing. Run the workload sequentially and time it, then run it under ThreadPoolExecutor with four workers. If threads close the gap to roughly a quarter, the work is waiting and you are done. If they do not improve anything within noise, the work is CPU-bound, and the next thing to try is ProcessPoolExecutor with the same worker count. If processes help a little and the job is large, the bottleneck is often the pickling of arguments and results rather than the arithmetic, and the fix is to send less, or to have each worker read the data itself.

A profile tells you the same thing with less code. python -X importtime covers start-up, cProfile covers Python-level time, and on Linux perf or py-spy will show you whether the process is on-CPU or blocked. Timing is the cheapest first cut and it needs no extra tools, which matters because you will do it on a machine that is not your laptop.

import time
from concurrent.futures import ThreadPoolExecutor
from urllib.request import urlopen

URLS = [f"http://127.0.0.1:8801/item?name=item-{n}" for n in range(1, 9)]


def fetch(url):
    with urlopen(url, timeout=5) as response:
        return response.status


if __name__ == "__main__":
    started = time.perf_counter()
    [fetch(url) for url in URLS]
    print(f"sequential {time.perf_counter() - started:5.2f}s")

    with ThreadPoolExecutor(max_workers=4) as pool:
        started = time.perf_counter()
        statuses = list(pool.map(fetch, URLS))
    print(f"threaded   {time.perf_counter() - started:5.2f}s {statuses}")