Multiprocessing vs. multithreading in Python — when doe...

The deciding factor is Python’s Global Interpreter Lock, which allows only one thread to execute Python bytecode at any given moment. Threads share the same memory space, and critically, the GIL is released while a thread waits on I/O — a network call, a disk read — so threading shines specifically for I/O-bound work. Calling 1,000 embedding API endpoints with a ThreadPoolExecutor gets you close to a 10x speedup, because almost all the wall-clock time is spent waiting on the network, and threads can overlap those waits.

Multiprocessing spins up entirely separate processes, each with its own memory and its own GIL, which means it genuinely runs on multiple CPU cores in parallel — the right tool for CPU-bound work like extracting features from 10,000 images, where the bottleneck is actual computation rather than waiting. The rule of thumb: calling external services, use threads; number-crunching or pixel-crunching, use processes. PyTorch’s DataLoader(num_workers=N) uses multiprocessing under the hood for exactly this reason, to overlap data loading with GPU training.