A self-paced course on how concurrency actually works in CPython — threads, the GIL, processes, and the APIs you build real systems with.
It starts from the operating system (what a thread is, who schedules it, what "shared memory" really means) and works up to production-shaped code: worker pools, producer/consumer pipelines, and a crypto-data case study that goes from a slow sequential script to a concurrent one.
Who this is for: you can write Python, but "use threads for I/O and processes for CPU" is advice you've repeated without being able to explain. By the end you'll know why, and you'll be able to predict which model wins before you benchmark.
- Explain what the GIL is, what it protects, and exactly when it stops being a problem.
- Look at a workload and correctly choose threads, processes, or
asyncio. - Recognise a race condition in code review, and fix it with the cheapest correct primitive.
- Reason about deadlocks instead of guessing (lock ordering, timeouts,
with). - Build a producer/consumer pipeline with bounded queues and clean shutdown.
- Use
concurrent.futureswell — including the error handling most tutorials skip.
Python basics (functions, classes, exceptions). No OS or systems-programming
background assumed — docs/01-os-foundations.md builds it.
git clone <this-repo>
cd Python-Concurrency-Threading-Multithreding-GIL-Operating-System
python3 -m venv .venv
source .venv/bin/activate # fish: source .venv/bin/activate.fish
pip install -r requirements.txt
jupyter labThen open notebooks/ and work through them in order.
Python 3.9+ works for everything here. Python 3.13+ additionally lets you see the GIL
being removed — see the free-threading section of docs/03-the-gil.md.
Other setups
Poetry: poetry install && poetry run jupyter lab
The 2020 lockfile: requirements-lock-2020.txt is the original hash-pinned
environment from the PyCon tutorial. It's kept for reproducibility but will not install
on modern Python — use requirements.txt.
Timing caveat: every benchmark in these notebooks depends on your core count and what else your machine is doing. Your numbers will differ from the recorded outputs. The ratios are what carry the lesson.
Read the doc in the same row before or alongside each notebook — the notebooks are the hands-on part, the docs are the mental model.
| # | Notebook | What it teaches | Read alongside |
|---|---|---|---|
| 1 | Thread basics | Starting threads, thread states, identity, join, subclassing Thread, how many threads you can actually start |
OS foundations |
| 2 | Shared data & race conditions | Why counter += 1 is not atomic, critical sections, Lock, the cost of synchronization |
Threads & shared state |
| 3 | Locks & deadlocks | How two correct locks produce a hung program, the four conditions, prevention strategies | Threads & shared state |
| 4 | Producer / consumer | queue.Queue as the synchronization primitive, tracking work with task_done/join, backpressure |
Patterns & pitfalls |
| 5 | The Python GIL | Threads making CPU-bound code slower, and the same threads making I/O-bound code 10× faster | The GIL |
| 6 | Multiprocessing | Process, real parallelism, the cost of it, sharing data with queues and pipes, process pools |
Choosing a model |
| 7 | concurrent.futures |
Executors, futures, map vs submit/as_completed, where exceptions go |
Patterns & pitfalls |
| 8 | The parallel library |
A higher-level third-party wrapper, and its error handling | — |
| 9 | asyncio & the event loop |
Cooperative concurrency: one thread, thousands of sockets, and where await actually yields |
Choosing a model |
Then the case study: notebooks/case_study_crypto/ —
downloading OHLC data from a dozen exchanges, sequentially and then concurrently. This is
where the course stops being toy examples. Start with its
README.
- Choosing a model: threads vs processes vs asyncio — the decision table to keep
- Patterns & pitfalls — the shapes worth copying, and the mistakes worth recognising
- Glossary — every term, defined once
- Exercises — per-chapter practice, no solutions given
- Further reading — talks, PEPs, books
| One sitting (~2h) | Notebooks 1, 2, 5 + the GIL. The minimum to stop being confused about threads in Python. |
| A weekend | Notebooks 1–7 + choosing a model. |
| A study group, 4 weeks | Week 1: 1–3. Week 2: 4 + 5. Week 3: 6 + 7. Week 4: 9 + the case study. One exercise set per week. |
notebooks/ the course, in order
case_study_crypto/ end-to-end example: slow script → concurrent system
docs/ the written explanations and reference material
img/ diagrams used by the notebooks and docs
data/ datasets (prime_mixture.txt, used by notebooks 5 & 6)
scripts/ small helper programs invoked from notebooks
Notebooks reference assets as ../img/... and ../data/..., so run Jupyter from the
repository root (or anywhere — the relative paths are resolved from the notebook's own
directory).
This course is built on Santiago Basulto's PyCon 2020 tutorial
— Concurrency and Parallelism for Beginners (slides).
The notebooks are his work; this repository reorganises them into a guided course and adds
the docs/ material, the asyncio chapter, and the exercises.
Original code MIT licensed — see LICENSE.
Supplementary video playlists: Operating Systems · Python