Python Async Foundations: The Event Loop and Coroutines

Concurrency vs Parallelism

Concurrency is about structure — dealing with many things at once by interleaving them. Parallelism is about execution — literally doing many things at the same instant on multiple CPU cores. A single-core machine can be concurrent (it rapidly switches between tasks) but never truly parallel. As Rob Pike put it: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once."

A useful analogy is a short-order cook. A single cook (one CPU core) prepares five orders concurrently by starting the eggs, chopping vegetables while they cook, then flipping the eggs — interleaving work so all five progress. The cook never does two things in the same instant; they switch whenever one task is waiting. Add a second cook and you get parallelism: two dishes genuinely advancing simultaneously.

Python's asyncio uses cooperative multitasking. Tasks run on a single thread and voluntarily surrender control at explicit points — no task is ever interrupted mid-statement. This contrasts with preemptive threading, where the OS can suspend any thread at any moment. Cooperative scheduling is easier to reason about because your code only pauses where you wrote await, but it carries an obligation: every task must yield promptly, or it starves the others.

The Global Interpreter Lock

The Global Interpreter Lock (GIL) is a mutex inside CPython that ensures only one thread executes Python bytecode at a time, even on a multi-core machine. It exists because CPython's reference-counting memory management is not thread-safe. Ten CPU-bound threads on an eight-core machine still effectively run one at a time, taking turns holding the GIL.

The crucial escape hatch: the GIL is released during blocking I/O. When a thread performs a blocking system call — reading a socket, writing a file, waiting on the network — CPython releases the GIL so other threads can run, and reacquires it when the call returns. This is why the GIL "has never been much of a problem for I/O-bound workloads."

Figure 2.1: Choosing a concurrency tool by workload shape

When async wins: many idle connections, few CPU cycles

How does the GIL affect asyncio specifically? Barely at all. By design, asyncio runs entirely in a single thread, so there is only ever one thread contending for the lock. asyncio does not rely on the GIL being released; it achieves concurrency by never blocking in the first place — coroutines cooperatively yield at await points, and the loop uses non-blocking I/O to make progress on many operations while waiting.

This is exactly the shape of our gateway. Thousands of chat connections sit mostly idle, occasionally lighting up with a message that triggers a network call to an AI backend. There is very little CPU work and an enormous amount of waiting. One thread can service thousands of concurrent connections because, at any instant, almost all of them are parked at an await, consuming no CPU.

The corollary is a hard limit: asyncio gives you no CPU parallelism — a CPU-heavy computation inside a coroutine monopolizes the single thread and stalls every other task. The standard mental model holds: asyncio is concurrency without parallelism, one thread, cooperatively scheduled, excellent for an I/O-bound gateway handling many simultaneous channels.

Coroutines, Awaitables & the Event Loop

async def, await, and the coroutine lifecycle

A coroutine is a function defined with async def. The single most misunderstood fact in all of asyncio: simply calling a coroutine function does not run it — it merely creates a coroutine object.

import asyncio

async def greet(name):
    print(f"Hello, {name}")

coro = greet("Slack")   # nothing prints! coro is just a coroutine object
print(type(coro))       # <class 'coroutine'>

asyncio.run(coro)       # NOW it runs: prints "Hello, Slack"

To execute a coroutine it must be driven by the event loop through one of three mechanisms: asyncio.run() (the top-level entry point), an await expression inside another coroutine, or asyncio.create_task(), which schedules it for concurrent execution. Think of a coroutine object as a recipe card rather than a cooked meal — writing "scramble the eggs" is not the same as scrambling them; a cook (the loop) must pick up the card and follow it.

Figure 2.3: The coroutine object lifecycle from creation to completion

Awaitables: coroutines, Tasks, and Futures

A Future is a low-level object representing a result not yet available — a promise a value (or exception) will eventually be set. A Task is a subclass of Future that "drives" a coroutine: it steps the coroutine forward until it yields control, records what it is waiting on, and arranges to be resumed later. Awaiting a coroutine directly runs it inline; create_task() hands it to the loop to run concurrently.

async def fetch(channel):
    print(f"start {channel}")
    await asyncio.sleep(1)          # simulate a 1s network wait
    print(f"done {channel}")
    return channel

async def main():
    t1 = asyncio.create_task(fetch("slack"))
    t2 = asyncio.create_task(fetch("discord"))
    results = await asyncio.gather(t1, t2)
    print(results)                 # ['slack', 'discord']

Both fetch calls run concurrently: the whole program takes about one second, not two, because while slack is parked at its await, the loop advances discord.

How await yields control back to the event loop

When a Task steps its coroutine and it reaches an await on something unfinished — say, a Future for a socket read that has not arrived — the coroutine yields control all the way back up to the loop. The Task adds a done-callback to the awaited Future so that when it resolves, the Task is rescheduled. Critically, the paused coroutine's entire local state — variables and code position — is preserved on the suspended stack frame. No coroutine is ever preempted mid-statement; it surrenders the CPU voluntarily at an await.

Figure 2.4: How await yields control back to the event loop and resumes later

The event loop iteration cycle

The event loop is a single-threaded scheduler that runs tasks and callbacks, performs network I/O, and manages subprocesses. Conceptually it is an infinite while loop asking two questions: "Is any callback ready to run right now?" and "Is any I/O or timer ready?" Internally it maintains three key structures: the ready queue (a FIFO deque of callbacks ready to run), the scheduled heap (a time-ordered min-heap of timer handles powering asyncio.sleep), and the selector (an OS-level I/O multiplexer watching many file descriptors at once).

One iteration: (1) compute a timeout — 0 if the ready queue is non-empty, else the time to the nearest scheduled timer; (2) call selector.select(timeout), blocking efficiently until an FD is ready or the timeout expires; (3) move each ready FD's callback into the ready queue; (4) move now-due timers into the ready queue; (5) drain the ready queue, calling each callback once. Then repeat.

Figure 2.2: One iteration of the event loop cycle

The selector underneath (epoll / kqueue / IOCP)

When a coroutine performs a socket operation that would block, asyncio registers that socket's file descriptor with the selector together with a callback, and creates a Future for the coroutine to await. The selector monitors all registered descriptors in a single system call — on Linux, one epoll_wait. When data arrives on any watched socket, the selector wakes, the callback runs, the Future resolves, and the coroutine resumes.

A traditional thread-per-connection server would spin up thousands of OS threads, each consuming memory and kernel scheduling. The event loop instead hands the kernel a single list of file descriptors and asks, in one syscall, "wake me when any of these has activity." One thread, one epoll_wait, thousands of connections — the architectural payoff that makes an async gateway both lightweight and scalable.

The Blocking Trap

Identifying blocking calls

The cardinal rule of asyncio is: never run blocking code directly on the event loop. The loop is single-threaded and cooperative — it runs one callback at a time and only regains control when that code voluntarily yields at an await. A blocking call is any function that holds the thread and does not yield: time.sleep(), synchronous requests.get(), a blocking database driver, blocking file I/O, or a long CPU computation.

While such a call runs, the loop's thread is stuck inside it — it cannot reach its next epoll_wait, dispatch I/O events, resume any coroutine, or fire any timer. As the docs put it: "If a function performs a CPU-intensive calculation for 1 second, all concurrent asyncio Tasks and IO operations would be delayed by 1 second." A single stray time.sleep(0.1) in a hot path freezes the entire loop for 100 ms, cascading across thousands of in-flight requests. The classic trap is requests: it looks async because it sits inside an async def, but nothing yields to the loop.

# WRONG — freezes the entire loop for one second
async def bad():
    time.sleep(1)                 # blocking!
    html = requests.get(url).text # blocking!

# RIGHT — yields to the loop; other tasks run during the wait
async def good():
    await asyncio.sleep(1)        # scheduled timer, yields control
    async with httpx.AsyncClient() as client:
        html = (await client.get(url)).text   # async I/O

Figure 2.5: The blocking trap versus offloading with run_in_executor / to_thread

Offloading with run_in_executor and to_thread

Sometimes no native-async replacement exists — a legacy library, a synchronous vendor SDK, or genuinely CPU-bound work. Then you must offload the blocking call to a separate thread or process so the loop's thread stays free. The lower-level primitive is loop.run_in_executor(executor, func, *args): it submits func to a concurrent.futures executor and returns an awaitable Future.

import asyncio, concurrent.futures, requests

def blocking_fetch(url):
    return requests.get(url).text     # blocking, but runs off-loop

async def main():
    loop = asyncio.get_running_loop()
    # None -> the loop's default ThreadPoolExecutor
    html = await loop.run_in_executor(None, blocking_fetch, "https://example.com")

The high-level convenience wrapper (Python 3.9+) is asyncio.to_thread(func, *args, **kwargs). It runs a synchronous function in a separate thread and returns a coroutine you can await, accepts keyword arguments directly, and propagates contextvars — the recommended, ergonomic choice for offloading blocking I/O. Because of the GIL, it is genuinely useful only for I/O-bound blocking calls; for CPU-bound tasks it provides no parallelism, so use run_in_executor with a ProcessPoolExecutor.

Diagnosing a stalled event loop

Even with discipline, blocking calls sneak in — often deep inside a third-party dependency. Enabling debug mode — via asyncio.run(main(), debug=True), loop.set_debug(True), or PYTHONASYNCIODEBUG=1 — makes the loop log a warning whenever a callback or task runs longer than loop.slow_callback_duration (default 100 ms), a strong signal something is blocking. Debug mode also warns about coroutines that were never awaited and non-thread-safe API misuse. A stalled loop is not a slow feature — it is a system-wide outage in slow motion, and debug mode is your early-warning system.