Chapter 6: Classic Patterns: Producer-Consumer, Reader-Writer, Thread Pools

Why this matters

These three patterns recur constantly in real concurrent systems — under different names, in different languages, but always solving the same underlying coordination problem. Recognizing the pattern by shape, rather than reinventing an ad hoc solution each time, is what separates experienced concurrent-systems engineers from those who solve each new instance from scratch.

Producer-consumer

[BEGINNER] The producer-consumer pattern decouples work generation from work processing: one or more producer threads generate items and place them in a shared queue; one or more consumer threads take items from that queue and process them — neither side needs to know anything about the other, only about the queue between them. Chapter 5 already showed the implementation (queue.Queue); here's the pattern with multiple producers and consumers, its more common real-world shape:

import queue
import threading

work_queue = queue.Queue(maxsize=10)  # bounded -- prevents unbounded memory growth if
                                        # producers outpace consumers (a real, practical concern)

def producer(producer_id):
    for i in range(5):
        item = f"item-{producer_id}-{i}"
        work_queue.put(item)  # BLOCKS if the queue is full at maxsize -- automatic backpressure
        print(f"Produced {item}")

def consumer(consumer_id):
    while True:
        item = work_queue.get()
        if item is None:  # a "poison pill" -- the standard way to signal a consumer to stop
            break
        print(f"Consumer {consumer_id} processed {item}")

producers = [threading.Thread(target=producer, args=(i,)) for i in range(2)]
consumers = [threading.Thread(target=consumer, args=(i,)) for i in range(3)]
for t in producers + consumers:
    t.start()
for t in producers:
    t.join()
for _ in consumers:
    work_queue.put(None)  # send a poison pill per consumer, to cleanly signal shutdown
for t in consumers:
    t.join()

[SENIOR INSIGHT] The bounded queue (maxsize=10) is doing real work here beyond memory management: it provides automatic backpressure — if consumers can't keep up, producers naturally block rather than piling up unlimited work in memory. An unbounded queue would let a slow consumer and a fast producer silently accumulate unbounded memory usage, potentially leading to an out-of-memory condition (the companion Operating System book's Chapter 6) under sustained load — bounding the queue converts an invisible, slow-building memory problem into an immediately visible, back-pressuring slowdown instead.

Reader-writer pattern

[BEGINNER] The reader-writer problem addresses a specific, common asymmetry: many operations only read shared data (safe to do concurrently with other reads), while some operations write it (which must exclude all other readers and writers while it happens). A plain mutex treats every access identically (exclusive), which is unnecessarily restrictive when reads vastly outnumber writes — a reader-writer lock allows multiple concurrent readers, or exactly one exclusive writer, but never both at once.

import threading

class ReadWriteLock:
    """A basic reader-writer lock: many readers concurrently, OR one exclusive writer."""
    def __init__(self):
        self._readers = 0
        self._read_lock = threading.Lock()   # protects the _readers counter itself
        self._write_lock = threading.Lock()  # held exclusively by a writer, or by the FIRST reader

    def acquire_read(self):
        with self._read_lock:
            self._readers += 1
            if self._readers == 1:
                self._write_lock.acquire()  # first reader blocks any writer from starting

    def release_read(self):
        with self._read_lock:
            self._readers -= 1
            if self._readers == 0:
                self._write_lock.release()  # last reader allows a writer to proceed

    def acquire_write(self):
        self._write_lock.acquire()  # excludes ALL readers and other writers

    def release_write(self):
        self._write_lock.release()

[IN PRACTICE] This pattern's real-world payoff shows up specifically in read-heavy workloads: a configuration object read constantly by many request-handling threads but updated only rarely (a config reload) benefits enormously from allowing all those frequent reads to proceed concurrently, reserving exclusivity only for the rare write — using a plain mutex here would serialize every single read unnecessarily, even though reads never actually conflict with each other.

[SENIOR INSIGHT] A subtlety worth knowing: naive reader-writer lock implementations (including the simplified one above) can suffer writer starvation — if readers arrive continuously and overlap with each other, a waiting writer might never find a moment when reader count drops to zero. Production-grade reader-writer lock implementations typically add a fairness mechanism (e.g., once a writer is waiting, new readers are queued behind it rather than allowed to keep jumping ahead) specifically to prevent this — a direct, concrete instance of Chapter 3's starvation hazard, requiring the same category of fix (fairness).

Thread pools and executors

[BEGINNER] Creating a new thread has real overhead (the companion OS book's Chapter 2 covers thread creation/context-switch costs), and creating one thread per task, for a workload with many short-lived tasks, wastes that overhead repeatedly. A thread pool pre-creates a fixed number of worker threads that pull tasks from a shared queue (directly built on the producer-consumer pattern above) and process them, reusing the same threads across many tasks rather than paying creation overhead per task.

from concurrent.futures import ThreadPoolExecutor
import time

def process_task(task_id):
    time.sleep(0.1)  # simulate I/O-bound work
    return f"Task {task_id} complete"

with ThreadPoolExecutor(max_workers=5) as executor:
    # submit() returns a Future immediately; the actual work happens on a pooled worker thread
    futures = [executor.submit(process_task, i) for i in range(20)]
    for future in futures:
        print(future.result())  # blocks until THIS specific task completes
# The pool's 5 threads are reused across all 20 tasks -- no per-task thread creation cost.

Choosing pool size: for I/O-bound work, pool size can reasonably exceed the number of CPU cores (since threads spend most of their time waiting, not computing — the OS book's Chapter 9 distinction between I/O-bound and CPU-bound applies directly); for CPU-bound work in a GIL-constrained runtime, a thread pool provides no parallelism benefit at all (Chapter 10 explains why in depth), and a ProcessPoolExecutor (process-based, sidestepping the GIL) is the correct tool instead.

Common mistakes

1. Using an unbounded queue in a producer-consumer system, missing the backpressure benefit a bounded queue provides and risking unbounded memory growth if production outpaces consumption.
2. Using a plain mutex for a genuinely read-heavy, write-rare workload, unnecessarily serializing concurrent reads that could safely proceed in parallel with a proper reader-writer lock.
3. Not accounting for writer starvation in a naive reader-writer lock implementation under continuous read load — production systems should use a well-tested, fairness-aware implementation rather than a naive one like this chapter's illustrative example.
4. Using a ThreadPoolExecutor for CPU-bound work in CPython, expecting a speedup that the GIL (Chapter 10) prevents — ProcessPoolExecutor is the correct tool for genuine CPU-bound parallelism in Python.

Discussion / interview questions

1. "What problem does a bounded queue solve in a producer-consumer system that an unbounded queue doesn't?" Strong answer: Provides automatic backpressure — if consumers can't keep up with producers, the bounded queue causes producers to block rather than accumulate unlimited, unprocessed work in memory, converting a slow, invisible memory-growth problem into an immediately visible slowdown instead.

2. "Why would you use a reader-writer lock instead of a plain mutex for a configuration object read frequently and updated rarely?" Strong answer: A plain mutex serializes every access, including reads that never actually conflict with each other; a reader-writer lock allows many concurrent readers, reserving exclusivity only for the rare write, significantly improving throughput for a read-heavy workload.

3. "What is writer starvation in a reader-writer lock, and how is it typically prevented?" Strong answer: Continuous, overlapping reader arrivals can prevent a waiting writer from ever finding a moment with zero active readers; prevented by a fairness mechanism that queues new readers behind an already-waiting writer, rather than always letting readers proceed ahead of it.

4. "When would a thread pool provide no performance benefit for a CPU-bound Python workload, and what would you use instead?" Strong answer: In CPython, the GIL prevents true parallel execution of Python bytecode across threads, so a ThreadPoolExecutor provides no speedup for CPU-bound work; a ProcessPoolExecutor (using separate processes, each with its own interpreter/GIL) is the correct tool for genuine CPU-bound parallelism.

Chapter summary / cheat sheet

• Producer-consumer: decouples work generation from processing via a shared queue — a bounded queue provides essential backpressure, preventing unbounded memory growth.
• Reader-writer lock: allows many concurrent readers or one exclusive writer, significantly improving throughput for read-heavy workloads over a plain mutex — watch for writer starvation in naive implementations.
• Thread pools: reuse a fixed set of worker threads across many tasks, avoiding per-task thread creation overhead — appropriate pool sizing and pool TYPE (thread vs. process) depends on whether the workload is I/O-bound or CPU-bound.
• All three patterns are built from the primitives in Chapter 4 (locks, condition variables) — recognizing which pattern fits a given coordination problem is usually faster and safer than reinventing an ad hoc equivalent.

---