Chapter 8: Load Balancers, Reverse Proxies, and CDNs

Why this matters

A request from a browser almost never talks directly to the exact server that generates its response — it passes through one or more intermediaries first, each doing a specific, well-defined job. Understanding what each intermediary actually does (and doesn't do) explains a large share of real production networking confusion: why a client's IP address shows up wrong in application logs, why "the load balancer" gets blamed for problems it didn't cause, and why a CDN can serve traffic for a site that's completely down at its origin.

Reverse Proxy: standing in front of a server

[BEGINNER] A reverse proxy sits between clients and one or more backend servers, forwarding client requests to a backend and returning the backend's response to the client — from the client's perspective, the proxy is the server, since the actual backend is invisible behind it. (This is the mirror image of a forward proxy, which sits in front of clients, forwarding their outgoing requests on their behalf — used for things like corporate outbound filtering — a different, less commonly confused case worth distinguishing by name.)

Client ----> Reverse Proxy (e.g., nginx) ----> Backend Server (e.g., your application)
             (the client only ever "sees" the proxy's address, never the backend's directly)

Common jobs a reverse proxy takes on: TLS termination (handling the TLS handshake from Chapter 7 once, at the proxy, so backend servers only ever deal with plain, decrypted HTTP internally), request/response header manipulation, serving static files directly without involving the application server at all, and basic routing (sending /api/* to one backend, /static/* to another).

Load Balancer: distributing traffic across multiple servers

[BEGINNER] A load balancer is a specialized reverse proxy whose primary job is choosing which of several available backend servers should handle a given request — enabling horizontal scaling (many servers sharing load) and fault tolerance (automatically routing around a server that's failed health checks).

Layer 4 vs. Layer 7 load balancing (directly using Chapter 2's OSI layer vocabulary):

	Layer 4 (Transport)	Layer 7 (Application)
Decides routing based on	IP address and port only	Full HTTP request content — path, headers, cookies
Can it route /api/* and /static/* differently?	No — has no visibility into HTTP content	Yes — this is exactly what it's good at
Overhead	Lower — doesn't need to parse HTTP at all	Higher — must parse and understand HTTP
TLS visibility	Passes encrypted traffic through untouched	Can terminate TLS (see the backend's actual plaintext requests)

Common load-balancing algorithms:

Algorithm	How it decides	Trade-off
Round Robin	Cycles through servers in order	Simple, but ignores each server's actual current load
Least Connections	Sends to whichever server has the fewest active connections	Better load awareness, slightly more bookkeeping
IP Hash	Routes based on a hash of the client's IP	Same client consistently reaches the same server ("session affinity"/"sticky sessions") — useful when server-side session state isn't shared across servers

The client IP problem: X-Forwarded-For

[SENIOR INSIGHT] Because a reverse proxy/load balancer sits between the client and the backend, the backend's TCP connection is technically with the proxy, not the original client — so naively reading the connecting IP address at the backend gives you the proxy's IP, not the real client's. The standard fix: the proxy adds an X-Forwarded-For header containing the original client's IP before forwarding the request, and the backend must be explicitly configured to read that header instead of the raw connection IP.

Client (203.0.113.5) ---> Load Balancer (10.0.0.1) ---> Backend Server
Backend sees the TCP connection FROM 10.0.0.1 (the load balancer), NOT 203.0.113.5.
The load balancer adds: X-Forwarded-For: 203.0.113.5
Backend must read THIS header (from a TRUSTED proxy) to get the real client IP —
and must NOT blindly trust this header from untrusted, direct external clients,
since it's just a regular header anyone could forge if allowed to connect directly.

This single detail explains a very common production bug class: rate limiting, geo-blocking, or audit logging implemented against the raw connection IP silently applies to the load balancer's IP for every single request, rather than the real client's — a bug that's invisible in local development (no load balancer in the path) and appears only in production.

CDN: pushing content geographically closer to users

[BEGINNER] A Content Delivery Network (CDN) is a globally distributed network of servers that cache and serve content from a location physically close to each requesting user, rather than every request traveling all the way to one origin server — directly leveraging Chapter 9's point that physical distance is a real, physics-bound contributor to latency (the speed of light is a hard limit, not an engineering problem to optimize away).

Without a CDN: User in Tokyo <---------- (long round-trip) ----------> Origin server in Virginia
With a CDN:    User in Tokyo <-- (short round-trip) --> CDN edge server in Tokyo
                                                              |
                                                    (only on a cache MISS)
                                                              v
                                                    Origin server in Virginia

[IN PRACTICE] A well-configured CDN can continue serving cached content (a "stale-while-revalidate" or similar strategy) even when the origin server is completely unreachable — which is exactly why a site's static assets, or even entire cached pages, can sometimes remain available during an origin outage, while anything requiring a fresh, uncached response (a personalized page, a write operation) fails. Recognizing this distinction ("is what's failing cacheable, or does it require the origin?") is often the fastest way to understand the actual blast radius of an origin-side incident.

Common mistakes

1. Reading the raw connecting IP address at a backend server behind a load balancer, silently logging/rate-limiting/geo-blocking against the load balancer's IP for every request instead of the real client's — always use X-Forwarded-For (or an equivalent, framework-specific mechanism) when behind a proxy.
2. Trusting X-Forwarded-For blindly from untrusted, directly-reachable clients — it's just a header, forgeable by anyone who can connect directly to the backend without going through the trusted proxy; only trust it when the backend is provably only reachable through the trusted proxy.
3. Confusing Layer 4 and Layer 7 load balancing capabilities — expecting a Layer 4 load balancer to route based on URL path or cookies, which it structurally cannot do since it never inspects HTTP content at all.
4. Assuming a CDN outage means the whole site is down, or conversely assuming cached CDN content proves the origin is healthy — these are separate failure domains, and distinguishing "cacheable, served by the CDN" from "requires the origin" is the key diagnostic question during an incident.

Discussion / interview questions

1. "What's the difference between a reverse proxy and a load balancer?" Strong answer: A reverse proxy sits in front of one or more backends, forwarding client requests and hiding backend details from the client; a load balancer is a specialized reverse proxy whose primary job is choosing which of several backend servers should handle each request, for scaling and fault tolerance.

2. "What's the difference between Layer 4 and Layer 7 load balancing?" Strong answer: Layer 4 routes based only on IP/port, with no visibility into HTTP content (lower overhead, can't route by path/cookie); Layer 7 parses the actual HTTP request and can route based on path, headers, or cookies (higher overhead, more routing flexibility).

3. "Why would a backend server behind a load balancer see the wrong client IP address, and how is this typically fixed?" Strong answer: The backend's TCP connection is actually with the load balancer, not the original client, so the raw connection IP is the load balancer's; the standard fix is the X-Forwarded-For header, which the trusted proxy sets to the real client IP, and which the backend must be explicitly configured to read (while not trusting it from untrusted, directly-reachable sources).

4. "How can a CDN keep serving content during an origin server outage, and what's the limit of that resilience?" Strong answer: The CDN can continue serving already-cached content from its edge servers without needing to reach the origin; this only covers cacheable content — anything requiring a fresh response from the origin (personalized data, write operations) still fails during the outage, so the actual blast radius depends on how much of the site's traffic is genuinely cacheable.

Chapter summary / cheat sheet

• Reverse proxy: sits in front of backend(s), hides them from the client, commonly handles TLS termination and routing. Forward proxy: the mirror image, sits in front of clients.
• Load balancer: a specialized reverse proxy distributing traffic across multiple backends — Layer 4 (IP/port only, low overhead) vs. Layer 7 (full HTTP visibility, more routing flexibility), using algorithms like round robin, least connections, or IP hash (sticky sessions).
• Backends behind a proxy see the proxy's IP as the connection source — X-Forwarded-For carries the real client IP, and should only be trusted from provably trusted proxies.
• CDN: caches content geographically close to users, reducing physics-bound latency (Chapter 9) and providing partial resilience during origin outages — but only for genuinely cacheable content.

---