04 — Backend Performance: Network (Proximity, Keep-Alive, CDN, Prefetch)

Between the backend and the user sits the wire, and four items live on it [1]. The model that holds them together: every round trip has a floor of latency, and you can only do two things about it — reduce the number of round trips, or shorten the distance each one travels. Four items, two per lever.

Shorten the distance

• Minimize network latency by hosting your backend close to your users — geography is destiny. A request from Singapore to a Virginia origin pays ~200 ms of network round trip before any bytes move; a request from Singapore to a Singapore region pays ~5 ms. The choice isn't always yours (legacy infra, a single on-prem origin, regulatory questions) but when you can pick, edge or multi-region hosting removes the largest non-software latency on the path. For a global user base this single move can dwarf every other backend optimization.
• Use CDNs for static and frequently accessed assets — the same idea, applied to what you can't move. The CDN sits the static bytes (and, these days, parts of dynamic responses — full-page cache at the edge, ISR) close to the user so the long haul back to your origin happens once per edge node, not once per request. The CDN edge takes the round trip your origin would have paid. The mental check: every byte the user gets back that didn't have to cross to your origin is a free latency win.

Reduce the number of round trips

• Utilize HTTP keep-alive to reduce connection overhead — the wire-level version of connection pooling. Every fresh TCP connection pays the handshake (a round trip, plus TLS's extra one or two if it's HTTPS) before any payload moves. HTTP keep-alive keeps the connection open across requests, so the second and every subsequent request to the same host starts from bytes, not from a handshake. A backend client (a service-to-service HTTP call, an external API integration) without keep-alive pays two to three round trips per call — often tens of milliseconds on a real network, on every single call. Server-side, the fix is ensuring your server/framework keeps the keep-alive default (most do) and that the client passes a connection pool; connection reuse is usually the network's largest invisible win.
• Prefetch or preload resources, data, or dependencies needed for subsequent requests to minimize latency — the proactive move. You pay round trips on demand; prefetching pays them ahead of the demand. Three forms:
• Data prefetch in your code: when a request comes in, kick off the secondary reads you'll need (the user's permissions, the cached view) in parallel instead of sequentially. Concurrency collapses sequential round trips into one wall-clock second.
• Service-to-service prefetch: ask a downstream service for the next page of results while you're still rendering the current one, so the next request is already in flight.
• Edge prefetch: the CDN or framework prefetches the next likely page so the navigation is instant. Right when the next request is predictable (it usually is — same user, similar flow); wasteful when it isn't. Don't prefetch everything; prefetch the one next thing.

The two levers, two items each

There's a deliberate symmetry in the four items — two widen the network within reach (host close, use a CDN), two collapse round trips (keep-alive, prefetch). When I'm auditing a slow backend the first question is always which lever I haven't pulled. A backend in a single region serving global traffic has nothing to fix with keep-alive — distance is the bottleneck — and a backend at the edge doing fresh connections per request has nothing to fix with region choice — round trips are. The four items are not a checklist to work through; they're a 2x2 — _distance × count_ — and the slow backend is usually slacking on one quadrant while obsessing over another.

The strongest single move is almost always HTTP keep-alive, because it's free, default-on in modern servers, and frequently accidentally off in clients (a requests.Session you forgot to keep around, an http.Client without a transport pool in Go). The second is CDN in front of everything capable of being cached — including HTML via ISR — because the round trips the edge absorbs are round trips the origin never pays. Together they remove the two largest network costs on a typical backend without any code change at all.

References

• [1] roadmap.sh, "Backend Performance Best Practices — Network," roadmap.sh, 2024. [Online]. Available: https://roadmap.sh/backend-performance-best-practices
• [2] MDN, "HTTP keep-alive," Mozilla, 2024. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive
• [3] Cloudflare, "What is a CDN?," Cloudflare Learning, 2024. [Online]. Available: https://www.cloudflare.com/learning/cdn/what-is-a-cdn/