03 — Availability vs Consistency, and the CAP Trade-off

The trade-off at the heart of every distributed system was something I nodded along to in meetings without a crisp model of my own. Writing it down forced the idea into focus: availability is the system's ability to keep answering requests in the face of failures; consistency is the guarantee that every client sees the same data at the same time. [1] In a distributed system you often trade one for the other, and the CAP theorem is the precise statement of why you can't always have both.

The framing that finally clicked is that the trade-off is _forced_, not chosen for fun. Networks partition — cables get cut, switches fail, a data center loses contact with the others. When that happens, a distributed store faces a binary choice: keep answering from whichever nodes are reachable (available, but they might return stale or conflicting data), or refuse to answer until the partition heals and every replica can agree again (consistent, but effectively unavailable). You do not get to opt out of the network being unreliable, so partition tolerance is a given, and the real decision is consistency-vs-availability _during a partition_ [1][2].

The three guarantees, and which two you actually pick

CAP gives every distributed system three properties to weigh [2]:

• Consistency — every read receives the most recent write, or an error.
• Availability — every request receives a response, without a guarantee that it contains the most recent version.
• Partition Tolerance — the system keeps operating despite arbitrary message loss or failure between nodes.

Networks are not reliable, so you must support partition tolerance [2]. That collapses the design space to two practical flavors:

• CP — consistency + partition tolerance. When a node can't reach the others, the system waits rather than risk returning a non-latest value. Good when your business rules require atomic reads and writes [2].
• AP — availability + partition tolerance. The system returns the most readily available version of the data on any reachable node, even if it isn't the latest. Writes propagate once the partition heals. Good when eventual consistency is acceptable, or when staying up through external errors matters more than perfect freshness [2][3].

Why this is forced, and why "eventual consistency" is the escape hatch

The thing I had to internalize is that there is no "CA" option in a real network. The instant two nodes can't talk, the system either keeps serving (AP) or stops serving stale reads (CP). The much-misunderstood corollary is that "eventual consistency" is not a bug — it is the explicit, chosen trade-off for AP systems: once writes stop and the partition heals, all replicas converge to the same value _eventually_, even if they briefly disagreed [3]. A social feed showing a like count a few seconds stale is fine; a bank balance showing two different values on two nodes during a transfer is not.

The practical upshot is that the choice is driven by the data, not the database. The same application often mixes modes — strong consistency for the payments ledger, eventual consistency for the activity feed — and that mix is normal, not a compromise.

How I use this

When I model a data store now, I ask one question before anything else: when the network partitions, does this data refuse to answer or serve stale? Payments, inventory, and auth tokens are CP — a wrong answer is more expensive than no answer. Feeds, likes, recommendations, and analytics are AP — a slightly stale answer is fine, and staying up matters more. Naming that choice per data type, up front, is the whole habit. The CAP theorem did not give me a new tool; it gave me a vocabulary for a decision I was already implicitly making, often badly.

References

[1] R. Greiner, "CAP theorem revisited," robertgreiner.com, 2014. [Online]. Available: http://robertgreiner.com/2014/08/cap-theorem-revisited/

[2] H. Robinson, "CAP FAQ," GitHub (open source), 2017. [Online]. Available: https://github.com/henryr/cap-faq

[3] "A plain english introduction to CAP theorem," ksat.me. [Online]. Available: http://ksat.me/a-plain-english-introduction-to-cap-theorem

[4] "The CAP theorem," YouTube, 2020. [Video]. Available: https://www.youtube.com/watch?v=k-Yaq8AHlFA

[5] "CAP theorem," YouTube, 2021. [Video]. Available: https://www.youtube.com/watch?v=_RbsFXWRZ10&t=1s