---
title: "20 — NoSQL and Operating at Scale — Pick Your Data Model, Then Survive the Load"
uid: nosql-and-operating-at-scale
tags: ["observability", "roadmap:backend", "nosql", "circuit-breaker", "cassandra", "scalability", "neo4j", "redis", "mongodb", "backpressure"]
excerpt: "NoSQL is four data models chosen by query shape; operating at scale is one discipline repeated — assume failure, observe everything, degrade gracefully."
date: 2026-08-13T03:28:23+0000
source: https://www.aveshina.my.id/en/blog/nosql-and-operating-at-scale
---

NoSQL storage and "building for scale" operations looked like a mismatched final cluster until one frame unified them. The frame: **NoSQL is a choice of data model matched to query shape, and operating at scale is one discipline repeated — assume failure, observe everything, degrade gracefully.** [1][8] The data model decides how you store; the operations discipline decides how you survive.

The thread is that scale breaks the assumptions that worked at small size. A single-server relational database is consistent and simple because there's one copy of the data; the moment you shard or distribute for scale, you've chosen to relax something (CAP, from the previous post), and the operations cluster is the set of patterns that make the resulting system survivable. NoSQL databases were largely invented *because* scaling relational databases past one machine is hard — they bake specific trade-offs (AP availability, denormalization, eventual consistency) into the data model so the application doesn't fight them at runtime.

## NoSQL: four data models, four query shapes

**NoSQL** ("not only SQL") is a family of databases that handle data the relational model handles awkwardly — semi-structured, rapidly changing, or needing horizontal scale [1]. They're not one thing; they're four distinct data models, each matched to a query shape:

- **Document stores** (MongoDB, CouchDB) store data in flexible JSON-like documents with no fixed schema [3]. Each document is self-contained, with nested fields. Use them when the data is naturally hierarchical and the schema evolves — content management, catalogs, user profiles. MongoDB is the dominant choice, with horizontal scaling via sharding and high availability via replica sets.
- **Key-value stores** (Redis, DynamoDB) store data as a map from a key to a value, with no query language beyond "get/set by key" [2][4]. Use them when access is always by key — caching, sessions, rate-limit counters, leaderboards. Redis is the in-memory speed champion; DynamoDB is the AWS-managed, single-digit-millisecond key-value/document store.
- **Column-family stores** (Cassandra) store data in column families optimized for massive write throughput across many nodes [5]. Use them for time-series, event logs, and write-heavy workloads at enormous scale where availability matters more than strong consistency. Cassandra is AP (available + partition-tolerant) by design.
- **Graph databases** (Neo4j, AWS Neptune) store data as nodes and relationships, optimized for traversing connections [6]. Use them when the query is about relationships — social networks, recommendations, fraud detection. Neo4j is the open-source leader; Neptune is the managed AWS option.

The decision is *what does the query look like*, not "is SQL bad." If every query is "get this user's profile by ID," key-value or document. If every query is "find all sensors reporting in this time window," column-family. If every query is "find friends of friends of this user," graph. Relational databases can technically do all of these; the specialized stores do specific ones far better at scale.

There are also the specialty stores for particular shapes: **time-series** databases (InfluxDB, TimescaleDB) for timestamped metrics and events [7]; **columnar OLAP** stores (ClickHouse) for analytics over large datasets. These extend the same principle — match the storage engine to the query shape.

## Building for scale: assume failure, observe, degrade

Once distributed, systems fail in new ways — partial failures, network partitions, slow downstream services, cascading overload. The "Building for Scale" cluster is the set of patterns that make a distributed system survivable. The unifying discipline is three habits: assume failure is constant, observe everything, and degrade gracefully when something breaks.

## Observability, monitoring, telemetry

**Observability** is the ability to understand the system's internal state from its external outputs — primarily **metrics, logs, and traces** [9][10][11]. The three pillars:

- **Metrics** — numeric measurements over time (request rate, error rate, latency, CPU). Aggregatable, cheap, ideal for dashboards and alerting. Prometheus is the dominant collector; Grafana the dominant dashboard.
- **Logs** — discrete event records with context. Essential for debugging specific failures. Structured logging (JSON with fields) makes them searchable.
- **Traces** — the path of a single request across services, with timing per span. Essential for finding where latency lives in a distributed system. Jaeger and Zipkin are common.

**Monitoring** is the real-time observation of metrics for anomalies and performance issues, with dashboards and alerts [12]. **Telemetry** is the automated collection of this data from distributed systems. The point of all three is to know what's happening before users tell you — to detect, diagnose, and resolve issues proactively.

The practical discipline: every service emits structured logs, standard metrics (the RED metrics — Rate, Errors, Duration — for HTTP services), and distributed traces. Without these, operating at scale is flying blind; with them, the cause of a slow page is a query, not a guess.

## Throttling, backpressure, and load shifting

When load exceeds capacity, three patterns manage the excess:

- **Throttling** (rate limiting) controls the rate of request processing — limiting requests per time period per client, rejecting or queuing the excess [13]. Throttling protects the system from overload and ensures fair usage; it's the front gate that prevents a single noisy client from starving everyone else.
- **Backpressure** is flow control where the receiver signals its capacity to the sender — "slow down, I can't keep up" [14]. It prevents memory overflow when a fast producer feeds a slow consumer (a fast stream into a slow database). Backpressure is the polite alternative to unbounded buffering.
- **Load shifting** moves work from peak to off-peak periods — running a batch job at 2am, shifting processing to a cheaper region, deferring non-urgent work [15]. It balances demand against capacity and cost.

These three are the load-management toolkit. Throttling drops or queues excess at the edge; backpressure propagates "slow down" upstream; load shifting moves work in time. Together they keep a system responsive under load instead of collapsing.

## Circuit breaker and graceful degradation

Two patterns handle the inevitable downstream failure:

- **Circuit breaker** protects against cascading failures by temporarily stopping calls to a failing service [16]. Three states: **closed** (normal — calls go through), **open** (the downstream is failing — calls are short-circuited immediately, not even attempted), **half-open** (testing whether the downstream has recovered — a few calls go through to check). The circuit breaker prevents one failing dependency from dragging down the whole system by tying up threads waiting for timeouts.
- **Graceful degradation** ensures the system keeps functioning, partially, when components are unavailable [17]. If the recommendation service is down, show popular items instead of recommendations. If the personalization service times out, show the generic page. The system degrades rather than fails — fewer features, but the core still works.

These two compose: the circuit breaker trips, the system degrades gracefully around the missing dependency, the user gets a slightly reduced experience instead of an error page. This is how distributed systems stay up when their parts don't.

## How I use this

The storage half and the operations half each have a discipline. For storage: **start with Postgres unless a specific query shape demands otherwise.** The portfolio uses Postgres (via Supabase) for user data and Redis-adjacent patterns for caching. Most applications never need a specialized NoSQL store — the relational model covers the majority, and a specialized store is justified only when a query shape (graph traversal, time-series, massive key-value) genuinely outgrows what Postgres can do. Picking a NoSQL store because it's "modern" rather than because the query shape fits is a common, expensive mistake.

For operations: **build for failure from the start, because at scale failure is constant.** Every service emits metrics, logs, and traces. Every downstream call has a timeout and ideally a circuit breaker. Every feature can degrade when its dependencies fail. Load is throttled at the edge. The discipline isn't paranoid; it's the difference between a system that stays up when a dependency blips and one that takes a full outage because a downstream got slow.

The thread connecting the two halves — *match the data model to the query shape, then assume failure and observe everything* — is the operating model of a backend at scale. The NoSQL stores exist because scaling the relational model is hard; the operations patterns exist because scaling anything introduces failure modes the single-server world never had. Both are responses to the same pressure (more load than one machine can handle), and both reward deliberateness: choose the storage by query shape, survive the load by design.

## References

[1] MongoDB, "NoSQL Explained." [Online]. Available: [https://www.mongodb.com/nosql-explained](https://www.mongodb.com/nosql-explained)

[2] Redis Ltd., "Redis Documentation." [Online]. Available: [https://redis.io/docs/latest/](https://redis.io/docs/latest/)

[3] MongoDB Inc., "MongoDB." [Online]. Available: [https://learn.mongodb.com/catalog](https://learn.mongodb.com/catalog)

[4] AWS, "AWS DynamoDB." [Online]. Available: [https://aws.amazon.com/dynamodb/](https://aws.amazon.com/dynamodb/)

[5] Apache Software Foundation, "Apache Cassandra." [Online]. Available: [https://cassandra.apache.org/_/index.html](https://cassandra.apache.org/_/index.html)

[6] Neo4j, "Neo4j Website." [Online]. Available: [https://neo4j.com](https://neo4j.com)

[7] InfluxData, "InfluxDB Documentation." [Online]. Available: [https://docs.influxdata.com/influxdb/cloud/](https://docs.influxdata.com/influxdb/cloud/)

[8] "Scalable Architecture: A Definition and How-To Guide," SentinelOne. [Online]. Available: [https://www.sentinelone.com/blog/scalable-architecture/](https://www.sentinelone.com/blog/scalable-architecture/)

[9] New Relic, "Observability and Instrumentation." [Online]. Available: [https://newrelic.com/blog/best-practices/observability-instrumentation](https://newrelic.com/blog/best-practices/observability-instrumentation)

[10] "What is Instrumentation?," Wikipedia. [Online]. Available: [https://en.wikipedia.org/wiki/Instrumentation_(computer_programming)](https://en.wikipedia.org/wiki/Instrumentation_(computer_programming))

[11] "What is Telemetry?," Sumo Logic. [Online]. Available: [https://www.sumologic.com/insight/what-is-telemetry/](https://www.sumologic.com/insight/what-is-telemetry/)

[12] Prometheus Authors, "Prometheus Documentation." [Online]. Available: [https://prometheus.io/docs/introduction/overview/](https://prometheus.io/docs/introduction/overview/)

[13] AWS, "Throttling — Well-Architected Framework." [Online]. Available: [https://docs.aws.amazon.com/wellarchitected/2022-03-31/framework/rel_mitigate_interaction_failure_throttle_requests.html](https://docs.aws.amazon.com/wellarchitected/2022-03-31/framework/rel_mitigate_interaction_failure_throttle_requests.html)

[14] "Backpressure explained — the flow of data through software," Jay Phelps. [Online]. Available: [https://medium.com/@jayphelps/backpressure-explained-the-flow-of-data-through-software-2350b3e77ce7](https://medium.com/@jayphelps/backpressure-explained-the-flow-of-data-through-software-2350b3e77ce7)

[15] "Load Shifting," Wikipedia. [Online]. Available: [https://en.wikipedia.org/wiki/Load_shifting](https://en.wikipedia.org/wiki/Load_shifting)

[16] Microsoft, "Circuit Breaker — Azure Architecture Patterns." [Online]. Available: [https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker](https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker)

[17] "What is Graceful Degradation & Why Does it Matter?," HubSpot. [Online]. Available: [https://blog.hubspot.com/website/graceful-degradation](https://blog.hubspot.com/website/graceful-degradation)

```quiz
Q: NoSQL is not one thing but four data models matched to query shapes. Which model fits 'find friends of friends of this user'?
- Graph databases (Neo4j) — store nodes and relationships, optimized for traversing connections
- Key-value stores — get/set by key
correct: 0
explain: Graph queries (friends-of-friends, recommendations, fraud rings) are the graph database's specialty — they traverse relationships efficiently. Key-value stores only fetch by key. The model choice is driven by the query shape, not by fashion.

Q: Why is 'start with Postgres unless a specific query shape demands otherwise' sound advice?
- The relational model covers most query shapes; specialized NoSQL stores are justified only when a shape (graph, time-series, massive key-value) genuinely outgrows Postgres
- Postgres is always faster than every NoSQL store for every workload
correct: 0
explain: Most applications' queries fit the relational model. NoSQL is justified when a specific shape — graph traversal, high-write time-series, massive key-value — outgrows what Postgres can do. Picking NoSQL by fashion rather than query fit is a common, expensive mistake.

Q: What are the three pillars of observability, and what is each best for?
- Metrics (numeric measurements over time — dashboards/alerting), logs (discrete events with context — debugging), traces (a request's path across services — finding latency)
- CPU, RAM, disk
correct: 0
explain: Metrics are aggregatable and cheap (Rate/Errors/Duration dashboards, alerting). Logs give per-event context for debugging. Traces show where latency lives across services. Together they let you understand internal state from external outputs — essential at scale.

Q: What does a circuit breaker do, and what are its three states?
- It stops calls to a failing service to prevent cascading failures. States: closed (normal), open (short-circuit, don't call), half-open (test recovery with a few calls)
- It retries failed calls forever until they succeed
correct: 0
explain: A circuit breaker trips to open when a downstream fails repeatedly, short-circuiting further calls instead of waiting for timeouts. Half-open tests whether the downstream has recovered. This prevents one failing dependency from tying up threads and cascading failure through the whole system.

Q: What is graceful degradation, and how does it compose with the circuit breaker?
- The system keeps functioning partially when a component is unavailable (show popular items if recommendations are down). When the breaker trips, the system degrades around the missing dependency rather than erroring
- The system crashes immediately if any component is unavailable
correct: 1
explain: Graceful degradation keeps the core working when a dependency fails — fewer features, not an error page. The circuit breaker trips on the failing dependency; the system degrades gracefully around it. Together they keep the user experience alive when parts break.
```