14 — Asynchronism: Message Queues, Task Queues, and Back Pressure

"Just background it" was my queue strategy, and it skipped the two decisions that make the pattern safe at scale. Writing it down gave the pattern a precise shape: a queue is a buffer that decouples the producer (who hands off work) from the consumer (who does it), so the producer returns fast and the consumer works at its own pace. [1][3] The two things that make that decoupling safe at scale are choosing the right queue type and applying back pressure when the buffer fills.

The framing that landed is that asynchronism exists to fix a mismatch in _rate_. A user request arrives in milliseconds; the work it triggers (encoding a video, fanning out a notification, running a report) takes seconds or minutes. Holding the request open for the work to finish ties up a thread and exhausts the server under load. A queue absorbs the rate mismatch — the producer drops a message and leaves; the consumer pulls messages at whatever rate it can sustain [1].

Two queue types for two jobs

The roadmap distinguishes message queues from task queues, and the distinction is real [3][4]:

• Message queues receive, hold, and deliver messages. When an operation is too slow to perform inline, the workflow is: the application publishes a job to the queue and notifies the user of status; a worker picks up the job, processes it, then signals completion [4]. The user is not blocked. During processing, the client might do a little optimistic work to make the task _seem_ done — a posted tweet appears in the poster's timeline instantly, even if it takes time to fan out to all followers [4].
• Task queues receive tasks _and their related data_, run them, and deliver results. They support scheduling and are well-suited to computationally-intensive jobs in the background. Celery is the canonical Python example, with explicit scheduling support [3].

The line between them is fuzzy in products (RabbitMQ, Redis, Kafka, SQS each blur it), but the mental split is useful: a message queue carries "something happened, react to it" notifications; a task queue carries "run this job with this input" work items.

Back pressure: protecting the system when the queue fills

Queues are buffers, but buffers are finite. If producers consistently outpace consumers, the queue grows without bound — and once it exceeds memory, performance collapses into cache misses and disk reads that make everything slower [2]. Back pressure is the answer: limit the queue size to maintain high throughput and good response times for the jobs already in flight. Once the queue is full, clients get a "server busy" or HTTP 503 and are told to try again later, ideally with exponential backoff so retries space out instead of hammering a recovering system [2].

Think of it as a pressure valve. A queue without back pressure is a pipe that bursts under sustained load — the queue itself becomes the bottleneck, the very thing it was meant to absorb. A queue with back pressure degrades gracefully: it serves the jobs it can at full speed and refuses new ones loudly, so producers can shed load instead of silently piling up latency.

How I use this

The decision check is whether work on the request path takes longer than the request should hold open. If yes — anything that calls a slow third-party API, encodes media, fans out notifications, or runs a report — it goes onto a queue and the request returns immediately with a status handle. I default to a message queue for "react to this event" flows and a task queue for "run this expensive job" flows. And I always wire back pressure in from day one, because the failure mode of an unbounded queue under load is catastrophic and silent — by the time you notice, the queue is the outage.

References

[1] "Patterns for microservices — sync vs async," Medium, 2022. [Online]. Available: https://medium.com/inspiredbrilliance/patterns-for-microservices-e57a2d71ff9e

[2] M. Thompson, "Applying back pressure when overloaded," Mechanical Sympathy blog, 2012. [Online]. Available: http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html

[3] Celery, "Celery — distributed task queue," 2024. [Online]. Available: https://docs.celeryq.dev/en/stable/

[4] D. Martin, "Message queues," system-design-primer (open source), 2024. [Online]. Available: https://github.com/donnemartin/system-design-primer#message-queues

[5] "Little's law," Wikipedia. [Online]. Available: https://en.wikipedia.org/wiki/Little%27s_law

[6] Apache, "Apache Kafka," 2024. [Online]. Available: https://kafka.apache.org/