baam's wiki

Queues Are Everywhere

Evergreen

May 19, 2025 3 min read

Some queues you build on purpose - SQS, RabbitMQ, a Celery broker. Most queues you simply have, whether you named them or not: the kernel’s socket buffer, the scheduler’s run queue, the buffer in front of a disk, the event loop’s task list, even the pipeline inside a CPU. A queue is the most common structure in software that nobody draws on the architecture diagram.

What a queue is for

At its core a queue does one thing: it decouples a producer from a consumer so the two need not move at the same speed. The producer drops work and moves on; the consumer takes work when it is ready. That buffer absorbs bursts and smooths a mismatch in rates - the same reason a factory keeps buffer stock between two machines that run at different speeds.

Where they hide

  • Message brokers (SQS, RabbitMQ, Kafka) - the queues you meant to add.
  • OS socket and disk buffers - the kernel queues bytes you never asked it to.When packets arrive faster than your code calls read(), the kernel parks the bytes in a socket receive buffer - a FIFO in kernel memory you never allocated. Let it fill and TCP flow control tells the sender to slow down. That is backpressure, built into the network stack whether you asked for it or not.
  • The event loop - a queue of callbacks waiting their turn on a single thread.One thread runs one callback at a time, so everything else waits in a ready queue behind it. This is why a single slow synchronous function freezes an entire Node process or browser tab - it isn’t sharing the queue, it’s holding it hostage.
  • CPU pipelines and store buffers - queues in silicon.Even the chip refuses to wait politely. Instructions are kept in flight in a pipeline, and your finished writes sit in a store buffer before they reach memory, so the core can keep going while the slow parts drain. A fast producer decoupled from a slow consumer, three millimeters wide.

Once you see the shape, the mental model transfers everywhere.

Backpressure: when the consumer cannot keep up

A queue papers over a rate mismatch only as long as the consumer eventually catches up. If producers outrun consumers for good, the queue grows - and an unbounded queue is a trap. It converts a visible failure (rejecting work) into an invisible one (rising latency and memory) right up until something bursts. The fix is backpressure: a signal back to the producer to slow down or shed load. Applying that signal at the boundary is exactly what Rate limiting as an invariant does.

A queue is a small feedback loop

This is why the note is tagged for feedback loops and not just plumbing. A healthy queue is a control system: its depth is the signal, backpressure is the correction, and a bounded buffer is the setpoint. And it is governed by a single equation - Little’s Law, which ties queue length, arrival rate, and wait time together (see Latency vs Throughput vs IOPS: why your fast API still fails at scale). That law holds for every queue above, whether or not it has a name.

The bottom line

Reach for an explicit queue when you need to decouple producers from consumers or absorb bursts - but always bound it and always watch its depth. The dangerous queues are the ones you forgot you had.