Performance & Latency MOC

Performance is not a number, it is a set of tradeoffs: time against space, latency against throughput, simplicity against scale. These notes map the territory - first how to measure it honestly, then the mechanisms that produce it, then how to defend it when load arrives.

Measuring

• Latency vs Throughput vs IOPS: why your fast API still fails at scale - the three numbers people confuse.
• Tail latency: the number the average hides - why the p99.9 is what users actually feel.

Mechanisms

• Caching Isn't Cheating - trading space and a little staleness for a lot of time.
• Queues Are Everywhere - buffers that decouple producers from consumers, hiding everywhere from SQS to CPU pipelines.

Defenses (staying fast under load)

• Rate limiting as an invariant - protecting the system and keeping access fair when demand spikes.
• CPU-bound vs IO-bound - knowing which kind of work you have, so you pick the right concurrency model.

Related boundaries: Why APIs break and Retries and idempotency sit at the edge of this map, where performance meets reliability.