baam's wiki

The cost of async

Seedling

3 min read

The pitch for async is that it makes everything faster, so you should reach for it by default. The truth is narrower and more useful. Async buys one specific thing - the ability to keep many waits in flight cheaply - and it charges for that whether or not you use what you bought. When the work is IO-bound and concurrent, the trade is a bargain. When it is not, you are paying for machinery that does nothing.

Where the cost goes

The overhead stops being mysterious once you remember that every async function becomes a state machine (see Async/await: syntax over a state machine).

  • Allocation. Each in-flight task is an object on the heap - the state machine holding its resume point and surviving locals. A thousand concurrent tasks is a thousand small allocations the collector will later have to walk (see Garbage collection: the pause you didn't schedule).
  • Scheduling. Parking a task, registering its wakeup, and pushing its continuation back onto the ready queue is real bookkeeping the runtime does at every suspension point. A synchronous call does none of it.
  • Lost locality. Straight-line code stays hot in cache. A resumed task is a jump back into a half-finished computation, often cache-cold, sometimes on a different core. The CPU works harder just to pick up where it left off.

When synchronous wins

  • The operation is fast or already done. If an await almost always has its result ready, the state-machine scaffolding costs more than the wait it was meant to hide. Good runtimes short-circuit a completed await, but the machinery still has a floor.
  • The work is CPU-bound. Async overlaps waiting, and CPU-bound work does not wait, it computes. Wrapping a number-crunching loop in async adds overhead to something that was never going to yield anyway (see CPU-bound vs IO-bound).
  • Concurrency is low. A handful of concurrent operations does not need thousands-of-tasks machinery. Plain threads, or plain blocking calls, can be both simpler and faster.

It does not create capacity

The most common misread is treating async as if it adds throughput. It adds not a single cycle of CPU. It lets a thread that would have blocked go do other useful work instead. If there is no other useful work - because everything is CPU-bound, or because there is only one request in flight - there is nothing to overlap, and async is pure tax. And the tax has a way of surfacing in the tail: scheduler contention and the collection of all those task objects show up at p99, not in the average.

The bottom line

Async is a lever, not a default. Pull it when you have many concurrent operations that spend their time waiting - network services, proxies, anything IO-bound at scale. Leave it alone when you have few operations, fast operations, or CPU-bound ones. “Is this faster with async?” has the same answer as most performance questions: it depends entirely on whether you are waiting or working.