baam's wiki

CPU-bound vs IO-bound

Budding

Sep 14, 2025 3 min read

Most performance advice is really about one prior question: is your program CPU-bound or IO-bound? The answer decides which resource you are fighting, which concurrency model fits, and which languages will feel effortless versus which will fight you the whole way.

Telling them apart

  • CPU-bound work spends its time computing: number crunching, transformations, encryption, image and video encoding. It is limited by how fast the processor can execute instructions - clock speed, core count, cache locality, vectorization. You make it faster with more or faster cores and data-parallel code.
  • IO-bound work spends its time waiting: disk reads and writes, network calls, database queries. The CPU sits mostly idle while the latency of the external operation dominates. You make it faster by overlapping the waiting - non-blocking IO and many lightweight concurrent tasks.

A quick test: if pinning the CPU at 100% is what makes the job finish sooner, you are CPU-bound; if the CPU idles while you wait on something else, you are IO-bound.

Why it dictates the concurrency model

The bottleneck picks the tool.

  • CPU-bound work wants parallelism - real threads on real cores, or SIMD and GPU offload - to do more computation at once. An event loop buys nothing when the CPU is already the limit.
  • IO-bound work wants cheap concurrency - a way to keep thousands of waits in flight without burning a thread on each. Thread-per-connection collapses under context-switch cost; event loops and lightweight tasks (goroutines, coroutines, async/await) shine.

Confusing the two is the classic mistake: throwing async at a CPU-bound loop, or spawning a thread per socket for tens of thousands of idle connections.

Where the languages fall

The workload maps straight onto language strengths:

  • C, C++, Rust own CPU-bound work - compiled to the metal, manual or zero-cost memory control, aggressive optimization. Rust pairs that with a strong async story (Tokio) for IO too.
  • Java and C# are capable all-rounders - JIT-optimized hot loops for compute, mature async frameworks for IO - at the cost of GC tuning and warm-up time.
  • Go is built for IO concurrency: goroutines and a non-blocking net stack make massive concurrency trivial with almost no ceremony. It is competent but not a leader at raw CPU-bound number crunching, where the garbage collector and scheduler start to show.
  • Python fits IO-bound web work well (asyncio, FastAPI), but the GIL and interpreter make it a poor choice for raw compute unless you drop into C-extensions (NumPy) or multiprocessing.
  • Node.js is superb at IO - its whole API is non-blocking by design - and a poor choice for heavy computation, which blocks its single event-loop thread until you offload to worker threads.

The bridge

“CPU-bound or IO-bound” is the question sitting underneath the entire concurrency conversation - threads versus coroutines, why Go’s goroutines feel free, why an event loop can stall on one bad function. That machinery is its own topic: see threads_vs_coroutines.