What is Little's Law?

Little's Law states that in a stable system, the average number of items present (N) equals the average arrival rate (λ) multiplied by the average time each item spends in the system (W): N = λ × W. For a web service, N is the concurrent workers needed, λ is the request rate, and W is the average latency.

Should I always provision exactly N = λ × W threads?

That is the minimum for a fully-loaded system. In practice, add a safety buffer (10–30%) so that latency spikes or garbage-collection pauses do not instantly saturate the pool and cause queuing. The tool lets you dial in a buffer percentage.

What happens if my pool is smaller than the Little's Law result?

Requests queue waiting for a free thread. Queuing latency adds to the average W, which raises N further — a feedback loop that compounds under sustained load. The symptom is P99 latency growing much faster than P50.

What is the CPU-bound ceiling?

For a worker pool, the concurrency ceiling before all cores are busy is cores / CPU-active fraction per request. For example, 8 cores and a 25% CPU-active fraction support about 32 concurrent workers; the rest can wait on I/O. This is a planning estimate, so confirm it with a load test.

Does this apply to async / event-loop runtimes?

Little's Law still holds for event-loop systems — the "N" becomes in-flight async tasks rather than OS threads. The key difference is that async runtimes can reach high concurrency with far fewer OS threads, so memory overhead per concurrent request is lower.

How does database connection pool size relate to this?

Apply the same formula to the DB layer: λ_db (DB queries per second) × W_db (average query latency). That gives the minimum connection count. Database servers typically cap connections at 100–500 due to memory limits, so this calculation helps you negotiate that ceiling against your service's needs.

Concurrency Calculator

Enter target throughput, average latency, and CPU core count to calculate the ideal thread pool size using Little's Law (N = λ × W).

Service Parameters

Little's Law: N = λ × W

Target Throughput (λ) — req/s

Requests per second the service must sustain at steady state.

Average Latency (W) — ms

End-to-end time per request including DB, network, and downstream calls.

CPU Cores

Logical cores (vCPUs) available to the service process.

Advanced

Safety buffer (%)

Extra headroom above the Little's Law minimum. Recommended: 20–30%.

CPU active fraction

Fraction of each request's time spent on CPU (not waiting). I/O-heavy: 0.05–0.20. CPU-heavy: 0.70–1.0.

Little's Law Result

Min Concurrency (N)

—

threads / workers

Recommended Pool

—

with 25% buffer

Latency Impact on Pool Size

How pool size changes if average latency drifts from your baseline.

Latency	N (min)	N (recommended)

Set your parameters and click Calculate

Summary

Concurrency Calculator applies Little's Law — N = λ × W — to determine how many concurrent workers (threads, goroutines, or connections) a service needs to sustain a target throughput at a given average latency. It compares the theoretical minimum against the practical CPU-bound ceiling, flags over-provisioning, and shows a scaling table so you can plan fleet size before hitting production.

How it works

Enter the target throughput (requests per second) your service must sustain.
Enter the average end-to-end latency per request in milliseconds (include DB and downstream wait time).
Enter the number of CPU cores available to the service process.
Optionally set a safety buffer (%) to add headroom above the Little's Law minimum.
Read the recommended pool size, the CPU-bound ceiling, and the utilization breakdown.

Use cases

Size a thread pool for a new microservice before the first load test.
Diagnose thread starvation: if your pool is smaller than N = λ × W, queuing latency compounds.
Prevent over-provisioning: a pool much larger than the CPU ceiling wastes memory and increases context-switch overhead.
Plan connection pool limits for database drivers (HikariCP, pgBouncer, SQLAlchemy).
Model the concurrency impact of a 2× latency regression without changing throughput targets.
Compare thread vs. goroutine concurrency needs for the same workload.

Frequently Asked Questions

Last updated: 2026-06-19 · Reviewed by Nham Vu

Concurrency Calculator

Service Parameters

Little's Law Result

CPU-Bound Ceiling

Breakdown

Latency Impact on Pool Size

Summary

How it works

Use cases

Frequently Asked Questions

Concurrency Calculator

Use the Concurrency Calculator

Service Parameters

Little's Law Result

CPU-Bound Ceiling

Breakdown

Latency Impact on Pool Size

Summary

How it works

Use cases

Frequently Asked Questions