Guide sets out performance principles for Tokio-based Rust applications
A first-draft guide drawn from RustConf and TokioConf discussions focuses on diagnosing scheduling delays, controlling contention and balancing latency against throughput.
A new guide has outlined practical principles for improving the performance and latency of Rust applications running on the Tokio asynchronous runtime, while stressing that results depend on the workload and operating environment.
Published through a Hacker News-linked post, “Principles for Fast Tokio Applications” recommends starting with the metric an application is trying to improve. Tokio’s schedule-latency histogram can show how long a task waits between becoming ready and being polled, although the measure does not itself identify the cause of a delay.
The guide highlights the trade-off between fairness and batching. In heavily pipelined workloads, repeatedly processing immediately-ready data can create long polls and allow one client’s work to delay another’s. Yielding after each request can reduce latency, while yielding after several consecutive ready reads may preserve more batching efficiency.
It also warns that excessive task spawning, blocking work, lock contention and unbounded concurrency can add runtime overhead or stall workers. Suggested responses include batching related blocking operations, keeping lock critical sections very short and using a semaphore or similar mechanism to limit concurrency. The guide cautions that tools such as spawn_blocking are not universal solutions and that workload-specific observations should not be treated as benchmarks.
For highly latency-sensitive services, the post discusses isolating Tokio workers and background threads on separate CPU cores, using multiple runtimes and, in carefully controlled circumstances, briefly spinning instead of yielding. These approaches consume resources and are presented as advanced, context-dependent choices.
The document is described as a first draft and proposed living guide. It also records a Tokio implementation change: a sharded blocking queue shipped briefly in version 1.52.0, was reverted in 1.52.1 after a reported regression, and later returned as an unstable feature disabled by default.
