AI agents slash the cost of software performance optimisation

A technical article published by Dan Luu argues that artificial intelligence agents have drastically reduced the cost and complexity of software performance optimisation. Luu contends that techniques previously requiring rare specialist expertise, such as writing Just-In-Time compilers or complex multi-threading algorithms, can now be executed by general developers using AI coding agents with minimal human input. This shift suggests that dynamic, workload-specific software optimisation is becoming a viable standard, potentially rendering traditional performance engineering roles less critical for bounded problems.

Luu cites specific experiments to support this thesis. In one instance, an AI agent built a competitive AI for the board game Azul that outperformed existing world-class AIs, primarily due to optimisations such as multi-threading and native code versions that would have taken weeks for a human to implement. In another experiment, an AI agent optimised a regular expression engine for a specific workload, achieving a two per cent speedup on holdout benchmarks after only minutes of human setup. A separate experiment involving a native code compiler for regex matching yielded a two to four times performance improvement for long queries, though only a seven per cent speedup for representative holdout queries.

The article highlights the changing dynamics of technical expertise. Jamie Brandon, a performance engineer, noted that the AI model Claude achieved better results on Anthropic’s performance take-home assignment than he did. Brandon cited “crazy” optimisations that he would not have attempted without weeks of work, suggesting that on well-defined problems, a decent model can outperform a human expert given comparable time controls. Luu argues that while AI agents may lack the judgement for open-ended problems, they are sufficient to achieve reasonable performance on a wide range of tasks.

Marc Brooker, whose observation is cited in the article, described “dynamic custom software, fitted to a particular workload rather than a class of workloads” as a likely outcome of this trend. This approach mirrors historical techniques in demoscene and high-performance computing, where software was tailored to specific hardware or problems. Luu notes that this trend is already visible in projects like pgrust, where workload-specific optimisations are being added as needed without extensive framework setup.

The implications for the industry are significant. Historically, performance optimisation was expensive and required rare skills, making it worthwhile only for large-scale or highly lucrative projects. Luu states that the cost of formerly specialised performance work has dropped by many orders of magnitude. This means that developers can now implement optimisations that were previously too expensive to be worthwhile, such as building a fast local index for a machine or creating custom JIT compilers for databases.

Luu acknowledges that the time required to make results rigorous has not decreased, but the time to run an experiment and see a result has. This allows developers to test more optimisations and iterate faster. The article concludes that while AI agents do not replace the judgement of expert engineers, they lower the barrier to entry enough to make high-performance software a more common standard across the industry.