AI Auto-Research Yields 232x Speedup in GPU Kernel Optimisation Contest
A 14-day auto-research challenge demonstrates how AI-driven loop engineering can drastically reduce kernel runtimes, though domain expertise remains critical for steering optimisation.
Sankalp has secured 12th place out of 183 participants in a GPU Mode and Core Automation auto-research contest, achieving a 232-fold speedup over the baseline solution. The competition, part of the Linear Algebra Kernels in the Age of Research series, required competitors to implement a batched square compact-Householder QR factorization. By utilising AI models Codex and Claude, Sankalp reduced the kernel runtime from approximately 419 milliseconds to 1,805 microseconds across various matrix shapes.
The contest mandated the use of Householder reflections to convert matrices into orthogonal and upper triangular forms. Sankalp implemented a blocked Householder algorithm with a trailing WY-update, a method that confines serial work to a narrow panel of columns while allowing the remainder of the matrix to be updated via efficient matrix-multiplication operations. This approach was designed to maximise utilisation of GPU tensor cores, which typically struggle with the serial dependencies inherent in standard QR decomposition methods.
Over the 14-day period, Sankalp submitted more than 1,500 iterations. The workflow relied on ChatGPT Pro and Claude Pro, with Modal used for profiling. The participant maintained a beam of three to five candidate solutions to avoid getting stuck in local maxima, a common pitfall in automated optimisation. Claude served as an advisor model to generate strategic ideas, while Codex executed the code changes and loop engineering tasks.
Key bottlenecks identified during the process were launch overhead and panel overhead, rather than memory or compute constraints. Sankalp noted that domain expertise was crucial for effectively steering the AI models, converting unknown unknowns into known unknowns. The participant acknowledged missed opportunities, including the failure to utilise NVIDIA’s tcgen05 instructions for Blackwell B200 tensor cores and the inability to exploit specific input distributions such as low-rank cases.
The results highlight the evolving landscape of high-performance computing, where AI agents can handle iterative refinement and code generation. However, the contest also revealed that while agents can achieve significant performance gains, human oversight remains necessary to guide the search space and interpret complex profiling data from tools like NCU and Modal. The workspace structure included detailed logging via AGENTS.md and log.md to track submission status and timings, ensuring that previous attempts were not repeated.
