AI lowers the barrier to fast JIT compilation in Rust

Historically, building a fast Just-In-Time (JIT) compiler was considered a black art, requiring deep knowledge of assembly language. This complexity meant that most production-ready databases relied on LLVM or generated C and C++ code, both of which suffered from high compile times. A new project, however, suggests that artificial intelligence is changing this dynamic by making it significantly easier to write JIT compilers that target assembly directly.

The developer, working on the pgrust database, detailed the construction of a JIT compiler in Rust that compiles code in approximately five microseconds. This speed enables the system to JIT compile every SQL query, rather than just a subset, contributing to the database’s overall performance. The project leverages a copy-and-patch approach to generate ARM64 machine code, a method that achieves performance comparable to handwritten implementations.

To demonstrate the technique, the developer built a simple regular expression engine supporting literal strings and repetition. Benchmarks comparing the JIT-compiled version against an interpreter and a handwritten implementation showed that the JIT version performed neck and neck with the hand-rolled code. In some instances, the JIT version was faster, while in others, the handwritten version held the edge, but both significantly outperformed the interpreter.

The copy-and-patch method relies on a series of assembly templates, or stencils, for different operations. When compiling an operation, the system takes the associated stencil and makes small tweaks based on the specifics of the task. By stringing together these filled stencils, the compiler constructs a program at runtime that mirrors the efficiency of manually written assembly.

The developer noted that AI assistance was crucial in overcoming the historical difficulty of writing assembly. By providing the general shape of the JIT compiler to a coding agent, the AI handled the intricate details of instruction modification and memory management. This approach has effectively lowered the barrier to entry for software projects that previously found JIT compilation too difficult to implement.

This development represents a shift in how high-performance code generation is approached. While most databases have traditionally avoided custom JIT compilers due to the engineering overhead, the integration of AI tools suggests that new databases can be more ambitious in their architectural designs. The pgrust project stands as a practical example of how these tools can enable faster, more efficient data processing.