Cua research layer boosts macOS VM LLM inference by up to 16x on Apple Silicon
New benchmarks on an M1 Ultra show significant performance gains for llama.cpp within virtualised environments, though results vary by framework.
Cua has released a research compatibility layer designed to accelerate large language model inference within macOS virtual machines on Apple Silicon hardware. The tool, built on the Lume virtualisation stack, intercepts Metal API queries to expose newer GPU capabilities to the llama.cpp framework. Benchmarks conducted on an M1 Ultra running macOS Tahoe indicate that this shim can increase prompt processing speeds by up to 11.08 times and token generation by up to 16.36 times compared to standard Virtualization.framework settings.
The technique utilises Apple’s existing paravirtualised graphics path and does not require direct hardware or PCI passthrough. In stock virtual machines, the paravirtualised device often reports older capability limits, such as an Apple 5-era family and restricted threadgroup memory. This causes inference engines like llama.cpp to select slower processing kernels, even though the underlying hardware could support more efficient paths.
Testing on an M1 Ultra with the TinyLlama 1.1B model showed prompt processing speeds reaching 98% of bare-metal performance and token generation reaching 72.06% of bare-metal speed. When tested with Google’s Gemma 4 12B QAT Q4_0 model, the shim improved prompt processing by 7.20 times and token generation by 14.54 times, with the virtual machine achieving 99.59% of bare-metal prompt speed and 94.82% of bare-metal generation speed.
The compatibility layer operates by modifying two specific reported values: the Apple-family answers and the threadgroup-memory limits. Other Metal settings, including common, Mac, and working-set-size values, remain at their stock configurations. This approach allows applications to select newer SIMD-group reduction and bfloat16 paths without altering the host kernel or requiring physical GPU assignment.
Performance gains were not universal across all frameworks. Tests using MLX-LM showed no improvement, as the stock virtual machine was already optimised for that specific framework. The release includes source code, build scripts, and raw benchmark logs to allow for reproducibility, with Cua inviting Apple engineers to clarify the supportability of unrestricted feature levels for paravirtualised graphics.
