Cactus Compute Unveils Needle 2: 14MB Agentic LLM for Edge Devices
Utilising 2-bit quantisation and a custom C++ inference engine, Needle 2 operates without GPUs or NPUs, with initial deployment already underway in Pebble’s Index 01 app.
Cactus Compute has released Needle 2, an open-source 45-million-parameter agentic large language model designed specifically for on-device execution. The model is optimised for tool calling, device control, and structured data extraction, targeting hardware with limited resources such as smartphones, wearables, and microcontrollers. Needle 2 operates as a 14MB binary requiring only 28MB of session RAM, utilising 2-bit quantisation and a custom C++ inference engine to function without the need for GPUs or NPUs.
The release addresses the disparity between high-end edge computing and the vast majority of connected devices, which often cost under $200 and lack dedicated AI accelerators. By focusing on function mapping rather than open-ended prose generation, the 45-million-parameter formulation aims to provide sufficient capability for routine device actions while maintaining privacy and low latency. Initial deployment includes use by Pebble in its Index 01 app for the Pebble Index Ring, allowing the wearable to process spoken requests locally without relying on a network connection.
Needle 2 is trained on a proprietary 115B-token corpus and post-trained on 38B tokens, incorporating compact reasoning traces and a carefully designed dataset distribution. The architecture features a Hadamard MLP to reduce weight read costs, an "engram" system for world knowledge via hashed n-gram tables, and multi-lane residual streams to provide routing flexibility. This design allows the model to maintain performance on resource-constrained silicon by minimising the arithmetic operations required per token.
The inference engine employs lossless 2-bit quantisation baked into the training process, known as Cactus Quants, ensuring the deployed model matches the training configuration. It utilises an end-to-end int8 arithmetic path and a grammar-based optimisation that skips up to 98% of vocabulary projection on structural tokens. The system uses a 256-token sliding window for KV cache management, ensuring deterministic RAM usage regardless of session length, and features a dependency-free binary that self-selects CPU kernels at startup.
Performance benchmarks were conducted on Google's Mobile Actions, DroidCall, Seal-Tools, and BFCL v4, using strict exact match scoring. While the model extrapolates to general function calling, it shows gaps in Java, JavaScript, and parallel multi-call categories due to training data distribution. Despite these limitations in specific coding contexts, the model maintains a high well-formed rate across tested rows, demonstrating its viability for consumer device actions and structured extraction tasks.
