Tech

Tech

Technology, AI, platforms, chips, and the race to define the next digital order.

Stories available
6930
Full section coverage

Latest in Tech

These startups are chasing the next big thing in LLMs
TechDraft

Startups challenge transformer dominance in large language models

A wave of artificial intelligence startups is developing alternative architectures to the transformer model, which currently underpins most large language models. Companies including Subquadratic, Manifest AI, Liquid AI, Inception, and Pathway are introducing innovations such as sparse attention, power retention, liquid neural networks, diffusion-based text generation, and state space models. These technologies aim to address the high computational costs, energy consumption, and context limitations associated with transformers, seeking to create faster, more efficient, and potentially more capable AI systems.