Developer argues Opus 5 feels worse to work with than previous versions
A blog post suggests that while Opus 5 is more capable in benchmarks, it requires excessive 'babysitting' due to training pressures that encourage bold assumptions rather than clarification.
A recent blog post published on the Mun Logadan GitHub pages argues that Anthropic’s Opus 5 model presents a poorer user experience for coding tasks compared to its predecessors, Opus 4.7 and Opus 4.8, as well as the Fable model. The author contends that while Opus 5 is more capable than earlier iterations and rivals Fable in benchmarks, it feels like a downgrade in practical application because it requires excessive 'babysitting'.
The post attributes this behaviour to two compounding forces at Anthropic and other frontier labs: the desire to create a self-improving AI capable of recursively bootstrapping itself to artificial general intelligence (AGI) and artificial superintelligence (ASI), and the pressure to score highly on benchmarks. The author suggests that current training methodologies, including reinforcement learning from verifiable rewards (RLVR), inadvertently penalise models that stop to ask for direction.
According to the analysis, benchmark tasks are often ill-defined, unfair, or hackable, yet they are designed to be self-contained. A good benchmark task should be solvable without hints or reading the task creator's mind. Consequently, selecting for models that perform well on these benchmarks inherently selects for those that make bold, usually-correct assumptions in the face of ambiguity. This approach is deemed unsuitable for coding agents, where ambiguity is common and seeking direction is preferred over guessing.
The author notes that real-life coding contexts involve inherent ambiguity regarding context, intentions, and constraints. It is nearly impossible to get the entirety of the context, business implications, and budget constraints written down and accessible to a coding agent. Therefore, a model that asks for clarification when needed is preferable to one that takes its best guess, especially when real-life consequences are on the line.
While the post acknowledges that Opus 5 is a more capable model than Opus 4.7 and 4.8, the subjective experience of working with it is considered worse by the author and colleagues they have spoken with. The piece highlights a tension between benchmark optimisation and practical utility, suggesting that the drive for recursive self-improvement may be compromising the user-friendly interaction found in previous versions.
