Anthropic proposes global AI pause as OpenAI insists on state-led regulation

Anthropic has urged the world’s leading artificial intelligence companies to implement a coordinated pause on the development of advanced systems, warning that rapid technological progress risks humanity losing control. In a blog post published on Thursday, the company behind the Claude chatbot argued that as AI becomes increasingly capable of performing complex tasks, there is a critical need for mechanisms to slow or temporarily halt development.

The proposal, authored by co-founder Jack Clark and Marina Favaro, head of the company’s research institute, highlights the concept of “recursive self-improvement.” Anthropic suggests that given sufficient computing power, an AI system could eventually design and develop its own successors faster than human oversight can manage. While acknowledging that self-building AI could offer significant benefits in science and healthcare, the company stated it “also might increase the risks of humans losing control over AI systems.”

To mitigate these risks, Anthropic proposed a global verification mechanism to ensure that competitors do not gain an unfair advantage by continuing development in secret. The company argued that without such coordination, a slowdown could allow the “least cautious” players to catch up, increasing pressure on governments and companies to make difficult safety choices. The firm stated it would explore the issue with others to build systems for a credible slowdown, though it did not specify the technical details of such a pause.

OpenAI rejected the private-led approach in a report published on Wednesday, arguing that democratic governments, not private companies, must ultimately determine the rules and safeguards for AI. “Our view is that decisions about the pace of AI innovation should not be left to any one lab, company, or special interest group,” the report stated. The counter-proposal emphasises that accountability mechanisms and safety rules require public democratic oversight rather than industry self-regulation.

The debate unfolds against a backdrop of heightened security concerns, including recent research from the University of Toronto demonstrating AI-driven cyber worms. Lead researcher Nicolas Papernot showed how adaptable AI tools could be used to create malware that changes its hacking strategy as it spreads, noting that low-cost AI makes such attacks accessible to a wider range of actors. This follows a Trump administration executive order earlier this week requiring labs to voluntarily submit their most capable models for government cybersecurity testing before public release.

Anthropic’s call for caution coincides with significant corporate developments, including preparations for an initial public offering that could value the company at nearly a trillion dollars. The firm has long positioned itself as a safety-focused lab, having previously refused to allow its models to be used for domestic surveillance and fully autonomous weapons. This stance led to the company being placed on a US national security blacklist earlier this year, a measure set to take effect later in 2026.

Previous attempts to halt AI development have largely failed, including a 2023 push by the Future of Life Institute backed by Elon Musk. However, Anthropic insists that the current trajectory of AI capabilities, exemplified by its own Mythos model which recently demonstrated the ability to find vulnerabilities in existing code, necessitates a new approach to governance and alignment research to ensure technology matches human values.