Anthropic calls for global AI pause as it eyes public listing

Anthropic has proposed a temporary global pause in artificial intelligence development, urging the industry to halt progress until societal structures and alignment research can adequately manage the technology. The proposal stems from concerns that AI systems are advancing rapidly and may soon become capable of building their own successors, a capability the company warns could lead to humans losing control.

The suggestion is grounded in research from the Anthropic Institute, a division established in March to study the challenges associated with advanced AI. The institute’s findings indicate that any meaningful slowdown would require multiple well-resourced laboratories across different countries to agree to stop development under identical conditions. Crucially, the proposal demands that each participant verify that others have complied with the halt, citing the difficulty of ensuring no entity gains an advantage by developing technology in secret.

Anthropic acknowledges the complexity of such an agreement, noting that verifying compliance would be challenging. The company pointed to nuclear-weapons treaties as a precedent, admitting that while those agreements took decades to establish, the current pace of AI development does not allow for such a prolonged timeline. The firm stated it is not impossible to achieve a global pause, provided all major players pledge to stop development simultaneously.

The timing of the proposal coincides with significant financial developments for the company. Anthropic is reportedly on track to report its first profitable quarter and has filed paperwork with the US Securities and Exchange Commission to go public, a move likely to occur before the end of the year. This financial milestone distinguishes it from several competitors that have yet to generate profit.

Critics have dismissed the proposal as a marketing ploy, particularly in light of the limited release of Anthropic’s cybersecurity model, Mythos. The company released the model to a select group of partners, citing the potential damage its ability to identify vulnerabilities could cause if misused. However, some observers argue the restriction was designed to hype the product or ensure it is sold only to large enterprises.

Anthropic plans to engage in talks with policymakers, researchers, and other industry players over the coming months to discuss the feasibility of a slowdown. The company intends to publish the results of these conversations, aiming to foster a coordinated approach to managing the risks of self-improving AI systems.