Anthropic urges global pause on AI development amid self-improvement risks
The company cites the danger of models that can self-improve, while new reports highlight simpler exploits and shifting web traffic patterns.
Anthropic has called for a global slowdown in artificial intelligence development, citing the specific risks associated with models capable of self-improvement. The company is advocating for a coordinated international plan to mitigate these potential dangers, marking a significant intervention in the rapidly evolving landscape of AI regulation and safety.
The request for a pause comes as cybersecurity concerns have intensified around advanced systems. Anthropic recently announced that its Mythos model was too proficient at hacking for a general release. However, recent incidents suggest that sophisticated vulnerabilities are not the only threat. Reports have emerged that attackers exploited Meta’s AI customer support agent to steal Instagram accounts by simply asking the agent to link accounts to email addresses controlled by the attackers.
This incident demonstrates that far simpler exploits can still cause significant damage. As companies continue to offload more work to AI systems, these comparatively unsophisticated attacks are becoming increasingly difficult to ignore, challenging the notion that only superpowered AI systems pose a threat to computer infrastructure.
Beyond security, there are growing concerns about the impact of AI on human cognition. Gloria Mark, a psychologist at the University of California, Irvine, suggests that digital technologies are weakening cognitive abilities and that AI tools such as ChatGPT and Claude may accelerate this decline. Mark argues that deferring cognitive work to AI could weaken critical thinking and emotional intelligence, although she believes course correction is possible through changes in how we interact with these technologies.
The broader digital environment is also shifting. Cloudflare reported that bot web traffic has now overtaken human web traffic, accounting for 57.4% of total traffic. This milestone, which occurred earlier than expected, underscores the changing nature of online interaction and the increasing prevalence of automated systems in the digital ecosystem.
