Former Anthropic researcher warns rapid AI progress could threaten humanity

Former Anthropic researcher Jacob Coxon has warned that rapid advances in artificial intelligence could pose a serious threat to humanity, telling the BBC that people working in the field were “genuinely frightened” by the pace of progress.

Coxon said there was a strong chance of severe consequences, potentially including human extinction, if development continued at its current rate. He said some AI workers feared danger could arrive within two years, while a swarm of AI bots capable of taking over the internet could become realistic within six months to a year.

His comments followed Anthropic chief Dario Amodei’s call for AI development to slow. Amodei has described the risks as serious while supporting continued technological progress, and has called for regulation and independent monitoring of AI model development.

OpenAI chief Sam Altman and xAI chief Elon Musk have reportedly supported industry-wide deceleration, regulation and independent monitoring. Coxon said any slowdown would need to be co-ordinated with China to avoid an international race.

Anthropic said it builds models with strong safeguards, tests them aggressively and publishes findings to support scrutiny and prevent AI misalignment. The company also said the industry would benefit from lawful, verifiable cooperation to pace the release of powerful models.

Warnings about existential risks have drawn mixed responses. Anthropic scientist Evan Hubinger and computer scientist Geoffrey Hinton have expressed concern about the possibility of AI killing humans, while Faculty chief executive Marc Warner said assigning a probability was extremely difficult. Hugging Face chief executive Clement Delangue questioned the framing of extinction warnings, and Nvidia chief Jensen Huang was reported by conference attendees to have dismissed Coxon’s comments.