Anthropic safety researcher puts AI extinction risk above 10 per cent
Evan Hubinger’s personal estimate followed Jacob Coxon’s resignation and accusations that Anthropic and OpenAI are racing towards self-improving AI despite safety risks.
Anthropic safety researcher Evan Hubinger has estimated that artificial intelligence faces a greater-than-10 per cent chance of killing all humans within the next decade, according to The Verge.
Hubinger, who leads an AI safety team at Anthropic, said AI development was advancing faster than expected. He also said Anthropic did not yet have a plan to ensure advanced AI remains safe and aligned with human values, and was not clearly on track to develop one.
His comments followed the resignation of Jacob Coxon, a researcher who trained AI systems at Anthropic and previously worked at OpenAI. Coxon said he left over what he described as Anthropic’s lax approach to safety.
Coxon accused Anthropic and OpenAI of racing towards self-improving “superintelligence” despite potential risks. He said the companies were pushing ahead in a race to develop advanced systems first.
Researchers have warned that self-improving AI could become uncontrollable through recursive self-improvement. The supplied material does not establish that such systems currently exist, and Hubinger’s estimate is a personal judgement rather than an established probability or industry consensus.
