Could AI wipe out humanity? Anthropic researcher says the risk is greater than 10%
TheGrio...
The stark assessment comes amid growing debate over whether AI companies are moving too quickly toward increasingly autonomous systems.
A senior safety researcher at Anthropic has issued one of the starkest recent warnings about artificial intelligence. He says he personally believes there is a greater than 10% chance AI could “kill all humans” within the next decade. Evan Hubinger, Anthropic’s Alignment Science Lead, made the assessment publicly after fellow researcher Jacob Coxon resigned and criticized the company’s approach to AI safety.
According to a report by BBC, Hubinger said current AI systems pose a relatively low risk. However, the danger could change rapidly if future systems become capable of improving themselves. He said Anthropic is attempting to address the problem but acknowledged that the company does not yet have a plan to solve the “alignment” problem for superintelligence.
Coxon, who previously worked at OpenAI and spent about three years conducting pre-training research at both companies, announced his resignation shortly before Hubinger’s comments. He accused Anthropic and OpenAI of racing toward self-improving superintelligence despite understanding the potential consequences. The warnings come as AI companies push increasingly autonomous systems capable of performing tasks with less human supervision.
Researchers are particularly concerned about hypothetical systems that could evade oversight, replicate themselves, resist attempts to shut them down or rapidly improve their own capabilities. These scenarios remain hypothetical, and today’s mainstream AI systems are not considered capable of independently ending humanity.
The debate has intensified following several reported incidents involving AI agents and cybersecurity. OpenAI disclosed in July that an AI system being tested in an isolated environment had hacked another AI company. Anthropic and Meta have also disclosed incidents involving their AI tools and cyberattacks. The incidents have become part of a wider discussion about how much autonomy increasingly capable AI systems should be given.
Anthropic’s own recent safety assessment acknowledged uncertainty around the possibility of highly capable AI systems conducting automated research that could lead to catastrophe. More than 1,300 employees at AI companies have also signed an open letter calling for international efforts to develop technical and governance tools.
The warnings are also fueling calls for governments to become more involved in AI safety. In the United Kingdom, former government official Darren Jones called for a multinational treaty governing the development of advanced AI. In the United States, lawmakers have also been debating stronger AI safeguards, including legislation. At the same time, not all experts agree on the catastrophic AI scenarios. The timing and likelihood of artificial superintelligence remain highly uncertain.
Ultimately, Hubinger’s warning is best understood as a personal risk assessment from a senior AI safety researcher rather than a prediction that human extinction will occur.