A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.
Evan Hubinger said in a post on X, external the risk from the models which currently exist was “low” but he was “worried” the technology might develop and improve itself soon to the point where it posed an existential risk to humanity.
He did not spell out how he thought AI systems could in future result in humans being wiped out.
But his comments are the latest in a series of increasingly stark warnings about AI, with the debate shifting from whether it truly poses a risk to how big that risk is.
Hubinger’s intervention was in response to another post on X, external from Jacob Coxon, an AI researcher who has just quit Anthropic and previously worked at OpenAI.
“Neither company is acting responsibly,” he wrote.
“These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources.”
OpenAI has been approached for comment.
Dame Wendy Hall, a computer scientist who advises the UN on AI, told the BBC she was “shocked” by Hubinger and Coxon’s social media posts.
She told the World at One on BBC Radio Four some of it could be “PR and marketing” however, as Anthropic and OpenAI raced towards highly anticipated stock market debuts.
But she added: “Why would someone want to say that? I would plead with investors not to invest in this company if that is their value system.”
The resignation prompted Darren Jones, former chief secretary to the Treasury and chief secretary to Sir Keir Starmer, to write an open letter to Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI.
He told the BBC governments had to come together to “collaborate” about what a treaty based on the development of superintelligence should look like.
“Unless governments take these warnings seriously enough and step up to it, the pace of development could mean that we end up with problems before we’ve started to look at whether it is an issue for us or not,” he said.
Separately, the Financial Times reported, external Anthropic withheld its latest model from the UK’s AI Security Institute (AISI), one of the leading bodies in the world for assessing AI risk.
Anthropic has declined to comment on the posts by its employees or the situation with the AISI.
A Cabinet Office spokesperson did not comment on whether the latest model had been withheld from the AISI – instead saying it “continues to collaborate closely with industry partners, including Anthropic, to make models safer”.
