AI experts have claimed that ‘something needs to change’ despite suggesting a warning that robot tech could wipe out humanity in the next few years is ‘sensationalist’.
We spoke to a number of authorities on artificial intelligence after former Anthropic researcher Jacob Coxon, went viral earlier this week.
Coxon resigned from his role in a public statement on X and claimed there is a fear that humanity could be wiped out by 2030.
He said: “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
“The people building AI earnestly believe that it could kill us all by the end of the decade.”
Now, speaking to the Daily Star, former GCHQ security advisor and co-founder of Cyber Security Associates, James Griffiths, said: “I would say that the concerns Coxon raises are broadly credible, although I wouldn’t put a percentage on how accurate his individual claims are without seeing the evidence behind them.
“For me the important takeaway is that this should be treated as a warning that something needs to change.”
He said that regulation was key, not to stop AI development from happening but to make sure it is done safely and with transparency into what safeguards are in place.
“The industry is effectively involved in an arms race to develop increasingly powerful models and tools, so relying entirely on individual companies to police themselves creates obvious questions about whether commercial incentives could take priority over safety,” he said.
Paul Bischoff, Consumer Privacy Advocate at cyber-security publication Comparitech, expressed similar feelings.
“I think some of Coxon’s concerns about super intelligence and AGI are sensationalised, but he does underline the irresponsibility of AI companies as they plough ahead without considering the consequences,” he said.
Several experts said that Coxon’s warning is not far from how the rest of the industry sees itself, as they have begun to prioritise safety concerns after recent troubles.
July’s Hugging Face attack saw OpenAI’s models break out of restrictions designed to restrict them, when 1,200 AI agents started chatting, leading to a mass attack on the AI platform Hugging Face to find the solution to a problem researchers had set out.
Tristan Watkins, Director of Services Innovation for Advania UK, called for a more nuanced debate, saying the most important question was at what point AI companies would divert resources away from developing their systems and invest more in research on safety.
He said: “We are probably getting a little bit hung up on a percentage analysis.
“I think that is helpfully raising the issue of it but probably keeping it at a more agitated and panicky state than is appropriate.
“Ultimately what I was hoping to do, given that people are paying attention now, is direct people to the fact that there is ten plus years of serious concentrated thought about these sorts of problems, and effort to try and avoid them.”
He added that while international cooperation is needed, companies shifting their resources towards safety would be better than what would be a very long wait for global discussion.
Coxon’s statement went viral for its striking claims, as he said that the people building AI believe that it may one day kill us all.
He had spent three years working for both OpenAI and Anthropic and took to X to set out his grievances with where the industry was going.