A new resignation from inside companies developing the world’s most advanced artificial intelligence systems has once again put the AI safety debate at the center of public attention.
Twenty-seven-year-old British researcher Jacob Coxon, who worked on training advanced artificial intelligence models at OpenAI and Anthropic over the past three years, announced that he had left his position at Anthropic and the AI sector altogether.
Coxon’s reason is far more striking than an ordinary career change.
The researcher says he is concerned that the race being pursued by leading AI companies to develop systems capable of improving themselves could produce consequences beyond humanity’s control.
Coxon accused OpenAI and Anthropic of acting irresponsibly, arguing that the two companies were racing toward self-improving superintelligence and “gambling with our lives” in the process.
FROM OPENAI TO ANTHROPIC
Jacob Coxon worked in the AI sector primarily on the model pretraining process.
This is one of the most important stages of AI development, during which large language models acquire their core capabilities using enormous amounts of data.
According to his own account, Coxon conducted research in this field at both OpenAI and Anthropic over the past three years.
He left OpenAI and joined Anthropic at the beginning of 2026. Coxon said Anthropic’s emphasis on AI safety influenced his decision to choose the company.
However, he said his concerns had not disappeared after his time at Anthropic.
According to Coxon, people at Anthropic understand the seriousness of AI risks, but the company has difficulty unilaterally slowing its work on advanced systems because of competition with other laboratories.
“IT COULD SPIRAL OUT OF CONTROL BY THE END OF 2027”
One of Coxon’s most striking statements concerned the timeline for AI development.
Speaking to The Wall Street Journal, Coxon said he was concerned that, if the most aggressive scenarios materialized, some developments could already be out of control by the end of 2027.
Coxon’s main concern is that AI systems could become capable enough to contribute to their own development.
In such a scenario, an advanced AI system could do more than simply carry out tasks assigned by humans; it could also make a significant contribution to developing new AI systems.
In theory, this could accelerate AI development and make it difficult for people to keep pace with the growth in the systems’ capabilities.
“SELF-IMPROVING SUPERINTELLIGENCE”
One of the concepts at the center of the debate is “self-improving superintelligence,” or superintelligence capable of improving itself.
Systems such as today’s ChatGPT or Claude are not independent superintelligences that directly improve themselves without limit and have escaped human control.
Coxon’s warning concerns far more powerful systems that could emerge in the future.
The researcher is concerned that future AI systems could reach extremely high levels of capability in areas such as software development, cybersecurity, scientific research and the development of new AI models.
However, it is important to emphasize that these are not events that have occurred today, but risk scenarios concerning systems that could emerge in the future.
“PEOPLE DEVELOPING AI ARE TAKING THE RISK SERIOUSLY”
One of the most striking claims in Coxon’s statements concerned the atmosphere inside AI companies.
The researcher said some people developing AI genuinely believe that advanced AI could pose a risk capable of producing catastrophic consequences for humanity by the end of the decade.
According to Coxon, company executives and senior researchers use more cautious language in public statements, while risks are discussed much more starkly in private conversations within the companies.
Other statements from within Anthropic after Coxon’s remarks further expanded the debate.
STRIKING STATEMENT FROM ANTHROPIC’S ALIGNMENT LEAD
Evan Hubinger, Anthropic’s Alignment Science Lead, responded to Coxon’s remarks and shared his own risk assessment with the public.
Hubinger said he personally assigns a probability of more than 10 percent to a scenario in which advanced AI could lead to humanity’s extinction within the next decade.
Hubinger also drew attention to a fundamental problem concerning superintelligence that Anthropic has not yet solved.
According to the researcher, the company is working to solve the “alignment” problem of ensuring that superintelligence remains safely aligned with human goals, but it does not yet have a definitive solution.
Alignment refers to keeping an AI system’s goals and behavior consistent with human objectives, values and safety limits.
Hubinger also said the risk posed by current models was low, emphasizing that his main concern was much more powerful future systems capable of accelerating their own development.
CONCERNS ARE NOT LIMITED TO ANTHROPIC
Concerns about the pace of the AI race are not coming only from within Anthropic.
Senior figures at OpenAI have also recently begun issuing stronger warnings about the risks posed by advanced AI systems.
OpenAI Chief Scientist Jakub Pachocki called for “extreme caution” over the consequences of rapid machine intelligence development and urged coordination between companies and governments.
OpenAI CEO Sam Altman also said rapid action was needed against the cybersecurity risks that advanced AI systems could create.
As a result, calls from within the companies developing the world’s most powerful AI systems for more careful management of the technology’s pace of development are growing stronger.
THE REAL PROBLEM: CAN COMPANIES SLOW DOWN?
According to Jacob Coxon, the problem is not limited to the safety policies of OpenAI or Anthropic.
The central problem is competition between the companies.
The belief that a company could be overtaken by its rivals if it slows advanced AI work for safety reasons encourages companies to continue the development race.
For this reason, Coxon believes the problem cannot be solved solely through the decision of one company.
The researcher argues that options such as agreements among AI laboratories to limit the pace of development, government intervention and, when necessary, temporarily suspending the development of advanced model capabilities should be discussed.
A NEW ERA IN THE AI RACE?
Jacob Coxon’s resignation alone does not prove that AI is about to spiral out of control.
What makes the episode notable, however, is that the warning comes not from a critic observing the technology from the outside, but from a researcher who worked directly on advanced models at leading AI laboratories around the world.
Moreover, the public sharing of some of Coxon’s concerns by senior safety researchers at Anthropic is taking the AI safety debate in a different direction.
For now, there is no indication that ChatGPT, Claude and other major AI systems are superintelligences that have seized control on their own.
The debate centers on whether people can develop the technical and legal mechanisms needed to keep the next generation of systems under control in time if they develop far more rapidly than today’s systems.
Following Coxon’s resignation, this is precisely the question facing the technology world:
Will AI companies be willing to slow down in the race toward superintelligence, or will competitive pressure drive the technology forward faster than safety measures can keep up?
