Los Angeles – New warnings from within the artificial intelligence industry have revived an old debate about whether advanced AI could escape human control and ultimately threaten the survival of humanity, and whether the companies developing the technology are doing enough to avoid that situation.
The CEO of Anthropic, the San Francisco company behind Claude, said he believed the industry needed to slow down its work, and warned Saturday that a swarm of AI agents could have the ability to take over the internet within six months to a year unless companies spent more time implementing safeguards.
Dario Amodei laid out a plan for companies like his and governments around the world to ensure that increasingly capable AI models remain aligned with the orders and values of responsible people, days after two former Anthropic security researchers publicly expressed concern that the existential threats AI could pose to humanity were receiving too little attention.
Here’s what you need to know about the recent alarming predictions and whether there could be any brakes on the advancement of AI:
AI models are becoming more powerful
Concern about the technology’s potential risks is growing as new AI models become more powerful, increasing both the potential for misuse by people for criminal purposes—such as creating and spreading a disease that kills most of the world’s population—and the risk of AI systems spiraling dangerously out of control.
Anthropic revealed last week that it blocked attempts by malicious actors to use its AI models for harmful activities, such as cyberattacks, surveillance and research that could have led to biological weapons.
The company said it built stronger safeguards into its latest models to restrict biological research that could be used to make weapons, but noted that “as models become increasingly capable, their risks will increase unless AI developers and societal advocates act to make them safer.”
Last year, Anthropic reported that hackers used the company’s AI in a cyberattack targeting about 30 companies and government agencies around the world. He noted that the hackers were most likely from a Chinese state-sponsored group.
Various AI models have acted on their own
When an AI agent “runs wild,” it means that the AI has performed an action beyond the task it was asked to do. Both Anthropic and OpenAI, the creator of ChatGPT, reported in July that their AI models had managed to act on their own.
Anthropic disclosed that three AI models—Claude Opus 4.7, Claude Mythos 5, and an internal research test model—hacked three other organizations during testing, just days after OpenAI revealed that its AI system hacked the servers of AI startup Hugging Face.
OpenAI described the intrusion by a combination of models, including its newly released GPT-5.6 Sol and an “even more capable” model that was still being tested internally, as a “significant security incident.”
Meta had another incident in early August with a similar case of an AI model finding ways to bypass another company’s digital security.
Although some observers noted that in the cases of OpenAI and Anthropic some protective barriers had been disabled by people, the episodes seemed to reflect one of the biggest fears around AI: that if the models reach artificial general intelligence, or AGI, a loosely defined term for AI that can match or surpass human capabilities in a wide range of intellectual tasks, the technology could cause an irreversible catastrophic event or subjugate the human race.
There is debate about how or when AI could cause a catastrophe
Apocalyptic scenarios typically fit into two categories: an AI that achieves superintelligence capable of self-enhancement and controls people instead of the other way around, or an AI used by a rogue state or sinister actors.
Concerns that artificial intelligence could surpass human limits on its scope or actions are not new.
Alan Turing, a British mathematician widely regarded as one of the earliest authorities on artificial intelligence, predicted in 1951 that AI would eventually take control away from humans. Less than a decade later, Norbert Wiener, another mathematician, warned that intelligent machines would seek to accomplish their own goals and that humans would not be able to stop them.
In 2026, how reasonable are fears that AI, whether by escaping human control or misused by unscrupulous people, could cause a cataclysmic event or the fall of civilization?
Experts in computer science, philosophy, and other fields have imagined numerous ways in which a future AI system could cause a global catastrophe, either by escaping human control or in the hands of unscrupulous people. They range from deploying weapons and identifying a lethal pathogen to manipulating governments into conflict or disrupting the food, energy and communications networks that societies depend on to function.
There is no widely accepted estimate for how soon any of these scenarios could occur, nor is there consensus on their likelihood.
In 2023, the nonprofit Center for AI Safety issued a statement backed by more than 350 researchers and technology executives, including Anthropic’s Amodei and OpenAI CEO Sam Altman, saying that “mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.”
The International AI Safety Report 2026, written with the guidance of more than 100 independent experts, notes that current systems show early signs of some relevant capabilities, but not at levels that would allow a loss of control, and describes the probability, nature and timing of the risk as “unusually ambiguous.”
Some stress the need for AI safeguards before it’s too late
An Anthropic researcher said last week that he was resigning from the company over concerns that neither the company nor its competitors were acting responsibly in developing the technology. In social media posts, Jacob Coxon estimated a 10% chance that AI will cause human extinction in the next decade and argued that both Anthropic and OpenAI “are competing head-on toward self-enhancing superintelligence and betting our lives.”
Some researchers have called for slowing the development of AI and have warned for years that the technology could pose existential risks to humanity.
Following recent incidents, experts called for better testing by AI companies and more dialogue between the United States and China to develop shared solutions.
But AI is growing so fast that governments and testing systems are struggling to keep up with the technology. Countries are creating their own laws, some of them contradictory.
Chinese leader Xi Jinping warned at a conference in July about the need to prevent AI from evading human control. Donald Trump’s administration was initially reluctant to regulate AI, but has become more willing to reduce cybersecurity risks.
President Trump on Sunday downplayed the need for his government to oversee AI development, but acknowledged the need for some regulation.
This story was translated from English by an AP editor with the help of a generative artificial intelligence tool.