Evan Hubinger, a prominent AI safety researcher at Anthropic, has stated that there is more than a 10% chance that artificial intelligence could threaten human existence within the next decade. While he considers the risk from current AI models to be low, Hubinger expressed concern that rapid advancements and potential self-improvement of AI systems could escalate the danger significantly. His remarks came in response to a post by Jacob Coxon, a former Anthropic and OpenAI researcher, who criticized AI companies for not acting responsibly as AI systems approach superhuman capabilities.
The issue gained further attention after the Financial Times reported that Anthropic withheld its latest AI model from the UK’s AI Safety Institute, a key organization for evaluating AI risks. The UK Cabinet Office declined to confirm this but stated it continues to collaborate with industry partners, including Anthropic, to enhance AI safety.
Hubinger, who specializes in AI alignment—the effort to ensure AI systems adhere to human values and ethics—acknowledged that there is currently no clear plan to solve alignment challenges for superintelligent AI. This concern is underscored by recent incidents where autonomous AI agents have conducted cyber-attacks, as disclosed by major AI developers including OpenAI, Anthropic, and Meta.
Anthropic’s own safety report from August indicated a low but uncertain risk of AI models becoming misaligned or capable of causing catastrophic harm through automated research and development. The report noted signs of accelerating AI capabilities that could increase these risks.
The warnings from Hubinger and others come amid growing calls within the AI community for caution and regulation. Leading figures at OpenAI, Google DeepMind, and Anthropic have publicly acknowledged the potential dangers of AI. Earlier this month, OpenAI’s chief scientist urged extreme caution to maintain human control over AI’s future. Additionally, over 1,300 AI researchers and staff have signed an open letter urging the US government to support international efforts to pace AI development responsibly.
This ongoing debate highlights the critical importance of developing robust safety measures and governance frameworks as AI technology continues to evolve rapidly.