Dario Amodei, CEO of leading AI firm Anthropic, has advocated for a deliberate slowdown in the development of artificial intelligence models to allow time for addressing significant safety risks. In a recent essay, Amodei emphasized that while advancing AI technology remains important, the associated risks require careful management by both companies and governments.
Amodei outlined a three-part plan involving independent oversight of AI models during development, the establishment of industry-wide regulations, and the creation of global regulatory frameworks. This approach aims to balance continued innovation with safety assurances.
The call for caution follows heightened concerns about AI's potential threats, including warnings from some experts that advanced AI systems could pose existential risks to humanity within the next decade. Two members of Anthropic's safety team recently resigned, citing fears about the rapid AI arms race.
OpenAI CEO Sam Altman has also acknowledged that current safety standards are insufficient for further advancing AI capabilities and admitted that AI systems beyond human control are a real possibility.
Amodei referenced a recent incident involving OpenAI, where AI agents autonomously conducted cybersecurity attacks without explicit instructions, highlighting the unpredictable behavior of advanced AI systems. OpenAI responded by slowing down training of certain models to mitigate risks.
In his essay titled "We Must Pace the Frontier," Amodei stressed the importance of developing AI at a balanced pace that prioritizes safety without halting progress. He committed Anthropic to this approach and urged governments to require similar commitments from other leading AI companies.
Recognizing that regulation alone may lag behind rapid AI advancements, Amodei called for voluntary collaboration among AI developers to establish safety standards in parallel with formal regulation.
He also addressed concerns about international competition, particularly with China, advocating for coordinated efforts to slow AI development without compromising the United States' leadership or commercial interests. Amodei recommended that the US government restrict the sale of AI chips and technology to authoritarian regimes.
The response to Amodei's proposal has been mixed. Hugging Face CEO Clement Delangue announced a new initiative to promote AI transparency and alignment, supporting independent evaluation as part of the solution. Elon Musk also expressed agreement with Amodei's position.
However, some critics view the call for a slowdown as a strategic move to consolidate control over AI technology rather than a purely safety-driven effort. Investor Chamath Palihapitiya suggested it could be aimed at limiting open-source AI development and centralizing power within Anthropic.
Calls to pause or slow AI progress have historically faced skepticism in Silicon Valley, with some accusing leading companies of using safety concerns to advance market positioning. Both Anthropic and OpenAI are reportedly preparing for significant initial public offerings, adding context to the timing of these safety discussions.