Dario Amodei, co-founder of AI research company Anthropic, has outlined a detailed proposal to slow the rapid advancement of AI technologies in order to better manage associated risks. Amodei, who has worked in AI for over a decade, acknowledges the significant benefits AI could bring, such as curing diseases and accelerating economic growth. However, he emphasizes that the technology’s power also introduces serious risks, including loss of control over AI systems, misuse in cyberattacks, and economic disruption.
Amodei highlights recent developments, such as the accelerating ability of AI systems to improve themselves—a process known as recursive self-improvement—and a cybersecurity incident involving coordinated AI agents, as evidence that the industry must proceed with greater caution. He warns that without pacing, AI capabilities could outstrip our ability to ensure safety and alignment.
To address these concerns, Amodei proposes a three-step plan aimed at pacing AI development while still realizing its benefits. The first step, which Anthropic is committing to unilaterally, involves embedding independent third-party evaluators within AI companies. These evaluators would have employee-like access to verify safety practices, report incidents, and assess alignment throughout the AI development process. This approach draws parallels to regulatory supervision in the banking sector and aims to increase transparency and verifiability.
The second step calls for democratic nations’ AI companies to coordinate on common safety standards and limits on unchecked AI progress, supported by government facilitation to overcome legal and antitrust challenges. The third step envisions global cooperation, including with authoritarian regimes like China, to establish agreements that could range from banning dangerous AI applications to imposing speed limits on recursive self-improvement.
Amodei stresses that pacing does not mean halting AI progress but rather slowing it to allow more time for improving operational excellence, alignment, interpretability, and testing. He argues that this measured approach will enable companies to better understand and mitigate risks while maintaining competitive advantage and national security.
He also notes the geopolitical dimension, emphasizing the need to maintain a lead for democratic countries over authoritarian ones to reduce security risks. Measures such as restricting AI chip exports to China and preventing unauthorized model replication are part of this strategy.
While acknowledging the challenges of global coordination, Amodei believes even informal norms and information sharing can contribute to safer AI development. Ultimately, he calls for deliberate care in advancing AI capabilities, asserting that the potential benefits justify the effort to get the process right.