Lead: AI safety debate erupts as leading researchers and CEOs call for slowdown
A researcher’s dramatic resignation from Anthropic has triggered an unprecedented alignment across rival frontier labs. Jacob Coxon’s viral post warning that OpenAI and Anthropic are “racing straight to self-improving superintelligence and gambling with our lives” has now been echoed by senior researchers at both companies, who put the risk of human extinction in the next few years as high as 70% without regulation or coordinated slowdown.
Anthropic CEO Dario Amodei published a major essay calling for deliberate pacing of frontier model development, third-party safety evaluators inside labs, and eventual government-mediated coordination. Sam Altman (OpenAI) and Elon Musk publicly agreed. Altman also ruled out a 2026 OpenAI IPO citing safety concerns. The episode marks the clearest signal yet that even the builders of the most powerful models believe the current pace is unsustainable.
What else matters
- OpenAI backs US legislation to prevent AI accelerating biological weapons
OpenAI has thrown its weight behind bipartisan bills in Congress aimed at stopping advanced AI models from helping develop biological weapons. The move comes as the company and Anthropic have both reported blocking attempts to use their models for bioweapons research.
- Two clear camps emerge in the AI safety debate
President Trump, Nvidia CEO Jensen Huang, and David Sacks argue that existing laws are sufficient and that additional antitrust waivers or slowdown agreements are unnecessary. In the other camp: frontier lab leaders (Amodei, Altman, Musk) and a growing number of AI researchers calling for concrete safety measures and pacing.
- Sam Altman: “The world is right to be afraid” but should trust AI firms
In his first major public comments since the Coxon resignation, OpenAI’s CEO acknowledged public fear is justified given how quickly capabilities are advancing, but urged greater trust in the labs’ internal safety efforts. Critics, including Bernie Sanders and Steve Bannon, say self-regulation is no longer credible.
- Nvidia’s Jensen Huang rejects Anthropic’s call for antitrust waiver on slowdown coordination
Huang called proposals for legal waivers so that labs can coordinate on slowing frontier development “completely unnecessary,” arguing existing regulations already cover product safety and reliability. He believes the industry can manage safety without new coordination frameworks.
Worth your time
Dario Amodei – “We Must Pace the Frontier” (full essay – the clearest articulation yet of why leading labs believe a deliberate slowdown may now be required).
