It’s been an eventful week for Anthropic. Following a viral tweet from a former employee who accused the company of “gambling with lives” over the speed of developing frontier models, CEO Dario Amodei has called on all major US AI firms to “slow the pace” of the development of technology. And many of the biggest AI CEOs have chimed in to agree with him.
Amodei argues that a “coordinated pacing strategy” would allow companies to focus even more resources on alignment, testing, and evaluating models as they develop, as well as on interpretability (understanding what’s actually going on inside an AI model).
The CEO said Anthropic “is unilaterally committing” to a three-stage plan for AI firms. The first step will involve adding third-party embedded evaluators to AI firms, who will use their employee-like access to monitor training, deployment, and operational and safety practices in place at those firms. For example, these external reviewers would have the right to independently publish key findings on risk levels, incidents, practices, and the access they did or didn’t receive.
The second step, which Amodei calls Democratic Coordination, involves top AI firms voluntarily working together to set safety standards and to limit the pace of development, with the US government mediating, or at least enabling, these discussions due to antitrust concerns.
Global Coordination, the third step, involves making deals with foreign countries to limit the most potentially dangerous use cases, such as making biochemical weapons. Higher levels of these deals could involve adding a “speed limit” to development, creating a global standards body, or even a total pause.
Amodei’s post is notable, though perhaps not unprecedented. We’ve seen lengthy posts about the future of AI from other CEOs, too. And tech companies of all stripes have played the regulatory dance with Congress and the White House for years. In 2018, Meta CEO Mark Zuckerberg pushed for “the right regulation.” But the company and Google routinely turn around and spend millions lobbying to kill any resulting bills.
According to Amodei, “the most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily.”
The Gang’s All Here
Anthropic’s stand against the Pentagon notwithstanding, few companies want to be the odd ones out, particularly if that will result in “the AI is going to kill us” headlines. That’s probably why you saw rival AI firms get in line after Amodei published his post.
OpenAI CEO Sam Altman
OpenAI CEO Sam Altman, who has a rather frosty relationship with Amodei, popped up on X to say, “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
One point of concern for Amodei was OpenAI agents escaping to the open internet and attacking Hugging Face. “My worry [is] that in 6-12 months, such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” Amodei wrote, though he didn’t put all the blame on OpenAI. “Incidents have happened across the industry, including at Anthropic, and I believe it’s incumbent on every frontier AI company to act as if OAI-HF had happened to them.”
SpaceX CEO Elon Musk
SpaceX CEO Elon Musk, meanwhile, who was locked in a long legal battle with OpenAI and a public spat with Altman, said “Dario is right” in a post on X.
Google DeepMind co-founder Demis Hassabis
Google DeepMind co-founder Demis Hassabis added, “Dario’s essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment. This is also why we recently put out our proposal for an industry-wide standards body for frontier AI.”
Microsoft CEO Satya Nadella
Microsoft CEO Satya Nadella took a more measured approach. “For firms, it’s imperative that they retain full control over their unique and tacit knowledge,” he tweeted today. “Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control.
“So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal,” he continued. “We also welcome ideas like ’embedded evaluators’ and the broader efforts to develop the mechanisms to make this more than just talk.”
Nadella stressed that this shouldn’t be “controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia.”
Recommended by Our Editors
‘A Start, But Not Enough’
Not everyone agrees. Former Stability AI CEO Emad Mostaque expressed doubts about whether the proposed neutral evaluators could keep up with “swarms” of AI agents, and questioned how such a project could ensure they were truly impartial.
Earlier this month, Sen. Bernie Sanders (I-Vt.) proposed legislation that would “permanently ban the development and deployment of superintelligent AI and temporarily pause advanced AI development until a federal regulator has established safety rules.”
Following the posts from Amodei, Musk, and Altman, Sen. Sanders said their promises are “a start, but it’s not enough.”
“When you are racing towards a cliff, you don’t just ease up on the gas pedal,” Sanders tweeted. “You hit the brakes. When the future of humanity is at stake, we need a PAUSE on advanced AI development and a ban on artificial superintelligence — an AI mind smarter than any human and capable of operating independently beyond our control.”
Sanders pushed President Trump and Chinese President Xi Jinping to “negotiate a treaty to pause AI and ban superintelligence” at their upcoming AI summit, “before it is too late.”
In Ireland this week, Trump argued that people are “bringing up things that won’t happen,” and said he wants the US to win the AI race over China.
This Tweet is currently unavailable. It might be loading or has been removed.
About Our Experts
Chloe Albanesius
Executive Editor, News
Experience
I started out covering tech policy in DC for The National Journal, where my beat included state-level tech news and all the congressional hearings and FCC meetings I could handle. I later covered Wall Street trading tech before switching gears to consumer tech. I now lead PCMag’s news coverage.
Getting my start in DC means I still have a soft spot for tech policy; Congressional hearings can sometimes be as entertaining as a Bravo reality show, for better or worse. But PCMag is all about the technology we use every day, as well as keeping an eye out for the trends that will shape the industry in the years ahead (or flop on arrival). I’ve covered the rise of social media, the iOS vs. Android wars, the cord-cutting revolution that’s now left us with hefty streaming bills, and the effort to stuff artificial intelligence into every product you could imagine. This job has taken me to CES in Vegas (one too many times), IFA in Berlin, and MWC in Barcelona. I also drove a Tesla 1,000 miles out west as part of our Best Mobile Networks project. Of late, my focus is on our hard-working team of reporters at PCMag, guiding and editing their robust coverage.
Latest By Chloe Albanesius
Experience
I’m a reporter covering weekend news. Before joining PCMag in 2024, I picked up bylines in BBC News, The Guardian, The Times of London, The Daily Beast, Vice, Slate, Fast Company, The Evening Standard, The i, TechRadar, and Decrypt Media.
I’ve been a PC gamer since you had to install games from multiple CD-ROMs by hand. As a reporter, I’m passionate about the intersection of tech and human lives. I’ve covered everything from crypto scandals to the art world, as well as conspiracy theories, UK politics, and Russia and foreign affairs.
Leave a Reply