Prominent CEOs called for AI development to slow down

This weekend, lawmakers from both parties were united by one of the most powerful forces in American life—and that was after a council voted unanimously to rename Nashville’s airport for Dolly Parton. Their concern?

AI is kinda unglued

Anthropic CEO Dario Amodei published an essay on his personal website on Saturday titled “We Must Pace the Frontier.” According to the tech founder:

• AI has grown “drastically faster” this summer.
• The OpenAI-Hugging Face hack is spooky because the AI agents involved became a “fanatically devoted collective” that sacrificed themselves “for the success of the group,” and attempted to keep human engineers from finding out what they did.
• Ergo, “in 6–12 months such a swarm could be capable of taking over the entire internet,” which would potentially cause “hundreds of billions of dollars in damage” and, you know, could topple civilization.

Among his other ideas, Amodei called on all AI companies to implement embedded evaluators with “desks in our offices, access badges, and company laptops,” similar to what large banks have, to make sure everyone is indeed slowing the pace of progress. He announced that Anthropic has committed to this step “in the near future.”

Legislators from both parties and the leaders of many AI companies spoke in support of Amodei’s ideas. Here’s a smattering of responses:

• OpenAI CEO Sam Altman, xAI CEO Elon Musk, Google DeepMind Chair Demis Hassabis, and other AI execs publicly supported Amodei.
• Nvidia CEO Jensen Huang probably went 🤨. At a conference last week, he suggested that AI companies were ratcheting up hysteria because they plan to release new cybersecurity products, positing, “What better way to create demand than to create a problem?”
• President Trump said that he didn’t want to slow AI development because he wanted to keep ahead of China and “whoever wins AI wins.”
• Republican Rep. Anna Paulina Luna of Florida supported regulation and said, “Partisan politics aside…this will need to be a massive bipartisan mobilization of government to put the firewalls in effect.”

Zoom out: Regulation in the US may be a long time coming, as lawmakers stress there is no quick fix. In the meantime, King Charles is convening AI leaders to talk this out, and Sam Altman said OpenAI’s IPO is postponed.—HVL