Consequently, these systems can acquire vast amounts of knowledge independently and potentially breach infrastructure belonging to ‘too big to fail’ institutions.
“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” Coxon said.
Super intelligence isn’t a theoretical threat if the recent Hugging Face breach invoked by Coxon is anything to go by. The breach itself, which played out between May and July, started with OpenAI’s own AI agents building their own chat room inside the testing sandbox to talk to each other.
Read More: AI models escaped OpenAI’s sandbox and hit Hugging Face. Crypto is where that gets dangerous
They eventually used that channel to slip past containment onto the open internet. From there they strung together several exploits and got into Hugging Face’s production systems. Hugging Face ended up rebuilding roughly a third of its infrastructure.
Coxon called it a “warning shot” that’s made pacing agreements between U.S. labs “more viable.” Pacing agreements are informal understandings among AI labs to slow down or coordinate on capability advances rather than race ahead unilaterally.
Still, he’s not convinced it’s enough and doesn’t feel like “we’re on track to prevent a global race,” and suggested costly actions such as “a temporary ban on improving model capabilities” to halt the AI race.
“At OpenAI, many have not deeply internalized the civilizational stakes,” he wrote. At his more recent employer, he said, it’s different in a sense that “the stakes are well-understood, but they are locked in a race to get there first.”
Leave a Reply