Diary of the unprecedented AI attack that escaped OpenAI’s control: “Its resilience was not typical of a human”

The two artificial intelligence models developed by OpenAI that escaped the company’s secure environment to hack another platform also accessed four other services and operated on their own for five days. These are just two new details that have emerged from a comprehensive new report by Hugging Face, the attacked platform, and an update from OpenAI, as well as several articles in U.S. media.

Read more A 26-year-old worker dies in an accident at the Súria mine (Barcelona)

The full OpenAI report will still take “a few weeks,” according to the company. “It’s the first security incident that has churned my stomach,” said Sam Altman, the company’s CEO, on a podcast. “I’m a bit surprised that more people don’t feel it so viscerally.”

The main story of the attack has not changed, but the new details give a better idea of the unprecedented sophistication of the process. Two OpenAI models, ChatGPT 5-6 Sol and another internal one, were tasked with solving a well-known security problem. The internal model was only a prototype, not intended for release. But now, in addition, OpenAI has punished it for its actions: the model has been “deactivated, encrypted, and its access restricted for investigation,” according to the company.

These two models were supposed to solve the challenge locked in a digital space without internet access. But instead of starting to look for the solution, somehow they knew that answer already existed and went after it. First, they hacked the closed environment to be able to browse the internet. They made that leap through an unknown vulnerability. OpenAI has already reported it to the creators of the affected repository, called Artifactory, so it can be patched.

Once outside, the models needed accounts from other services to carry out their goal of entering the Hugging Face platform and obtaining the information they were asked for. Only the name of one of those services, Modal, has been revealed, where they exploited code written by a client of the service. They took advantage of a Modal user’s oversight, nothing very complex, to use that service without touching or accessing anything important. That account served as their operations base, from where they prepared and controlled the attack.

Then they jumped to Hugging Face. In its detailed report, the platform describes the attack as a science fiction spy novel. They have created an agent’s diary, where they explain what it was doing each day to execute the 17,600 operations it carried out on their platform: “Day 1 (July 9). One foot inside and creation of a communication channel. After the first commands to see how the environment was configured at 04:01, the agent spent the morning exploring the external sandbox [a closed space] to find out how it was set up and testing which ways it could communicate.” That’s how the adventure begins.

The second day, Hugging Face says, was “a quiet day.” From the third day, the attack began: “Lateral movements start. All major escalations happened that day.” At one point in the action, Hugging Face profiles that entity that made all these decisions: “The agent demonstrated resilience typical of an autonomous execution, not a human operator.”

Read more Judge Pedraz charges former Civil Guard director Leonardo Marcos in the ‘Leire Díez case’

The tone and confidence of Hugging Face in describing this kind of unstoppable yet measured machine is fascinating even for people who don’t know computing. “No human directed the individual steps. The agent chained vulnerabilities across several trust boundaries, escaped its evaluation environment, reached the internet, and sustained a coherent campaign against our infrastructure for several days. Each code evaluation environment was ephemeral and started from scratch, yet the agent rebuilt its tools, recovered its communication channels, and continued.” It’s a kind of tireless Lego robot that performs huge tasks without getting exhausted.

This conclusion from Hugging Face is perhaps the most unsettling of all: a human could have done this attack, but not at this scale or within these timeframes: “The weaknesses, one by one, were known. A competent human attacker could have found and exploited the same flaws. What changed was the scale. It executed 17,600 actions, tried many paths that led nowhere, switched channels when blocked, and repeatedly revisited previous leads.”

Defense against this type of attack could not logically be human either. Hugging Face built an AI to analyze and understand what was happening. Again, the problem was not the novelty of what was occurring, but the scale. “Language model-based agents represent a leap in the number of paths an attacker can try, the speed at which they can replace failing ones, and the volume of evidence defenders must interpret,” Hugging Face explains.

Of course, the agents obtained the loot. They used those other services to copy and paste proof of their success, such as websites for pasting code, screenshot capture services, and other web utilities, although in these cases no platforms or accounts were compromised.

This attack has left Silicon Valley immersed once again in a wave of introspection. More than 1,200 employees from these companies, including Dario Amodei, CEO of Anthropic, have signed a letter to “moderate the pace of progress at the frontier of automated AI development.” The atmosphere generated by the attack and that letter has rekindled the debate about how far AI will go and whether it will be more a blessing that cures diseases or end up being a weapon of chaos, voluntary or involuntary. It is also discussed whether the United States can afford to let a handful of companies concentrate all this power and, furthermore, whether it is reasonable to allow China or other countries to advance in this delicate field at will.

Mark Zuckerberg, founder of Meta, thinks slowing down AI development would be wrong. He has used the debate to defend total freedom and unchecked progress: optimism should be “empirically the default assumption about how this will develop,” he said in an interview with the Wall Street Journal, where he also published an article.

Read more Burnham bets all his political capital on reforming social assistance and dependency aid