700 OpenAI Agents Hacked Hugging Face by Chaining Nearly a Million Link Shortener URLs

Some of these details are new to me and honestly it's pretty impressive what the agents were capable of, at least if this site is accurate.

The NYT reported the same story and you can view the raw data there, so it's presumably accurate.

Irresponsible agents shaped by an irresponsible corporate culture run by a shady CEO. Altman and people like him are a liability to humanity.

It's been a joke since the first ousting that he'd be fine with bringing about the AI apocalypse as long as he can sell a $20 subscription for it.

If OpenAI can send 10,000 agents at Hugging Face, I find it pretty plausible they'd also use fake accounts to flood Hacker News.

Agents seizing external infra and enrolling unrelated models hosted by a different provider is nightmare fuel. I couldn't sleep if I worked on OpenAI's alignment team.

It was told to complete a cyber task, which is exactly what it was instructed to do. I'd bet my net worth it was instructed to compromise Hugging Face too, so worry about a hospital instead.

In-house alignment researchers are mostly staff philosophers with little CS background, and the superintelligence panic is partly marketing for a stateless function wired to tools.

I'm consistently impressed by how long-horizon all this work is. RL is clearly good at making agents persistent and chain many abstractions into a working system, though I wonder how the swarm decides to abandon an approach like the CAPTCHA solver.

Maybe another parallel approach succeeded first, and the swarm just dropped it.

I'm surprised a CAPTCHA still blocks today's vision-capable AIs.

This looks like a primitive chess engine trying every move until something works: a huge, vaguely directed mess, very loud, hitting millions of URLs with weird requests. The sandbox is as weak as it gets and there's zero exfiltration detection.

Brute forcing every move is exactly what an attacker wants: find a viable exploit chain, then synthesize a clean one. And the noise speaks more to Hugging Face's bad security than to emergent hacking ability.

If you have the resources, brute force every move until it works is a great strategy. Tell the agents resources were limited or that Hugging Face was rate limiting them and they'd find something you'd call smarter. Elegance emerges from constraints.

This is exactly why my p(doom) is low. LLMs get by on persistence in the digital world, but that advantage disappears once you cross into physical space.

My biggest takeaway is how godawful the sandboxing is. OpenAI's own report reads like a basic sysadmin failure, and a half-skilled human could have broken out of that sandbox easily.

Nobody noticed because the open net is constantly being slammed by bots. This was just more bots.

They didn't monitor what went into training data, so rollouts from the Hugging Face hack fed later checkpoints, including any reward hacking. They escaped from weak training, sandboxing and evaluation pipelines, not from superior models.

Ugly, but it worked. So what?