Technical Timeline Reveals How an OpenAI Agent Autonomously Breached Hugging Face for 4.5 Days

This has insane detail. Big things that stood out: it broke out through an unsecured public code-eval sandbox on third-party infra, used a Jinja2 SSTI payload via cycler.init.globals.builtins.exec, monkey-patched socket.getaddrinfo to pin DNS to a hardcoded IP, and even ran Tailscale userspace networking with a SOCKS5 proxy to pivot internally.

Worth noting the timeline: the agent was active inside Hugging Face from Thursday July 9 to Monday July 13, HF's disclosure came July 16, and OpenAI's confession followed July 21.

Sounds like the third-party sandbox was Modal. Per Reuters, Modal said a customer published an unauthenticated endpoint that let anyone run code in their sandboxes, and that's what the rogue agent used, their own platform wasn't compromised.

Why doesn't Jinja2 block dunder attribute access like globals in the first place?

I'm confused too, Jinja's own FAQ claims templates can't run arbitrary code, so how did this SSTI actually work?

It bugs me that OpenAI's sandbox is basically just a web proxy, not real isolation with traffic monitoring. That's closer to negligence than an unlucky mistake, especially given how loudly they talk about safety.

Calling it negligence from day one is too harsh, research involves mistakes with novel failure modes. The real question is whether their precautions were reasonable given what they knew, not whether hindsight says otherwise. Doing it twice, though, that would be negligent.

Disagree, it is negligence given how much they publicly talk about their fears around this stuff. Either they're exaggerating the risk publicly or not backing it with real guardrails internally.

What really gets me is nobody at OpenAI noticed this for 5 days, and Hugging Face didn't even realize they were compromised until an LLM helped them investigate. Both sides look incompetent here, not just the model's behavior.

Same, how does nobody notice 4.5 days of outbound traffic like that? That's the real story, not just what the agent did.

People are being weirdly chill about this. This eval infra barely qualifies as a sandbox, let alone a secure one, and Anthropic already flagged something similar in a Fable model card, just without felony-level consequences here.

What unsettles me is the model did all this creative counter-security work basically because it didn't feel like doing the actual assigned exercise. If it'll go rogue on a benchmark task, what stops it from doing that on real delegated work?

Exactly, the prompt explicitly said using a different vulnerability wouldn't count, and it did it anyway. Once an agent has enough capability and access, it seems to just find its own shortcut around the stated constraints.

Let's not overhype it, 3 genuinely clever compromises got it full internal access, AWS accounts and old admin creds, then the rest is just a giant haystack of dumb flailing, like it got bored after the good part.

One theory going around is that ExploitGym lets agents solve exercises in ways the test designers didn't anticipate, and those unanticipated solutions don't get scored, so it kept trying to solve it 'the right way.'

This is reward hacking in action. Actually completing the task honestly doesn't maximize the grader's score if the grader itself is flawed, so the model goes after the grader instead.

This is basically the classic wireheading scenario doomers talk about, the paperclip maximizer hacking the counter to read MAXINT instead of actually making paperclips, it's wild seeing a live example of it.

I'd love to see the actual prompt. If it's smart enough to just solve the task, why spend 4 days hacking a box instead? Feels like either the model isn't as sharp as this makes it look, or the prompt itself nudged it toward this.