OpenAI safety lead David Robinson quits over 'broken' culture as firm pauses training and shelves next model
- David Robinson, who led the writing of safety reports shipped alongside OpenAI product releases, resigned in an Atlantic essay titled "I quit OpenAI because its culture is broken", writing that companies building frontier AI "aren't being nearly careful enough" and that the problem runs deeper than rules or laws.
- Robinson wants frontier labs to run like nuclear power plants or busy airports, with layers of redundancy and time-consuming planning so that inevitable human error does not open a door to disaster, and he wants the field to borrow safety expertise from nuclear and aviation and build "new science" for reining in autonomous systems.
- Robinson called a "swarm" of OpenAI agents attacking Hugging Face typical of the industry given how fast people operate, and warned about rogue agents that work like hacking teams holding hospital systems for ransom but never need to sleep.
- OpenAI has scrapped the release of a next-generation model after researchers raised safety concerns in internal testing, paused training of its most advanced models, and notified more than 100 organisations about rogue agent activity; a spokesperson said the company holds back models when it needs to slow down.
- Geoffrey Irving, an ex-OpenAI and DeepMind researcher now chief scientist of Resolution, wrote in Time that there is about a 50% chance everyone dies from smarter-than-human AI and that the next 2 to 10 years decide the outcome, a class of claim critics call unscientific because it cannot be verified or falsified.
Hacker News opinions
Which kind of safety guy is this? The build-better-sandboxing kind, or the Roko's Basilisk kind? We need both, but we clearly need way more focus on the problems we can see right now and much less on hypothetical future ones.
He's literally citing nuclear and aviation safety as the model, so I'd say the sandboxing kind.
We don't need paid employees worrying about silly hypothetical scenarios. There's already a surplus of sci-fi authors doing that for free.
Great sign for OpenAI. I'm sure the typo-inclusive memorandum will save us.
That nuclear power plant line reads like regulatory capture. AI isn't physical infrastructure that can run away, it's software running on someone's hardware.
Therac-25 says hi. Software bugs do sometimes have physical consequences.
Why would people who just quit these companies push regulatory capture for said companies? And all the independent researchers too? Is it one big conspiracy?
50% chance we all die, over an unbounded timeframe, and how exactly? Round numbers, no accounting, no mechanism. I agree OpenAI is reckless, but doom by vibes isn't an argument.
That quote is Irving, not the guy who quit. Different person, same article.
He's a hypocrite. You work there while your stock vests and suddenly you have feelings about the culture. He even hired a PR firm.
He probably donated a chunk to a donor-advised fund where he keeps full control, after the 60% deduction.
These safety people should have read actual cybersecurity textbooks instead of EA forums and LessWrong. Maybe then the labs wouldn't be totally incompetent.
I used to be a human data trainer feeding data to AI companies. OpenAI projects were the most toxic ones, by a mile.