Terminal-Bench-Science launches with 70 research workflows, Opus 5 reaches 30%
- Terminal-Bench-Science 0.1 evaluates 70 expert-curated scientific workflow tasks, and Claude Opus 5 recorded the highest reported resolution rate at 30.0%.
- The leaderboard lists GPT-5.6 Sol at 22.4%, Claude Fable 5 at 21.4%, Claude Opus 4.8 at 10.5%, and GPT-5.6 Terra at 8.6%.
- Tasks span five scientific domains: life sciences, physical sciences, Earth sciences, mathematical sciences, and engineering sciences, including simulations, theorem proving, inverse problems, sensor calibration, and scientific machine learning.
- Each task runs in a realistic environment and uses reproducible, task-specific tests to grade concrete artifacts such as analyses, simulations, proofs, code, and data products.
- The project describes itself as a continuous benchmark: researchers can propose tasks on GitHub, review them with others, and improve existing tasks as frontier models change.
Hacker News 의견들
GPT Sol beating Opus in Mathematical Sciences is what I care about right now. I also much prefer GPT's prose style.
Actual research workflows are the right direction for evaluations. Most agent benchmarks still feel like toy tasks.
Luna is already good enough for me to hand it a parser specification and have it write the parser. I do wonder when that is better than using something like lex.
I'm disappointed Gemini is absent. I wonder whether the listed models receive special training for scientific performance.
I'm surprised Opus 5 beats Fable here. In my coding use, Opus 5 has felt worse than Fable on almost every dimension.
This measures scientific tasks, though. Fable's safety behavior once blocked my biomedical pipeline for analyzing bacterial pathogenicity islands, presumably due to bioterrorism guardrails.
My experience is that Fable has more depth for tracing and debugging, but it can confabulate, resist context switches, and go off the rails when given too much freedom. Opus is less creative but more stable for long, methodical work.
Claude feels better to me on specialized scientific and mathematical details, while Codex feels mostly tuned for coding. Codex is still very good at bug-finding and building games.
What mathematical nuances do you mean? On my dataset of subtly flawed informal math arguments, Sol saturates the benchmark while Fable stays below 50%, even under generous grading.
I've had good results from context engineering: I put my coding heuristics in AGENTS.md and point the agent to subdocuments for specific situations. I'd like to know whether scientists encode domain heuristics the same way.
I've been waiting for a benchmark like this because it should push agents toward real scientific workflows. I'm building agents for that purpose myself.
Saying AI should have started with science ignores DeepMind's work.