Math community tells AI labs: stop testing advanced math on proprietary models, fund human understanding
- An advisory group that collected over 600 replies from the mathematical community recommends AI labs release significant AI-generated math results responsibly and as soon as possible, and asks them to stop testing advanced math problems on proprietary models at all.
- The document rests on three principles: labs should release significant results quickly, a lab that ships output nobody understands must fund the work that builds that understanding, and the understanding effort must stay community led, not directed by the labs even when the labs produced the result.
- For papers a mathematician fully understands, the group says to follow existing norms: post a preprint, submit to a journal for peer review, and give talks. For papers nobody understands, Step I asks the lab to scour the literature and cite prior work even when the model found the idea independently, and to have a model rewrite each proof in standard paper style.
- Released results should be deposited in scholarly repositories that no AI lab controls, with persistent identifiers and other guaranteed standards.
- Step II asks labs to fund the mathematicians who will digest the output and look for applications. The paper concedes LLMs may not yet match human attribution or exposition, and says that shortfall does not remove the lab's obligation to do the best it can with its models.
Hacker News opinions
Asking labs to stop testing advanced math on their own models is like gatekeeping how someone breathes air. Math is just out there in the platonic realm, anyone can do what they like with it. Credit where credit is due is fine, but that's not what this asks for.
You're not alone on that, plenty of mathematicians feel the same way.
People have been trying to outlaw thinking for a while now.
Chemical compounds are out there too, available for anyone to use. Mixing ammonium chlorate with peroxide, why not? Or potassium permanganate with powdered aluminum. See how absurd that sounds?
Air is a fungible free resource everyone needs to live. AI companies are burning huge financial, human and compute resources on specialized models the public can't touch. The group's job is non-binding advice on how to use that private tech without harming the math community. Calling that gatekeeping misses the point.
I don't see why mathematicians deserve protection from AI more than any other profession. It's either everybody or nobody.
The labs drop tens of millions on publicity stunts while researchers who actually build general understanding go underfunded. When the PR value dries up nobody keeps pushing the work forward.
Gatekeeping is exactly what this is, you can tell from the first page of the text.
They specifically say proprietary models. Open models will hit this level in a year or so and the whole argument goes away.
That's what struck me too, plus the part where labs are supposed to fund human understanding. Publishing on math topics is a side quest for a company selling tokens. If mathematicians want their own labs to digest AI discoveries, fine, and they can petition for money. Dictating what labs do outside their business is what smells.
The real problem with proprietary models is you can't look inside and see how the thing got its answer, and that's what mathematicians do. The value of a theorem is less in the result than in the methods on the road to it. If we can inspect intermediate reasoning traces we might learn something about our own blind spots. Otherwise the output may as well come from an oracle.
Professions that insist there's a human element to the craft will come out better than the ones saying whatever, code is code. Look at writers and musicians, they're positioning themselves by shunning people who just pull the lever. Post a gen AI poem on an artist forum and you get eaten alive.
Noam Brown said their focus is shipping great models so everyone can make discoveries. Then OpenAI spends $15M of compute on an internal model to blitz a Navier-Stokes resolution someone else was already on track for, and keeps the model and tooling locked away. Asking them to stop an obviously damaging act isn't absurd, especially when they asked for the advice in the first place.
The law is also just a sequence of letters available to anyone, yet you can't practice without passing the bar. Doctors, journalists, electricians, same deal.
This reads like clinging to the past for its own sake rather than aiming for more predictable systems. Capping mechanical automation at the rate of human understanding is a very low ceiling.
Without mathematicians we wouldn't even know which problems matter. Famous conjectures are social constructs built from decades or centuries of attention. Lose that and math progress becomes tables of Lean statements plus a probable/unprovable bit from a model.
That could just be sampling bias. Humanity had 3000 years to come up with famous conjectures, AI mathematicians have had about a month. Give them time and they'll formulate consequential unsolved problems.
So the paper wants AI companies to pay human mathematicians to understand AI-generated math. That's an unusual ask.
If you're going to spend ten million dollars running 10000 agents at some hard problem, kicking back some grants to digest whatever they produced is reasonable. You could hire mathematicians in-house, but grants to PhD students are way cheaper than silicon valley salaries.