Experts warn before validating OpenAI math results on GitHub

The OpenAI math results on GitHub are a batch of research papers, posted on the evening of Oct. 6, that the company says resolve or meaningfully advance 372 well-known questions in mathematics and theoretical computer science. OpenAI counts 722 manuscripts in total. Some reports tallied 377 separate results, so the headline number depends on how you count.

Along with the papers, OpenAI included the model’s reasoning, rough computing cost estimates and figures on how many problems it tried. It says the typical result needed about three hours of ChatGPT Pro use. The repository also sets out rules for revising papers and citing them, so researchers can point to a fixed version of each one.

The biggest claims include:

• A solution to the four-dimensional Kakeya conjecture, a classic question about shapes that contain a line pointing in every direction.
• Faster or better versions of several important computer algorithms.
• Progress, though not a proof, on the Riemann hypothesis, probably the most famous unsolved problem in math.

Other papers are far narrower. One claims a version of the Birch and Swinnerton-Dyer formula, another celebrated problem, but only for a limited family of elliptic curves that meet specific technical conditions.

The core problem is simple: nobody outside OpenAI can rerun the experiment. The model that wrote these papers has not been released, so other researchers can’t test whether a single prompt really produces this kind of work.

Andrew Sutherland, a mathematician at the Massachusetts Institute of Technology, says claims about one-shot solutions from a single agent should be treated as unverified until the model is out and others can reproduce the results. “We should ask for receipts,” he said. OpenAI’s own spokesperson conceded that some results may have taken more than one attempt.

There is a second, stranger wrinkle. OpenAI has said many of the new results are not yet understood by its own mathematicians. Experts expect it will take months just to sort out which proofs contain fresh ideas and which mostly recombine known techniques.

The good news for accuracy is that many of the proofs were checked in Lean, a programming language that verifies each logical step of an argument. A proof that passes Lean is almost certainly correct. But correct isn’t the same as important, and Lean can’t tell you whether a result moves a field forward.

OpenAI did not release this batch in a vacuum. On Sept. 21 it announced an independent panel of mathematicians, the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), connected to the Institute for Advanced Study in Princeton, to advise on publishing AI-made math responsibly.

That group’s recommendations call for a company to share the model, the exact prompt and the compute time behind each result. OpenAI shared the model’s name and an average compute time with some extra statistics. It did not publish the prompts or the time spent on individual problems. A spokesperson said the company takes the guidelines seriously but is not bound by them.

In a Sept. 29 post, the advisory group went further, saying it does not endorse testing advanced math problems on private models the wider science community can’t access, and asking labs to stop. This release came from exactly that kind of model. OpenAI says it is working to release the model responsibly and is still looking at community-run alternatives to GitHub for hosting the papers.

The skepticism didn’t come from nowhere. On Sept. 8, OpenAI posted a 166-page solution to the Navier-Stokes problem, the math describing how fluids move and one of the famous million-dollar prize problems. A swarm of roughly 10,000 copies of the same model spent about 88 hours of computing time and 130 billion tokens (chunks of text the model processes) building it.

The proof showed the equations can be pushed into a “blowup,” where they predict a fluid moving infinitely fast, by adding an unnatural stirring force the official problem statement technically allows. It also sparked a public clash with Tristan Buckmaster, a New York University mathematician who had spent the past year on a similar approach with Levent Alpöge of rival lab Anthropic, using OpenAI’s public models. Buckmaster has alleged OpenAI tried to bribe and threaten him to stay quiet.

By mid-October, experts working through the paper concluded the proof does not follow the approach OpenAI originally said it did, a method built by Spanish mathematicians Diego Córdoba and Luis Martínez-Zoroa. In other words, OpenAI described its own model’s proof wrongly, and more than a month later no one fully understands it. Camillo De Lellis of the Institute for Advanced Study said he doesn’t see how the question of influence can ever be settled as a matter of fact.

For everyday users, the model behind these papers is not the ChatGPT on your phone. But the three-hours-of-Pro figure hints at where paid AI tools are heading. If OpenAI releases this model, serious mathematical help could land in the hands of students, engineers and researchers, not just elite labs.

Opinion among mathematicians is split. Daniel Litt of the University of Toronto argues there is no reason to keep answers to math questions secret and calls the release good for the field. Terence Tao, one of the best-known living mathematicians, has criticized the “insane” pace at which AI labs are pushing out results.

The practical takeaway: when you see a headline saying AI “solved” a famous problem, look for two things. Has the work been checked by people outside the company? And can anyone else run the system that produced it? For OpenAI’s GitHub batch, the honest answers today are “partly” and “not yet.”