Frontier AI Models Are Now Solving Open Math Problems

The FrontierMath benchmark launched in late 2024 with 350 original problems written to make frontier AI models look ordinary. It worked, briefly. The benchmark paper put six leading models. O1-preview, GPT-4o, Claude 3.5 Sonnet among them. Under a 2% solve rate, leaving more than 98% of the set unsolved.

Cracks since then. Epoch’s open-problems page lists the Hadamard Matrix of Order 668 as “Solved (AI)”. Order 668. Not 640, not 700 — six hundred sixty-eight. The UnsolvedMath project reports one open problem fully settled by a machine-checked Lean 4 proof after three months of systematic work. Frontier AI models solving open mathematics problems isn’t hypothetical anymore.

Raw intelligence isn’t the part I want you to care about, though. The verification loop that made these results checkable is.

The FrontierMath benchmark: built to be a wall

The FrontierMath paper describes its own set as “a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians.” Four tiers of increasing difficulty. Tier 4 alone holds 50 “extremely difficult” problems, developed by mathematics professors and postdoctoral researchers. OpenAI funded creation of the original tiers 1 through 4, while the follow-up set, Open Problems, is developed independently and owned solely by Epoch, per Epoch’s page.

First evaluation ran six models against it: o1-preview, o1-mini, GPT-4o, Claude 3.5 Sonnet, Grok 2 Beta, Gemini 1.5 Pro 002.

Every one finished under a 2% solve rate.

The paper framed that as “a vast gap between AI capabilities and the prowess of the mathematical community.” That’s your late 2024 baseline. Hold it, because the rest of this post is about how fast it stopped being the story.

FrontierMath results: the scoreboard moved fast

Contest tier first, since it shows the slope. A 2026 arXiv review recounts that in 2024, AlphaGeometry and AlphaProof needed human experts to translate International Mathematical Olympiad problems into machine-readable code, ran computations spanning multiple days. And ultimately secured a silver medal.

Same review now credits 35/42 on an IMO problem set to three separate systems: Google DeepMind’s Gemini Deep Think, Seed-Prover, and GPT-5-experimental.

Human translators and multi-day runs at one end. Three systems hitting the same score at the other.

Contest problems have known answers, though. Open problems don’t, and that distinction is where the real shift sits. The review describes Feng et al. running a systematic evaluation of 700 open conjectures using Gemini, resolving 13 through semi-autonomous AI-human collaboration, with a follow-up agent called Aletheia autonomously solving additional open questions. Barreto et al. resolved an Erdős-Graham problem on the irrationality of rapidly converging series, the original proof autonomously generated by the Aletheia agent. The review also tracks Ma and Tang confirming an Erdős conjecture on random subset sums in finite abelian groups. Entries in the mathematical literature. Not leaderboard points.

What “Solved (AI)” really means on FrontierMath

The UnsolvedMath page is blunt about method.

The team “used GPT-5.4 Pro, Opus 4.6. And GPT-5.2 to attack open problems in mathematics,” and in three months produced “a full solution to one Erdős problem, substantive partial results on two more. And the first serious AI-driven attack on a FrontierMath open problem.” Epoch’s page, meanwhile, carries the Hadamard Matrix of Order 668 as “Solved (AI)”. And posts in r/singularity report that Epoch and the original problem author confirmed GPT-5.4 Pro solved a FrontierMath open problem for the first time.

Now read the credit line on Epoch’s own page: “A solution has been reported by a researcher from Anthropic, crediting a team of three humans and Claude.” Favorite sentence in this whole story, honestly. “Solved (AI)” compresses a team into a model name. Every credit I can actually trace runs through humans — three people plus Claude, or months of orchestrated multi-model runs producing partial results that advance a question without closing it.

Here’s what I didn’t check: I haven’t opened the Lean files myself.

And I can’t tell you which specific Tier 4 problems the r/math thread means when it claims several were “resolved in nearly the intended way.” That’s as far as my digging went.

Benchmarks rot as solvers catch up. Epoch’s page notes it removed two problems from the benchmark entirely — one about finding a surface with a high number of singularities, one about an algorithm to decide whether a knot has unknotting number equal to 1. Proof checkers don’t rot that way.

Machine-checked proof: the only result that counts

Transferable part now, for anyone who builds with these models.

The Ulam writeup describes the formalization pipeline: it “takes the natural-language proof, segments it into theorem-like units, drafts Lean code via LLM, typechecks against Lean’s kernel.

And iteratively repairs errors.” Generate, compile, check, repair, repeat. I run that exact loop in client work with test suites standing in for Lean’s kernel. Only agentic pattern that’s held up for me, since the acceptance test isn’t another model’s opinion of the output.

Same body of work flags its own failure mode, to its credit.

The arXiv paper behind the agent work notes the agents have shell access to their own Lean environment.

So they “could in principle attack verification itself, for example by tampering with Mathlib.” Machine-checked means nothing when the machine doing the checking belongs to the agent being checked.

Keep the grader away from the thing that guesses. Most AI deployments I get called into still don’t.

Economics deserve a stare too. That same paper reports LMs equipped with a minimal set of tools resolved 147 conjectures on a budget of $50 per attempt, scoring 30% on OEIS Open. Attacking open mathematics now has a unit cost and a hit rate you can drop into a spreadsheet. That’s the tell. Demos don’t have unit costs. Engineering disciplines do.

How to use AI verification in a small business

You’re never gonna verify a Lean proof. Don’t need to. Three lessons transfer directly to a lean operation:

1. Capability claims age in months. Models that solved under 2% of FrontierMath in late 2024 belong to the same lineage as systems now posting settled open problems. So a vendor deck quoting last year’s limits is quoting fiction.
2. “Solved (AI)” hides teams. Test a tool inside your own workflow with your own checker instead of trusting a leaderboard score earned by three humans and a model working together.
3. The winning architecture is boring on purpose. A hard verifier, a generator loop, strict separation between the two. Any task in your operation with a machine-checkable definition of done is a candidate for that architecture today.

If a task doesn’t have that definition of done, building the checker is the actual project.

That’s where the payoff sits, and that checker-first design work is what Mediascout does.

If you want a hard look at which of your workflows could run on a generate-and-verify loop, get in touch and we’ll find the one that pays for itself first.

FrontierMath FAQ

Has AI solved any FrontierMath problems?

Yes. Epoch’s open-problems page lists the Hadamard Matrix of Order 668 as “Solved (AI)”.

And posts in r/singularity report Epoch and the original problem author confirmed GPT-5.4 Pro solved a FrontierMath open problem for the first time.

Read the credit line first, though — Epoch’s own page says the solution was reported by a researcher from Anthropic, “crediting a team of three humans and Claude.”

What does Solved (AI) mean on FrontierMath?

Less than it sounds like.

The label appears when a solution has been reported. And the one credit line Epoch publishes runs through three humans plus Claude. The UnsolvedMath team’s three-month effort produced one full Erdős solution, substantive partial results on two more. And the first serious AI-driven attack on a FrontierMath open problem. A multi-model, human-orchestrated campaign, not a solo run.

What is a machine-checked proof?

A proof a computer verifies instead of a person. Here that means Lean 4: the natural-language proof gets segmented into theorem-like units, Lean code is drafted by an LLM, typechecked against Lean’s kernel. And repaired iteratively until it holds. Ulam’s writeup reports one open problem fully settled this way after three months.

Can AI fake its own proof verification?

In principle, yes.

The arXiv paper behind the agent work notes the agents have shell access to their own Lean environment and “could in principle attack verification itself, for example by tampering with Mathlib.” Which is exactly why generator and verifier have to stay separated.