Comment by alethic
17 hours ago
The utility of autoformalization is not actually in confirming the correctness of human-checked results. Mathematicians have a pretty good peer review process. I'm not actually aware offhand of any mathematical results that were accepted and later found to be incorrect -- though I'm sure cases exist, it's astoundingly rare.
The mathematics community's motivation for formalizing problems like Maryna Viazovska's sphere packing results in Lean weren't because the results were in doubt -- she won a Fields Medal for it, it's an extremely examined proof -- but because formalizing those results would lead to a lot of interesting and useful mathematical objects needing to be formalized as a prerequisite, which could then be merged into Lean's Mathlib and become useful for anyone working with Lean, particularly students. Having a library of idiomatic proofs available in a formal system capable of checking your work is Really Cool! Working in a proof assistant is a great way to develop mathematical maturity, especially for people who might not have an undergraduate education available to them.
(Unfortunately in the sphere packing case, the research group working on it made the mistake of trusting one of the various "AI for Math" slop companies, who promptly rugpulled them: https://arxiv.org/html/2603.03684v3)
So the short answer is "it depends on what you want". Lean is an eminently usable system for humans and LLMs alike; Metamath is uh. Not. But yes, Metamath seems to have a more trustable kernel wrt. the independent verifiers, so if that's all you're after it would be a better pick. But... there's only so many bugs Lean's kernel can have. At some point, they'll all be found.
(I should note that there is also utility in confirming the correctness of results that are too complex to be adequately verified by humans. The recent result that NF is consistent wrt. ZFC, among others (Peter Scholze's Liquid Tensors project) are examples of this. But that's distinct from "autoformalization" as commonly applied.)
> I'm not actually aware offhand of any mathematical results that were accepted and later found to be incorrect
https://mathoverflow.net/questions/35468/widely-accepted-mat...
Yes, in particular the slides by Kevin Buzzard (linked in there or here: https://www.andrew.cmu.edu/user/avigad/meetings/fomm2020/sli... ) are funny and educational on that matter.