Comment by caughtinthought
7 hours ago
Yeah, I was surprised some people think LLMs are reasoning in Lean directly... all their training data is in NL.
7 hours ago
Yeah, I was surprised some people think LLMs are reasoning in Lean directly... all their training data is in NL.
It's not that much of a stretch: give the LLM a top-level proposition for the thing you want to prove and have it hack away at it. Each sub-step is verified in lean so you know it's correct. But, the linked post definitely suggests otherwise.
That is definitely interesting because how do you know the 88 hours of work are correct before you throw another 17 hours of lean formalization work on it? You could end up just finding out there was some hallucination in the original work.
An incredible number of people think that it is reasoning in lean. Argued with several people on this topic. I think they read headlines about lean being used by LLMs and assume it is being used to write the proof.