Comment by kenferry
1 day ago
> There is one other aspect of the situation that I would like to emphasise. After two weeks of very hard work, I have managed to understand only a small fraction of [TC26a]. When I ask Fable about the paper, it seems to read and understand the whole thing in just a few minutes, even starting from a fresh context. I know that some readers may be uncomfortable with my anthropomorphic terminology, but I believe that “understand” is accurate, in the sense that the LLM seems to be able to correctly and lucidly answer all kinds of questions about the paper, including about some very fiddly technical details. That is exactly how I would tell that a human colleague has “understood” something. This chasm between human and LLM “speed of thought” is unsettling to say the least.
This is very odd, because it just doesn't match my experience with these systems at all. Yes, they do generally SOUND confident, but code (say) that's hard for me to understand is usually hard for the machine to understand as well, and while it is more indefatigable than I am, it's not brighter.
This mathematician is in a very good place to evaluate whether Fable gets it or not, but it's weird and counter to my experience.
I would indeed expect most modern models to read the paper and be able to answer questions about them, usually even in a manner consistent with the original intent, irrespective of whether the source material is correct!
Heck, I can do this too, with my eyeballs and brain! Just slower and will less overall memorized knowledge than modern LLMs.
Does the model get things wrong? Or does it just give up in trying to completely understand some very messy codebases?
All models still gets things wrong. Even Fable 5.1 I have to regularly correct when working on sufficiently complex code. So, I still want models to get smarter!
Fable is good, especially at conceptual stuff, don’t get me wrong, but it has limits. Opus 5.5 is a machine at getting correct code out once the concepts are in place. But I can’t trust either to not make mistakes.
As an aside, getting things wrong isn’t simply about not being smart enough, it’s also about the model insufficiently assessing whether it actually knows enough to conclude that some hypothesis is correct. As a human I have inner feelings of certainty and uncertainty regarding the things I am thinking about, which guide me in what I need to ascertain or verify more closely. My impression is that LLMs are lacking in that area, and it’s why they are most effective when there are external verification mechanisms.
But your code was probably written by a human. This proof was generated by an LLM. Is it surprising then that it’s easier for an LLM to read than for a human?
if you understood only small part of the paper, how do you know that fable answers are correct and not just sound reasonable?
Because a math proof is generally de-composable into smaller constituent arguments that can be independently evaluated for correctness. So you can audit a sample of these arguments for correctness without reviewing everything. You would then verify that the arguments being true implies the conclusion.
He knows whether the explanations are coherent with eachother.
These models have no self awareness. I just spent days having it write some test framework with very clear goals and it was highly confident understanding exactly what I wanted only for the whole thing to be complete bullshit, one remark from me about the work being useless and it goes “oh you’re absolutely right that’s on me”…