Comment by layer8

As an aside, getting things wrong isn’t simply about not being smart enough, it’s also about the model insufficiently assessing whether it actually knows enough to conclude that some hypothesis is correct. As a human I have inner feelings of certainty and uncertainty regarding the things I am thinking about, which guide me in what I need to ascertain or verify more closely. My impression is that LLMs are lacking in that area, and it’s why they are most effective when there are external verification mechanisms.