Comment by lifeisloving
3 days ago
The problem with trusting AI to learn a complex subject, which I've tried multiple times, is it will always at some point confidently give you the wrong explanation for something important and you will have no way to detect it. Its quite frustrating and that realization might not come for hours or days and you've built lots of false mental models that can be hard to untangle.
You have to do it along with some sort of ground truth, like a book, and you have to be able to understand that book in order to be able to see that bad advice immediately.
Alas, there are no shortcuts in life. You still need to very eagerly read and understand the source materials you're studying as you would have before AI and to the extent which AI is helpful for that is debatable. Your brain still has to do the thinking.
I agree with the sentiment of mixing up sources. However there are not really many textbooks that rise to the level of "ground truth". Plenty of times that books give wrong explanations as well.
Well I said sort of, but I'd argue most significant works in significant fields do not give completely incorrect explanations, and if they do there's usually versions of those works and they are updated. Your llm conversation is not versioned, there is no consensus built around its correctness.
In my experience learning with llms is the equivalent of doing (bad) tutorials, and you end up in tutorial hell where you feel like you're learning but you find it hard to implement said thing without that guide. It creates surface understanding thats not really worth the effort. Its better to dive deep on your own.
We're all looking for that shortcut, myself included, but as the saying goes, "there's no free lunch".
So... use a RAG trained on the books you care about?
>The problem with trusting AI to learn a complex subject, which I've tried multiple times, is it will always at some point confidently give you the wrong explanation for something important and you will have no way to detect it. Its quite frustrating and that realization might not come for hours or days and you've built lots of false mental models that can be hard to untangle.
This is an amazing problem to have! Often, preAI, I'd reject models flat out since they weren't even convincing on their face. There is value to this even if it's not correct. How long were people stuck with the aether? a few days or months is nothing.