Comment by colechristensen
3 days ago
It doesn't have to be a mental illness.
Something which is very sorely missing from modern education is critical thinking. It's a phrase that's easy to gloss over without understanding the meaning. Being skilled at always including the aspect of "what could be wrong with this idea" and actually doing it in daily life isn't something that just automatically happens with everyone. Education tends to be the instructor, book, and facts are just correct and you should memorize this and be able to repeat it later. Instead of here are 4 slightly or not so slightly different takes on the same subject followed by analyzing and evaluating each compared to the others.
If you're just some guy who maybe likes reading popular science books and you've come to suspect that you've made a physics breakthrough with the help of an LLM, there are a dozen questions that you should automatically have in your mind to temper your enthusiasm. It is, of course, not impossible that a physics breakthrough could start with some guy having an idea, but in no, actually literally 0, circumstances could an amateur be certain that this was true over a weekend chatting with an LLM. You should know that it takes a lot of work to be sure or even excited about that kind of thing. You should have a solid knowledge of what you don't know.
It’s this. When you think you’ve discovered something novel, your first reaction should be, “what mistake have I made?” Then try to find every possible mistake you could have made, every invalid assumption you had, anything obvious you could have missed. If you really can’t find something, then you assume you just don’t know enough to find the mistake you made, so you turn to existing research and data to see if someone else has already discovered this. If you still can’t find anything, then assume you just don’t know enough about the field and ask an expert to take a look at your work and ask them what mistake you made.
It’s a huuuuuuuuuuuuge logical leap from LLM conversation yo novel physics. So huge a leap anyone ought to be immediately suspicious.
> Akin's Law #19: The odds are greatly against you being immensely smarter than everyone else in the field. If your analysis says your terminal velocity is twice the speed of light, you may have invented warp drive, but the chances are a lot better that you've screwed up.
Unfortunately people in the thrall of an LLM will tend to use the LLM itself as the checking device. They'll ask it what they could have missed, ask it if those things exclude the theory, etc... and the LLM will just blow smoke up their ass for all of those too.
> and ask an expert to take a look at your work
Which results in flooding experts with LLM glurge.
What to do when the trisector comes --- with an army?
Yeah, this was sorta what I was doing, I know LLMs are LLMs but I kinda tried to trick myself into thinking I could use an LLM to check an LLM, but I guess I'm also mentally stable (or educated) enough to know that's not sophisticated/realistic, and that was the conversation with my wife I mentioned. She's a professor and was basically like "LLMs are dumb, you're being dumb for using an LLM this way, DO NOT email some random professor about this, I already get enough of this shit, log off and go for a walk dumbass" - I would imagine someone like me with lesser stability around them would end up in a weird place, and I guess experts (as evidenced by my wife) are getting a lot of junk these days. (I still feel really foolish admitting all this, ha)
It must suck to be an expert right now?
1 reply →
I agree. It's not mental illness to make a mistake like this when one doesn't know any better - if anything, it points to gaps in education and that responsibility could fall on either side of the fence.