Comment by JumpCrisscross
3 hours ago
> using AI as a tutor, or even just for help with the hard assignments, produces slightly better learning outcomes
Source? The few people I’ve seen try to do this wind up with a terrible understanding of the material, with large knowledge gaps and one or two fundamental fuckups. In every case, an introductory textbook would have been better. (It would also have been harder.)
For coding specifically (there are many studies out there by now, but I know of these offhand):
https://www.mdpi.com/2076-3417/14/10/4115 -- probably the earliest one of its kind, finds over-reliance degrades critical skills but supplementary use is mostly harmless.
https://arxiv.org/html/2601.20245v2 -- Anthropic's study, same as above except supplementary use (like clarifying concepts) can actually be beneficial.
https://scale.stanford.edu/ai/repository/ai-meets-classroom-... -- "Students who use LLMs as personal tutors by conversing about the topic and asking for explanations benefit from usage. However, learning is impaired for students who excessively rely on LLMs to solve practice exercises for them and thus do not invest sufficient own mental effort." Interestingly, they found simply disabling copy-paste on the chatbot interface resulted in better outcomes!
Beyond coding, I recently came across this new meta-study; largely positive findings (which it admits may be biased) but does highlight evidence of negative effects of over-reliance: https://www.sciencedirect.com/science/article/pii/S2666920X2...
(Multiple studies find that the outcome depends on how AI is used. Surprisingly, incorrect guidance / unreliability / hallucinations appear to be a bigger problem than over-reliance! That could also explain poor performance in some cases.)
My intuition, supported by these studies, is that as long as students are willing to do the hard cognitive work -- for which there is no substitute, really -- having LLM assistance is a boon. Which makes sense, it's comparable to having a tutor explain difficult concepts. This is why in my mind the real problem is that the incentives to use AI as a crutch are just too strong.
https://arxiv.org/pdf/2604.04721
I think many on here are legit delusional and often talk about subjects with a lot of confidence when they clearly have a lot of mangled/broken knowledge - e.g. finance.