Comment by emil-lp
7 hours ago
Spoiler: they don't.
CS exercises that we can expect an average student to solve is trivially solved by LLMs. Even smaller local models.
7 hours ago
Spoiler: they don't.
CS exercises that we can expect an average student to solve is trivially solved by LLMs. Even smaller local models.
This comes to not "smartness" of LLMs. But reality that we do not even want anything novel in these exercises or exams. And same areas are repeated multiple times so naturally there is lot of these in training data.
This is one area where LLMs really should excel at. And that doesn't really mean that students should not also learn it and be able to solve same issues. Which is real dilemma for the school system...
Except some times they aren't solved by LLMs, but appear to be. A CS student should be able to look at it and tell the difference.