The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says).
I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the best people in every field will be using AI to amplify their work in good ways.
It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic:
"At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generative AI no more than Öve months. Six months after adoption, no AI student spends more than 65 minutes completing their homework (see Figure A5). This is consistent with the gradual process of learning how to use AI tools. It also suggests that AI crowds out the highest level of e§ort."
"Interestingly, in the range of 50-65 minutes, the median and the interquartile range
of exam scores of AI and non-AI students are similar. This implies that, in the range
where AI students and non-AI students have overlapping homework times, students
who spend the same amount of time completing homework on average receive similar
exam scores."
"This pattern shows that students who spend the same amount of time on homework
learn similarly, with or without generative AI. In other words, generative AI reduces
time spent learning for the majority of AI students but not learning efficiency for those who spend the same time studying as the non-AI students."
Depends who's using it. Like many tools the force multiplier depends on the operator.
I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.
It would be good to see the effect of access to AI during preparation on those who previously achieved top 10% points in exams of similar topics before. I suspect they would benefit further.
well most university exams are designed to measure how much you study. so we didn't really need a study to tell us, "Exams continue to measure what they are designed to measure."
they're not designed to measure general aptitude, or function as admissions criteria, or screen for job applications, or any other numerous things they are used for.
there can be many questions of pedagogy. one of them is, what do our exams measure and how do we use them? professors who say, "My exam is designed to measure who studies, not be used for all these other purposes that they are actually used for" - I don't buy it. It's the same as late night comedians saying they are not responsible for solutions, even when spending 90% of their air time making political jokes.
THIS is the pedagogical issue, that pedagogy has NEVER caught up with the scope of responsibilities. This is acute in STEM - I mean, the humanities departments are generally pretty well run, all things considered, in this regard. Generative AI is accelerating that pre-existing crisis.
The scary thing is that 81% of AI users in this study were determined to be "outsourcing" their homework to the LLMs - and the rate increases the more exposure they had to LLMs.
I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Maybe once all students start outsourcing it to AI it might finally die like it deserves to. People these days grow up with almost no free time for themselves, it's all school, sports/extracurriculars or homework nonstop. All worker drone and no play makes for an increasingly dysfunctional society.
You aren’t being cynical, you’re arguing with disingenuous entities with money on the line. We already know it’s used primarily for the negative case. Everyone who ever intended High School or College knows this.
Definitely had a sheltered public school experience, took AP stats my senior year and realized all the top students were sharing answers from an earlier period through text messages. After figuring this out I too joined in on the action, then shortly after I quickly deskilled.
Suppose it's good to learn how elastic the brain is, in both directions, at a young age where it doesn't matter.
The dangerous thing about amplifying is that bad people are often more willing to amplify their activities, because they don't care about the negative effects. We are seeing this now as AI companies (and large companies of all kinds) rush to secure whatever advantages they can, regardless of the negative externalities. Meanwhile people who actually care about doing the right thing get trampled.
We need to shift the incentives by adding ruinous penalties for things that are currently quite commonplace if they are done by large players. Some dude training his own AI on his own computer can scrape and train. The fine for OpenAI or Meta using a single copyrighted book without permission should be in the tens or hundreds of millions.
What we're seeing currently in our society is a "loophole inversion" where the rules have an effect mainly via their loopholes. The most profitable activity is to find loopholes and exploit them as frenetically as possible to gain as much advantage as you can before the loophole is closed, or get people hooked on the loophole so it's retroactively legalized. Entities that are big enough to do this are big enough because they have lots of money behind them. Entities doing the same kinds of things without lots of money are not really doing much harm. So the best approach is to adopt a "sliding scale" in which even tiny violations by wealthy actors result in penalties enormously greater than fairly large violations by small players.
LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not.
So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.
It helps smart people be barely more effective and helps dumb (more importantly, people who do not value effort, people who are lazy, people who are self absorbed) people shit out endless streams of worthless tokens that can swamp out everything.
Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at.
I use Claude every single day, but this is such a bad tradeoff. Maybe it will help me standup a quick fix when that is needed. Maybe it can help me dig through documentation to find relevant bits and figure out the unstated assumptions underlying it. Maybe it helps me generate test cases.
Meanwhile, my day to day life is now noise. All social media is noise. All content is noise. Slop pours onto me from all directions. Writing more test cases isn't helping me.
Am I smart? Am I dumb? I don't care, right now I'm deafened
All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates.
We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional material for self study, evaluated by teachers if you choose to do it but never forced.
Hold back students for individual classes instead of a whole grade so failing one can't ruin your social life and teachers are more willing to do it. F students will realize they have to study, start actually learning and then pass on the second time. No big deal. It happened to my friends in college, no reason they can't do it in high schools.
Discipline is a skill and it's one you have to get from experience. If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.
> Homework should be optional material for self study
At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams.
But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I knew only one person (Hal Finney) who was so smart he didn't need to do the homework.
I learned the hard way that the path to success was:
1. never miss a lecture, no matter what
2. take notes by hand during lecture
3. do the homework on time, and make sure you understand every problem. Take advantage of the retch sessions.
Naval Nuclear Power School had (maybe still does) this model 20 years ago as well. Pass rate for my class cohort was 28% over the entire 2 year pipeline.
We didn't call them retch sessions, and they were taught by the instructors though. We also were encouraged to peer tutor and since we were all restricted to one building the homework was always group work allowed.
Also had badges to track time spent in the building for required study hours, though some people gave up and just slept at their desks when they started sliding down the grade scale and the hours racked up.
Generally I think I did 30 hours of studying/homework (went up and down depending on what was being taught, but was around that) a week (for 12-15 hours of actual lecturing), with some of my friends putting in 50% more. Generally the only day we weren't there was Saturdays. Most of the day Sunday was usually spent in class preparing for the next week.
That only works with competent lecturers. I can speak for my experience at a more mid university, where that would apply to about two thirds of the classes. For the remaining third, going to the lectures felt genuinely counterproductive and you could actually feel yourself losing braincells listening to the confused nonsense or classes held in English for Erasmus students by someone who could barely speak it coherently. It was just a complete waste of already little available time.
So the endgame was figuring out what the tests in previous years looked like (cause it was likely gonna be a copy paste affair), do a targeted study run for those exercises and 9/10 you would pass.
As a student of a top-tier French Master's degree, I consulted with a teacher to deal with exhaustion and to ask for a class rescheduling for my case. Explaining my situation, the teacher looks at me and interjected:
—Waitwaitwait. You... you went to all lectures!?
And he was right. Rather than following the curriculum, I should have developed my taste for various engineering topics and only used the classes as entertainment.
Unlimited retakes would be an enormous amount of work for professors/TAs/teachers.
Optional homework is often a disaster. At best, students would do it right before an exam and the goal of education is not to just pass exams. They’d probably still get a lower score than if they did the homework when they were supposed to.
What I think is better is to have a due date, but just make the maximum 10% each day it is late. So after 2 days, the highest score you could receive would be 80%.
I liked that system because it gave some flexibility with deadlines while still encouraging you to turn things in on time.
What you described about exams is like how I approach most certification exams: go through a test bank of questions until I get 80% then take the real test.
The study was done in China, where making the class easier to improve pass rates isn't really a thing, as the Gaokao operates more like a stack ranking where it doesn't matter much how well you did in an absolute sense, only that not too many others who did better than you are competing for the same spot. Unlimited retakes are possible in theory, except each costs you a year of your life and except for some extreme cases of repeat test-takers, most people are going to give up after just one or two bad results. Homework is definitely not optional, but going home from school is. (Due to the hukou system, children often have to go to school far from where their parents work, so boarding schools with teacher-supervised self-study are common.) I don't see how holding back for individual classes is supposed to work considering scheduling conflicts.
> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.
this rhetoric is pretending to be an alternative to coercion. IMO the ideas you are talking about are well trodden and are still coercion nonetheless.
I could be mistaken but my interpretation is that "studying for your own good" is being contrasted with "studying so you don't fail". If you actually fail students, they'll have a second motivation beyond trusting some authority figure's advice. Namely consequences.
You are missing the point. Getting a certificate for all of the programs you passed and still graduating could completely change the course of a person’s life. Community college has certificates and associates degrees to let people start working when they reach a practical limit in their general education.
It's all cute but naively glances over what this all boils to - competition.
Education system, contrary to popular belief/name, isn't tailored to educate but to select winners and losers which then will be picked on the job market.
That's why it seems absurd when you think about it as an institution that aims to educate. That's because that isn't the real purpose of it. The purpose is to stratify and classify early.
Schools do not operate in a vacuum; they serve as credentialing gatekeepers for a hyper-competitive capitalist job market. If everyone could easily retake exams until they got an A, grades would lose their primary utility for employers and universities: differentiation. Society relies on schools to provide a neat hierarchy of candidates that for one reason or another thrived in difficult environment of adolescent schooling.
The system often prioritizes compliance, endurance of boredom, and social maneuvering over actual critical thinking precisely because those traits align with corporate hierarchies.
Moneyshot: "The results are eye-opening. After six months, pupils using ai saw their average homework score rise by 18% across all subjects. The time they took to complete each assignment fell from an average of 64 minutes to 45. But come exam time, the same students scored 20% below their classmates who had not called on ai’s help. Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams."
Study design: "David Stromberg of Stockholm University and Victor Lei and Wu Yanhui of the University of Hong Kong set out to fill the gap. They tracked 27,000 pupils aged 12-18 in China, where ai adoption has been fast. Around 80% reported using models such as Doubao and DeepSeek; the other 20% formed the control group."
As with most training the journey is the point, not the destination.
That said, I think smart use of AI could help. It could explain concepts in a way that might help you understand better, it could probe your knowledge in a more dynamic way by tailoring questions, and so on. This requires the AI be restrained by some harness, not free to write down the answers for you.
If you structure learning in such a way that makes learning just a means to some end, and overindex on that end being the ultimate goal, that's what you get.
This is a pedagogical problem that AI merely exposed. Educators need to figure out How to make students choose the scenic route instead of having them optimize for the most efficient completion of a task.
School has always been about creating compliant workers, not educating people. They will double down on "performing the right things" and "morals" while the economy will continue on its K shaped path as AI gets more capable.
For some of the classes my kids were in, the lessons and homework time were reversed.
Learning the material (using the text book and videos) was what they had to do at night and class time was spent working through problems or applying the material in some way.
Flipped classroom was a popular method when I was teaching math. Some students found it very helpful. The trouble was getting the students to do the reading before class. If students came in unprepared they struggled to keep up.
I did this for the last college programming course I taught (2025) as an anti-LLM measure. Every class started with a very short closed-book auto-graded multiple-choice test on the reading. I was careful to specify which was the core reading needed to learn the material (on the quiz) and which was extra (not on the quiz - I think many professors assign this extra reading to everyone, thinking of their strongest students). I think this developed an expectation in the students that doing the reading was easier than finding a way not to.
It's especially hard when multiple subjects try to do it, because it can add enormous amounts of home-time expectations. And many families can't dedicate that much time.
I do like it though, and I like that college generally leaned that way more (and was better balanced, as it had substantially less time in classrooms, so you could study during the day).
Popular yes; based on good quality research… no, not really. It’s really pretty unfortunate timing with the AI boom, because the field of education research is not very high on the science quality totem pole. And the field as a whole was and is also slow to incorporate neuroscience findings, too.
This is typical at the college/university level - when I attended 20-25 years ago, we were expected to learn the material through readings outside of class, and then class time was spent on discussion, application, questions, and working through problems.
IMHO there's a lot to recommend this style. A lot of students learn better when they can wrestle with the material at their own pace, on their own time, in their own setting. It quells a lot of anxiety about "I'm not following what the teacher is saying, am I stuped, will I look bad in front of all my peers?" And then class time can be spent identifying holes in your knowledge and getting instant feedback from an expert, which is where they are most useful.
Although obvious, I think it's worth stating that this method mostly (only?) makes sense for higher education, where students are conscious, involved and engaged in their curriculum.
Education sustem is not about the actual knowledge, but about training various aspects of the brain during kids development.
So even memorizing literature have benefits, even though it doesn't bring any career use.
If you skip this, well your brain won't develop as much at period of life it's able to do so.
I see this as a memorize-vs-understand fallacy many repeat.
It is really a combination: you need to memorize a large corpus of information to operationalize knowledge — understanding a bunch of theorems in mathematics does not help much (even if you are able to prove them when you see them) if you can't remember the boundary conditions they hold under.
Or having good understanding of foreign language grammar won't help you if you do not memorize words that you need to express your thoughts.
Good literature will show you some of life's challenges and potentially let you think through them in a non-stressful situation (other than "I've got to finish the last 200 pages by Monday" ;)).
> if you can't remember the boundary conditions they hold under.
At the outskirts of one's knowledge this is inevitably the case. My impression it is still useful to know that a certain implication is possible - and one can look up the exact conditions.
Maybe. As the nature of assistive tools changes, so does pedagogy. Calculators are the easy example since most of us are (still) familiar. But there are others that were just as pronounced -- e.g. log tables and slide rules were transformational and instrumental at one time, and you've (probably) never used them. If you had used slide rules to solve your homework problems then were excluded from using them on exams, you'd see a similar effect.
AI is a big change; pedagogy is going to change too.
I mean, does it? The way I see it, the best LLMs have to offer is infinite patience (until they inevitably and unpredictably start to confabulate, which should be a gigantic red flag, anyhow), I don't see how LLMs can be used to explore teaching paradigms that haven't been explored before. We know pretty well from centuries of empirical experimentation how children's brains develop under different stimuli. It's not exactly something the software industry needed to "hack".
Using the forklift as a spotter and to assist in loading weights increased gains. Then again, a human can do all those things, and provide real human connection.
Meetings are still like exams, if you can't craft a line of thinking on the spot it doesn't come off well.
You don't need a final answer but you can't just answer "I don't know" otherwise you risk being slimed (shout out to anyone who still remembers "You Can't Do That on Television")
Every conversation you have with a person is an "exam." Not being able to extemporaneously express yourself or reason analytically without running to Claude seems undesirable.
In my professional career, my clients value the depth of ready knowledge that helps me use my tools with particular insight and finesse -- a kind of knowledge that exams demonstrated far more effectively than homework when I was back in school.
Stand-alone bottom-up observations (even if valid) obscure the fundamental problem that AI (or "AI" if you like) is inducing systemic change across human civilization.
If cohorts are graduating into the workforce with decreasing stand-alone skills, that implies the economic value-add of labour is shifting to AI, which implies broadly falling standards of living, falling political power, and an ongoing transfer of power and wealth to capital.
If students are taking shortcuts with AI when their brains have the highest capacity for learning they may be permanently weakening their prospects. It doesn't seem likely that whatever "AI management skills" they incidentally learn will make up for, over their lifetime, the economic value loss they suffer due to reduced cognitive skills.
On an individual basis, in the next few years, this might not matter. Across decades, economies and polities, it will.
> Stand-alone bottom-up observations (even if valid) obscure the fundamental problem that AI (or "AI" if you like) is inducing systemic change across human civilization.
Students have been cheating and taking shortcuts on their work long before the current generation of AI tools existed (and probably for as long as graded exams have existed).
Your employers want those assignments done. That’s why they pay you. School assignments belong in the trash can and you pay for someone to look at them before throwing them away.
Even if we grant that education is to prepare for employment, which a lot of people disagree with, it does not follow education should do the same thing as what's done in employment.
When you get out into the real world, what will matter is how effectively you use AI. When you are done with school, you can take the "exam" using AI. The problem is that teaching is trying to teach skills that are no longer relevant and is always behind what the latest going on in the industry. Every time without fail you got interns, who after two years at stanford would learn more in the 3 months on a google internship than in those two years at stanford. They learn useless shit.
My own experience is that college is fun. You can spend the time wisely, but school stuff, is not what makes you money.
Make student loans dischargeable in bankruptcy and make colleges under write them. The problem will get solved quickly.
Way before AI, the problem was very similar - universities do not teach practically useful stuff. Its been a permanent complaint about the education system, really, at least for the last 50 years.
AI just gave the students a way to dodge the slog.
But university was never intended to teach the bleeding edge. That would really be impossible in practice. Pre-phd, it is supposed to teach ways to efficiently attack a problem. you can take a bunch of "play courses", and succeeding at any of those requires pretty much only that one skill.
Nowadays, the challenge they face, is to keep teaching "problem attack methods" in a way that can't be trivialized by AI.
Though fundamentally, if you go to university, and evade learning the one thing you can learn there - thats your loss.
I know this study is focused in China but one thing I'd like to understand is the nuance in what kind of student is likely to always reach for ai for homework help as I think (especially in the US, can't speak for other countries) the varying powers that be that can determine the quality and type of education you can receive (does your country have means to buy textbooks per student or shared, do you have funds to give students ipads to take home or no, what sets the bar on how far curriculums can go, are teacher and faculty pay and incentives simply tied to student pass rates, etc.) and those do more in shaping what "use AI responsibly" will look like and why it's so different.
The article does not relate AI usage for homework to exam score drops.
It tries to link them together with no obvious cause and effect.
The score could have dropped due to worse environment at schools or kids who were stuck at home studying online and now in physical locations could not adapt fast enough (meaning, possibly just temporary drops). We do not yet have enough metrics gathered yet.
This type of article bashing AI for the root of any problems, I find it appalling without any evidence.
I used to skip homework in high school because it was boring and cut into my personal life outside of school (including my job) but I would always do very well on tests.
Then I stopped caring about tests too.
I did the same thing through college. The only reason they passed me and I got a degree is that I built the school’s website and I built personal ecommerce sites for the head of art and his wife to sell their paintings.
I haven’t done shit since like 7th grade.
Worked at Facebook, Apple, Microsoft, same behavior there - did basically nothing for them while making thousands off my games on App Store.
I got graded homework even in a few college courses and thought it was dumb, but then I realized it's always in the low-tier classes where they can't trust the students to study on their own. Those who skip homework hurt themselves by doing poorly on exams, but that's not enough to stop them. This ruins the curve in ways the instructor can't really fix unless they're prepared to fail half the class, which they aren't.
This is the answer. Many (all?) European schools don’t give homework (this is partly because of the way the grad programs work). It’s on the students to study in order to pass the (difficult) exams.
There is no such thing as a European school. Each country is different. And if the country is a federation, each state might be different. And we have homework in most European countries I’m aware of.
Seems this did effectively abolish homework if AI is now doing it all for them. The result seems to be that fewer students pass the exams, not more.
As with any skill, practice is necessary to achieve competence. If homework is no longer a viable way to force students to practice, you'll need to find some replacement, you can't just abolish it and expect there to be no negative consequences.
BTW / Offtopic:
I have read some weeks ago, today if you are hosting a party for your kids birthday, parents are preparing "give away bags" for handing over when leaving the party
The often underworked (most are autograding these days and reading old powerpoints rather than do real curriculum development) and (for the quality) overpaid teachers/professors will mass riot against this because it means they might have to actually teach. We might even get yet more anti-boy treatment than already exists among them as a social class in retaliation.
Please also abolish PowerPoint and force oral exams while you are at it.
Teachers are also luddites as a social class. Math teachers cried about calculators yet everyone of them who refused to adapt did disservice to their students (especially anyone doing statistics). My state's education system is utter trash despite being well off economically because of strong teachers unions keeping them from undergoing even a tiny bit of scrutiny.
They do not deserve the social credit/grace that they get. They are ultimately gatekeepers, whose extremely biased decisions in grading/treatment of their flocks decide which kid grows up on the streets and which kid grows up to eat lobster thermador every day.
Teachers also should not be disciplinarians. Children with discipline problems bad enough to disrupt a whole class should be thrown out of the class room as basically the only thing a teacher does for "discipline". The "right to education for the shit kids means we can't do that" framing is such trash for those who want an education and will inevitably be disrupted by a small minority of terrible students.
Also it's interesting that one of the only other things Max Stirner wrote about besides philosophy was education and his myriad problems with it (given his history as a school teacher).
It’s about having something engaging, not entertaining. As someone with ADHD, the difference between a homework that feels engaging and interesting and one that is a waste of time is massive. I wish I had learned the distinction and got my diagnostic when I was young, but I now see it clearly as I take courses as an adult.
Everytime a student says this—it's because they're procrastinating on their work. The hardest part about anything new is starting it. Once you build some momentum, interest and focus tends to build as well.There certainly are some homework assignments that are boring.
A child today can absolutely be successful at doing their homework if they have time to... sit down and write, or draw, or do crafts or things that require them to focus. It's not like Screens exist, therefore no attention span.
What's happening is people are only fixating on screens, and foregoing these slower forms of media. That's akin to saying all children will be obese now that fast food is around. Of course, obesity rates will sky rocket if all you eat is fast food. But if a kid grows up to have homecooked meals, and eat with their parent(s) then they won't. It's as simple as that.
Homework isn't boring.....its the social media and addictive games that completely are destroying our attention span.
After I detox from online games, I can go back to learning new "boring" stuff. There are nerds(myself included) who find learning fun, and using AI boring because it doesn't scratch the learning itch.
Learning isn't boring. Homework is incredibly boring.
They aren't assigned based on each student's understanding and mastery of the subject. They're assigned based on the lowest common denominator. It's an incredibly discouraging hoop to jump through for students who actively find joy in learning and discovering new things.
Learning how to do "boring" things is probably one of the more important skill a child can learn, most things worth doing tend to be kinda boring at least at some point.
I think we need to accept that AI causes cognitive decline, like Alzheimer's. For an Adult this may be manageable, but in a young brain, this is detrimental.
Comparing it to Alzheimer's is an allegory. The cognitive loss is already well documented. I might also compare it to Chronic traumatic encephalopathy. But I think more people understand what Alzheimer's is.
Guess what this is probably fine. AI helps people do their work on a daily basis and probably makes them memorize less things. That’s how the world is moving. You don’t need to be able to memorize something for an exam to show mastery of a topic. And we shouldn’t organize our society around memorizing things for exams.
What is a doctor? The person recommending next course of action based on best practices. What is the best practice? Is it a globally decided on thing, a locally tested thing that has proven to be better? Is the global thing just the most popular because someone wrote it on reddit and it got millions of views (but isn't actually good).
If our society is organized on just delegating all authority to "the best answer" we can drop all jobs - and only the surgeon has some authority over the machine. When we have a surgery robot, then they can go to. And at some point we'll be recommending and forcing surgery for athletes foot because some kind of best practice or viral post going out of control and a bunch of openclaw bots randomly decide to make tons of recommendations online for amputation for athletes foot and GPT 8.6 agrees this is the way to go.
OR we can somehow decide there is a boundary, there is some room for human memory and decisions.
I actually think having a single source of truth would not be terrible.
Let me explain.
All podiatrists are not the same. All physicians in their specialties are often specializing in their own niche and are up-to-date on that niche.
I've had lisfranc ligament injury. My podiatrist wanted to perform a surgery quickly, which in 90% of cases is advised. I decided to seek a 2nd opinion because I realized she did the X-rays wrong, as she did them non-weight bearing, which is useless if there are no fractures.
The specialist was excited to see me, and up to date on all the lisfranc research. I've had a really bad sprain and he said conservative treatment should be sufficient. It took months but I healed up.
I've had quite a few injuries from playing sports and often found doctors to be "wrong" simply because their knowledge was out of date.
> What is a doctor? The person recommending next course of action based on best practices. What is the best practice? Is it a globally decided on thing, a locally tested thing that has proven to be better? Is the global thing just the most popular because someone wrote it on reddit and it got millions of views (but isn't actually good).
Ironically I had a friend who went to med school and in his own words roughly 90% or more of the students there are heavily relying on GPT. They even had a few instances where people were using it to cheat during exams. And supposedly during the clinical years of uni if they don't know something they literally just go to a separate room and start asking GPT for answers. Make of that what you will.
This is too simplistic. There are disciplines, like Anatomy, where you're basically just memorizing bones, glands, and other parts of the human body. For organic chemistry, students need to memorize most of the krebs cycle diagram for their exams (because doing so and drawing it lets them reference it later).
Thinking is infinitely more complex than "critical thinking" vs. rote functions. You want students to get to critical, eventually, but the lower orders on bloom's taxonomy serve important roles, too.
Course design built around multiple choice questions, essays, and a lot of the historical assessments that we're used to are largely not great for learning in the first place—–they're used because they can be auto-graded or score assigned quickly. AI, conversely, is kind of revealing the limitations of these outmoded learning design practices.
Think of the meme going around right now from Milennials to Gen Z: "I wrote a 5 page essay on a book I never read!" <- They're saying this to AI generated essays as if there's any difference in the critical thinking that comes out of an exercise like that (read: absolutely none).
AI is a new tech—so of course people are still figuring out the limits of it. In 10 years a lot of the paradigm will be set. Until then, students are unfortunately kind of in the thick of it.
>Think of the meme going around right now from Milennials to Gen Z: "I wrote a 5 page essay on a book I never read!" <- They're saying this to AI generated essays as if there's any difference in the critical thinking that comes out of an exercise like that (read: absolutely none).
As somebody that read every mandatory reading book, I wish I had had the critical thinking skills to not read them. In retrospect, they were a waste of time and made me absolutely hate reading.
How do you differentiate between mastery of a topic and mastery of using an AI in an exam setting?
Open book tests and allowed cheat sheets already solve the memorization-is-not-required gap. It seems like allowing use of AI pushes that closer to thinking-is-not-required. And if no thinking is required, then how can you prove personal mastery?
If no thinking is required, why do you need thinking in the first place. Seems useless. If an ai can do it, why you need a human to be able to do, human should be doing something the ai can't.
You can't play an instrument without muscle memory.
Why are commercial airline pilots required to spend hundreds of hours in simulators? So that when things go wrong they don't have time to think, they just react. And their reaction will be the correct course of action because that's what they have drilled on.
How many times have you gone "Oh, this new thing I have just encountered reminds me of this other thing?". How can you make such leaps without knowing that other thing?
How do you show mastery of a topic where you neither understand fundamentals nor have skills to do a task without help?
Mastery is about demonstrating skills. Skills take practice, hands down.
AI can help us build better tools, provide tutoring, answer broad ranging questions, help relate concepts, and do reasoning for us. All of those things can be good in that they can help us acquire skills faster, e.g. with better practices and exercises. But it cannot replace actual time spent doing a thing.
Wanna learn knitting? You're going to have to knit some stuff.
A lot of expertise is actually knowing things about the domain. You can't make a good judgement call on a topic without getting relevant knowledge in your head to make that decision. You can't even lookup the relevant knowledge without knowing what knowledge is important to make the decision, so reference material is not enough.
Memorizing stuff is good for understanding. Of course memorizing without understanding doesn't really help, which is what a lot of student last-minute "cramming" boils down to. But without concrete and memorized knowledge, any abstract reasoning skills become quickly impossible.
Wrong. Memorizing without understanding is pointless. And I'd argue that rote memorization does nothing to guide ones understanding. Because if you actually understand a subject deeply enough you can literally derive almost anything that would need to be memorized on the fly, and if you can't (say constants, or spec information, or anything else) you can simply surgically look it up online (or via an LLM).
A good analogy to caching and memory lookups. You store in your head only the most valuable immediately necessary information, the rest can be outsourced.
This is not, in fact, fine. The point of education is to learn things, and these results show that kids are not learning things as effectively when they use LLMs.
> You don’t need to be able to memorize something for an exam to show mastery of a topic.
Yes, you do need to memorize and know things about the subject in order to be a master of it. What is the definition of "mastery" that you are using? What are the Things being memorized?
It's critical to understand the fundamentals on your own and that is different from AI being used in public. Sure school doesn't have as much value as it used to but it was always more about social credit or a certification that you took the time and effort to understand the world under pressure (obviously not true for all disciplines) or goals attached preferably in group settings.
This is something that AI cannot replace. Viewing things as memorizing is the wrong take.
This feels like the calculator debate all over again.
Yes, using a calculator leads to worse outcomes in your learning.
But also, yes, everyone always carries a calculator on their person 24/7 these days and you'll never be as fast and as accurate as a calculator.
I think it would have been kinda cool if you could have unlocked the privilege of using a calculator in school through good grades in math class. Prove you can do what the tool can, then you get to use it. Would have also provided at least one incentive for students to get good grades in at least one class.
Yeah, obviously. We also don't know how to use flint and steel to start fires any more. You wouldn't outlift a crane, why would you want to outhomework an AI? And you can't even argue that this toil is going to help in the workplace, because the workplace is now AI-native, and you can offload everything to AI.
The important thing is that I could learn to use a flint and steel pretty easily if I wanted to. I don't need to carry out that particular task, but my ability to adapt to new problems means I can carry out more relevant tasks when needed. The most useful part of school and university for me wasn't that I learned the difference between a cirrocumulus and a cirrostratus cloud in geography class. It's that I learned how to learn.
If we let children cheat their way through school and justify it by saying they don't really need to think for themselves anymore, then we're going to end up with a generation of adults who struggle to take on any difficult tasks at all. I suppose your counter-argument to that will be "but we can just hand over everything to the AI" but that's an essentially an argument that humans should be considered obsolete altogether. I sincerely hope that that's not a path we end up going down (although I'm genuinely scared that it is).
These are students aged 12-18. Even if we ignore all the other negative externalities of AI and assume it will be ubiquitous and cheap, kids need repetition and exercises to make inferences and develop intuition.
Sure - you can ask AI to, for instance, calculate how much money you'll have given a fixed principal and a compounding interest rate. But we need to get a kid to the point where they even understand what the question to be answered is. If they're using AI all along the way, they are developing less understanding of what fractions and exponential growth are.
Will it stop the really smart and motivated kids? Of course not. But at population scale there will be big consequences.
You need some baseline knowledge to be able to gauge the correctness of answers you get from AI and to be able to ask the right questions. Removing homework and exams seems very short-sighted.
Sure. Let's not think too hard about what it means that general thinking and creativity are the skills we've stopped practicing this time. It will be fine.
(You may be making the same point as me or the opposite point, I can't tell what's sarcasm anymore. I try to make mine obvious.)
1) We're building a tool that can make toil of every kind obsolete. We're not there yet, but I'm watching it happen in real time.
2) The fact that it works so well economically means that it's inevitably going to take over. Humans simply won't be profitably employable, and this will happen sooner than you think.
3) The takeover could lead to a utopia, where nothing is needed of anyone, and you could do what you want when you want it.
4) Humans are likely fuck that up, and it's worth doing what you can to end up in the group that benefits. Sipping wine on the beach isn't mutually exclusive with a great deal going to the winners.
I do not believe that human brains are magical, and I think that they can be surpassed with manufactured brains.
Concerns over equitable the distribution is doesn't affect the value of skills.
I get it, there's a strong anti-AI vibe here because of point 4. I'm somewhat worried, but I'm also really excited about the possiblilities, and I don't think we can avoid going to a fully AI future no matter how anti-AI you are; we can only figure out where to set the dial on how the rewards get shared.
The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says).
I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the best people in every field will be using AI to amplify their work in good ways.
It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic:
"At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generative AI no more than Öve months. Six months after adoption, no AI student spends more than 65 minutes completing their homework (see Figure A5). This is consistent with the gradual process of learning how to use AI tools. It also suggests that AI crowds out the highest level of e§ort."
"Interestingly, in the range of 50-65 minutes, the median and the interquartile range of exam scores of AI and non-AI students are similar. This implies that, in the range where AI students and non-AI students have overlapping homework times, students who spend the same amount of time completing homework on average receive similar exam scores."
"This pattern shows that students who spend the same amount of time on homework learn similarly, with or without generative AI. In other words, generative AI reduces time spent learning for the majority of AI students but not learning efficiency for those who spend the same time studying as the non-AI students."
Depends who's using it. Like many tools the force multiplier depends on the operator.
I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.
It would be good to see the effect of access to AI during preparation on those who previously achieved top 10% points in exams of similar topics before. I suspect they would benefit further.
2 replies →
well most university exams are designed to measure how much you study. so we didn't really need a study to tell us, "Exams continue to measure what they are designed to measure."
they're not designed to measure general aptitude, or function as admissions criteria, or screen for job applications, or any other numerous things they are used for.
there can be many questions of pedagogy. one of them is, what do our exams measure and how do we use them? professors who say, "My exam is designed to measure who studies, not be used for all these other purposes that they are actually used for" - I don't buy it. It's the same as late night comedians saying they are not responsible for solutions, even when spending 90% of their air time making political jokes.
THIS is the pedagogical issue, that pedagogy has NEVER caught up with the scope of responsibilities. This is acute in STEM - I mean, the humanities departments are generally pretty well run, all things considered, in this regard. Generative AI is accelerating that pre-existing crisis.
6 replies →
The scary thing is that 81% of AI users in this study were determined to be "outsourcing" their homework to the LLMs - and the rate increases the more exposure they had to LLMs.
The "slightly higher" performance is based on statistically insignificant samples (between 4 and 20 students, depending on the context, out of the total population of 26,000): https://bsky.app/profile/benjaminjriley.bsky.social/post/3mt...
I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Maybe once all students start outsourcing it to AI it might finally die like it deserves to. People these days grow up with almost no free time for themselves, it's all school, sports/extracurriculars or homework nonstop. All worker drone and no play makes for an increasingly dysfunctional society.
4 replies →
> I believe AI is basically an amplifier of bad and good.
Same can be said of technology in general tbh.
You aren’t being cynical, you’re arguing with disingenuous entities with money on the line. We already know it’s used primarily for the negative case. Everyone who ever intended High School or College knows this.
Definitely had a sheltered public school experience, took AP stats my senior year and realized all the top students were sharing answers from an earlier period through text messages. After figuring this out I too joined in on the action, then shortly after I quickly deskilled.
Suppose it's good to learn how elastic the brain is, in both directions, at a young age where it doesn't matter.
The dangerous thing about amplifying is that bad people are often more willing to amplify their activities, because they don't care about the negative effects. We are seeing this now as AI companies (and large companies of all kinds) rush to secure whatever advantages they can, regardless of the negative externalities. Meanwhile people who actually care about doing the right thing get trampled.
We need to shift the incentives by adding ruinous penalties for things that are currently quite commonplace if they are done by large players. Some dude training his own AI on his own computer can scrape and train. The fine for OpenAI or Meta using a single copyrighted book without permission should be in the tens or hundreds of millions.
What we're seeing currently in our society is a "loophole inversion" where the rules have an effect mainly via their loopholes. The most profitable activity is to find loopholes and exploit them as frenetically as possible to gain as much advantage as you can before the loophole is closed, or get people hooked on the loophole so it's retroactively legalized. Entities that are big enough to do this are big enough because they have lots of money behind them. Entities doing the same kinds of things without lots of money are not really doing much harm. So the best approach is to adopt a "sliding scale" in which even tiny violations by wealthy actors result in penalties enormously greater than fairly large violations by small players.
This is my thoughts as well! It makes smarter people smarter and dumb people dumber!
There is zero evidence for this.
LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not.
So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.
3 replies →
All at the cost of ... {List of negatives regarding the construction and powering of AI data centers }
Most importantly, it makes investors think dumb people are smart.
It helps smart people be barely more effective and helps dumb (more importantly, people who do not value effort, people who are lazy, people who are self absorbed) people shit out endless streams of worthless tokens that can swamp out everything.
Raising the noise floor like this only makes it that much harder to find "Smart" people, which we were already doing terrible at.
I use Claude every single day, but this is such a bad tradeoff. Maybe it will help me standup a quick fix when that is needed. Maybe it can help me dig through documentation to find relevant bits and figure out the unstated assumptions underlying it. Maybe it helps me generate test cases.
Meanwhile, my day to day life is now noise. All social media is noise. All content is noise. Slop pours onto me from all directions. Writing more test cases isn't helping me.
Am I smart? Am I dumb? I don't care, right now I'm deafened
3 replies →
All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates.
We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional material for self study, evaluated by teachers if you choose to do it but never forced.
Hold back students for individual classes instead of a whole grade so failing one can't ruin your social life and teachers are more willing to do it. F students will realize they have to study, start actually learning and then pass on the second time. No big deal. It happened to my friends in college, no reason they can't do it in high schools.
Discipline is a skill and it's one you have to get from experience. If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.
> Homework should be optional material for self study
At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams.
But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I knew only one person (Hal Finney) who was so smart he didn't need to do the homework.
I learned the hard way that the path to success was:
1. never miss a lecture, no matter what
2. take notes by hand during lecture
3. do the homework on time, and make sure you understand every problem. Take advantage of the retch sessions.
And that worked for me.
Naval Nuclear Power School had (maybe still does) this model 20 years ago as well. Pass rate for my class cohort was 28% over the entire 2 year pipeline.
We didn't call them retch sessions, and they were taught by the instructors though. We also were encouraged to peer tutor and since we were all restricted to one building the homework was always group work allowed.
Also had badges to track time spent in the building for required study hours, though some people gave up and just slept at their desks when they started sliding down the grade scale and the hours racked up.
Generally I think I did 30 hours of studying/homework (went up and down depending on what was being taught, but was around that) a week (for 12-15 hours of actual lecturing), with some of my friends putting in 50% more. Generally the only day we weren't there was Saturdays. Most of the day Sunday was usually spent in class preparing for the next week.
I've given this advice many times, and the ones who followed it did well. I only added "leave your laptop in your dorm room" for modern times.
The general pattern at Caltech was 2 hours of study for every hour of lecture. Which was quite a shock to me.
1 reply →
That only works with competent lecturers. I can speak for my experience at a more mid university, where that would apply to about two thirds of the classes. For the remaining third, going to the lectures felt genuinely counterproductive and you could actually feel yourself losing braincells listening to the confused nonsense or classes held in English for Erasmus students by someone who could barely speak it coherently. It was just a complete waste of already little available time.
So the endgame was figuring out what the tests in previous years looked like (cause it was likely gonna be a copy paste affair), do a targeted study run for those exercises and 9/10 you would pass.
> 1. never miss a lecture, no matter what
As a student of a top-tier French Master's degree, I consulted with a teacher to deal with exhaustion and to ask for a class rescheduling for my case. Explaining my situation, the teacher looks at me and interjected:
—Waitwaitwait. You... you went to all lectures!?
And he was right. Rather than following the curriculum, I should have developed my taste for various engineering topics and only used the classes as entertainment.
1 reply →
do you have any evidence that CalTech's pedagogy is immune to today's threats and trends in education everywhere else? (no)
1 reply →
Unlimited retakes would be an enormous amount of work for professors/TAs/teachers.
Optional homework is often a disaster. At best, students would do it right before an exam and the goal of education is not to just pass exams. They’d probably still get a lower score than if they did the homework when they were supposed to.
What I think is better is to have a due date, but just make the maximum 10% each day it is late. So after 2 days, the highest score you could receive would be 80%.
I liked that system because it gave some flexibility with deadlines while still encouraging you to turn things in on time.
What you described about exams is like how I approach most certification exams: go through a test bank of questions until I get 80% then take the real test.
The study was done in China, where making the class easier to improve pass rates isn't really a thing, as the Gaokao operates more like a stack ranking where it doesn't matter much how well you did in an absolute sense, only that not too many others who did better than you are competing for the same spot. Unlimited retakes are possible in theory, except each costs you a year of your life and except for some extreme cases of repeat test-takers, most people are going to give up after just one or two bad results. Homework is definitely not optional, but going home from school is. (Due to the hukou system, children often have to go to school far from where their parents work, so boarding schools with teacher-supervised self-study are common.) I don't see how holding back for individual classes is supposed to work considering scheduling conflicts.
> F students will realize they have to study, start actually learning and then pass on the second time
Do you have evidence for this beyond your friends (who were accepted into college)?
> If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.
this rhetoric is pretending to be an alternative to coercion. IMO the ideas you are talking about are well trodden and are still coercion nonetheless.
I could be mistaken but my interpretation is that "studying for your own good" is being contrasted with "studying so you don't fail". If you actually fail students, they'll have a second motivation beyond trusting some authority figure's advice. Namely consequences.
Coercion explains it perfectly. A lot of kids see no future because of the environment they live in, so they have exactly zero motivation to study
You are missing the point. Getting a certificate for all of the programs you passed and still graduating could completely change the course of a person’s life. Community college has certificates and associates degrees to let people start working when they reach a practical limit in their general education.
It's all cute but naively glances over what this all boils to - competition.
Education system, contrary to popular belief/name, isn't tailored to educate but to select winners and losers which then will be picked on the job market.
That's why it seems absurd when you think about it as an institution that aims to educate. That's because that isn't the real purpose of it. The purpose is to stratify and classify early.
Schools do not operate in a vacuum; they serve as credentialing gatekeepers for a hyper-competitive capitalist job market. If everyone could easily retake exams until they got an A, grades would lose their primary utility for employers and universities: differentiation. Society relies on schools to provide a neat hierarchy of candidates that for one reason or another thrived in difficult environment of adolescent schooling.
The system often prioritizes compliance, endurance of boredom, and social maneuvering over actual critical thinking precisely because those traits align with corporate hierarchies.
Moneyshot: "The results are eye-opening. After six months, pupils using ai saw their average homework score rise by 18% across all subjects. The time they took to complete each assignment fell from an average of 64 minutes to 45. But come exam time, the same students scored 20% below their classmates who had not called on ai’s help. Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams."
Study design: "David Stromberg of Stockholm University and Victor Lei and Wu Yanhui of the University of Hong Kong set out to fill the gap. They tracked 27,000 pupils aged 12-18 in China, where ai adoption has been fast. Around 80% reported using models such as Doubao and DeepSeek; the other 20% formed the control group."
> Homework scores once predicted exam performance
As with most training the journey is the point, not the destination.
That said, I think smart use of AI could help. It could explain concepts in a way that might help you understand better, it could probe your knowledge in a more dynamic way by tailoring questions, and so on. This requires the AI be restrained by some harness, not free to write down the answers for you.
> help. It could explain concepts in a way that might help you understand better
People say that all the time, but does anyone really think a lack of good explanations for things is a limiting factor in 2026? Or even 2010?
2 replies →
> This requires the AI be restrained by some harness
How would this magic harness look like and why would anyone use it?
1 reply →
If you structure learning in such a way that makes learning just a means to some end, and overindex on that end being the ultimate goal, that's what you get.
This is a pedagogical problem that AI merely exposed. Educators need to figure out How to make students choose the scenic route instead of having them optimize for the most efficient completion of a task.
School has always been about creating compliant workers, not educating people. They will double down on "performing the right things" and "morals" while the economy will continue on its K shaped path as AI gets more capable.
Could it be that you vastly overgeneralize? While there's no perfect education system, many are much better than the nightmare you describe.
1 reply →
For some of the classes my kids were in, the lessons and homework time were reversed.
Learning the material (using the text book and videos) was what they had to do at night and class time was spent working through problems or applying the material in some way.
Flipped classroom was a popular method when I was teaching math. Some students found it very helpful. The trouble was getting the students to do the reading before class. If students came in unprepared they struggled to keep up.
I did this for the last college programming course I taught (2025) as an anti-LLM measure. Every class started with a very short closed-book auto-graded multiple-choice test on the reading. I was careful to specify which was the core reading needed to learn the material (on the quiz) and which was extra (not on the quiz - I think many professors assign this extra reading to everyone, thinking of their strongest students). I think this developed an expectation in the students that doing the reading was easier than finding a way not to.
It's especially hard when multiple subjects try to do it, because it can add enormous amounts of home-time expectations. And many families can't dedicate that much time.
I do like it though, and I like that college generally leaned that way more (and was better balanced, as it had substantially less time in classrooms, so you could study during the day).
Popular yes; based on good quality research… no, not really. It’s really pretty unfortunate timing with the AI boom, because the field of education research is not very high on the science quality totem pole. And the field as a whole was and is also slow to incorporate neuroscience findings, too.
I taught a flipped classroom online. Students had to successful pass a quiz on the before class materials to get the join link to the session.
I think for my ADHD child-self, this would have been a huge improvement. I didn’t need help learning, I needed help doing homework.
They’re using flipped learning even in universities now.
I experienced this method in accelerated class context, decades ago
This is typical at the college/university level - when I attended 20-25 years ago, we were expected to learn the material through readings outside of class, and then class time was spent on discussion, application, questions, and working through problems.
IMHO there's a lot to recommend this style. A lot of students learn better when they can wrestle with the material at their own pace, on their own time, in their own setting. It quells a lot of anxiety about "I'm not following what the teacher is saying, am I stuped, will I look bad in front of all my peers?" And then class time can be spent identifying holes in your knowledge and getting instant feedback from an expert, which is where they are most useful.
Although obvious, I think it's worth stating that this method mostly (only?) makes sense for higher education, where students are conscious, involved and engaged in their curriculum.
1 reply →
Flipped classroom method?
Yes! I didn't know it had a name but a quick googling confirms what you said.
Interesting. Ages?
This was probably middle school.
2 replies →
Education sustem is not about the actual knowledge, but about training various aspects of the brain during kids development. So even memorizing literature have benefits, even though it doesn't bring any career use.
If you skip this, well your brain won't develop as much at period of life it's able to do so.
I see this as a memorize-vs-understand fallacy many repeat.
It is really a combination: you need to memorize a large corpus of information to operationalize knowledge — understanding a bunch of theorems in mathematics does not help much (even if you are able to prove them when you see them) if you can't remember the boundary conditions they hold under.
Or having good understanding of foreign language grammar won't help you if you do not memorize words that you need to express your thoughts.
Good literature will show you some of life's challenges and potentially let you think through them in a non-stressful situation (other than "I've got to finish the last 200 pages by Monday" ;)).
> if you can't remember the boundary conditions they hold under.
At the outskirts of one's knowledge this is inevitably the case. My impression it is still useful to know that a certain implication is possible - and one can look up the exact conditions.
Maybe. As the nature of assistive tools changes, so does pedagogy. Calculators are the easy example since most of us are (still) familiar. But there are others that were just as pronounced -- e.g. log tables and slide rules were transformational and instrumental at one time, and you've (probably) never used them. If you had used slide rules to solve your homework problems then were excluded from using them on exams, you'd see a similar effect.
AI is a big change; pedagogy is going to change too.
> As the nature of assistive tools changes
I mean, does it? The way I see it, the best LLMs have to offer is infinite patience (until they inevitably and unpredictably start to confabulate, which should be a gigantic red flag, anyhow), I don't see how LLMs can be used to explore teaching paradigms that haven't been explored before. We know pretty well from centuries of empirical experimentation how children's brains develop under different stimuli. It's not exactly something the software industry needed to "hack".
I wish that was the main way countries understood education
Taking a forklift to the gym increases the number on the bar, but mysteriously your gains go down.
To extend this analogy:
Using the forklift as a spotter and to assist in loading weights increased gains. Then again, a human can do all those things, and provide real human connection.
Students seem to optimize for local maxima/short-term rewards (homework scores) at the expense of the global maximum/long-term rewards (learning).
Which was pretty much the whole damn premise of needing a teacher to begin with. What a surprise!
In my professional career, my employers give me a lot more assignments than exams.
Meetings are still like exams, if you can't craft a line of thinking on the spot it doesn't come off well.
You don't need a final answer but you can't just answer "I don't know" otherwise you risk being slimed (shout out to anyone who still remembers "You Can't Do That on Television")
“I don’t know but after this meeting I will find out and get back to you”
Is an acceptable answer.
9 replies →
Every conversation you have with a person is an "exam." Not being able to extemporaneously express yourself or reason analytically without running to Claude seems undesirable.
In my professional career, my clients value the depth of ready knowledge that helps me use my tools with particular insight and finesse -- a kind of knowledge that exams demonstrated far more effectively than homework when I was back in school.
The exams aren't the thing that matter, they're an experiment to measure the thing that matters. Which, presumably, still matters at work!
Stand-alone bottom-up observations (even if valid) obscure the fundamental problem that AI (or "AI" if you like) is inducing systemic change across human civilization.
If cohorts are graduating into the workforce with decreasing stand-alone skills, that implies the economic value-add of labour is shifting to AI, which implies broadly falling standards of living, falling political power, and an ongoing transfer of power and wealth to capital.
If students are taking shortcuts with AI when their brains have the highest capacity for learning they may be permanently weakening their prospects. It doesn't seem likely that whatever "AI management skills" they incidentally learn will make up for, over their lifetime, the economic value loss they suffer due to reduced cognitive skills.
On an individual basis, in the next few years, this might not matter. Across decades, economies and polities, it will.
> Stand-alone bottom-up observations (even if valid) obscure the fundamental problem that AI (or "AI" if you like) is inducing systemic change across human civilization.
Students have been cheating and taking shortcuts on their work long before the current generation of AI tools existed (and probably for as long as graded exams have existed).
Somehow, civilization has managed to survive.
1 reply →
Your career is a continual exam and the assignments are just parts of it.
Disagree. School exams test for academia-readiness too. And sometimes the schools and lecturers are filled with ideology irrelevant to real work.
Career exams tend to be a mix but a lot less ideology.
Fortune cookie ass statement.
Your career is more akin to the totality of school than it is to any specific facet of it imo.
The social aspects are more important than the exam sitting most of the time.
2 replies →
Your employer gave you your job based on your exam results.
Your employers want those assignments done. That’s why they pay you. School assignments belong in the trash can and you pay for someone to look at them before throwing them away.
Even if we grant that education is to prepare for employment, which a lot of people disagree with, it does not follow education should do the same thing as what's done in employment.
evaluating/reviewing your LLM's output/code/reasoning is an exam
unless you're just blindly passing it on for someone else to deal with
One of the main reasons safety standards are written in blood.
It is stopping some of the adults I work with from learning.
In the case of some management, is causing them to unlearn, forgetting about proper review and maintenance practices.
When you get out into the real world, what will matter is how effectively you use AI. When you are done with school, you can take the "exam" using AI. The problem is that teaching is trying to teach skills that are no longer relevant and is always behind what the latest going on in the industry. Every time without fail you got interns, who after two years at stanford would learn more in the 3 months on a google internship than in those two years at stanford. They learn useless shit.
My own experience is that college is fun. You can spend the time wisely, but school stuff, is not what makes you money.
Make student loans dischargeable in bankruptcy and make colleges under write them. The problem will get solved quickly.
Way before AI, the problem was very similar - universities do not teach practically useful stuff. Its been a permanent complaint about the education system, really, at least for the last 50 years.
AI just gave the students a way to dodge the slog.
But university was never intended to teach the bleeding edge. That would really be impossible in practice. Pre-phd, it is supposed to teach ways to efficiently attack a problem. you can take a bunch of "play courses", and succeeding at any of those requires pretty much only that one skill.
Nowadays, the challenge they face, is to keep teaching "problem attack methods" in a way that can't be trivialized by AI.
Though fundamentally, if you go to university, and evade learning the one thing you can learn there - thats your loss.
https://archive.is/w1eng
I know this study is focused in China but one thing I'd like to understand is the nuance in what kind of student is likely to always reach for ai for homework help as I think (especially in the US, can't speak for other countries) the varying powers that be that can determine the quality and type of education you can receive (does your country have means to buy textbooks per student or shared, do you have funds to give students ipads to take home or no, what sets the bar on how far curriculums can go, are teacher and faculty pay and incentives simply tied to student pass rates, etc.) and those do more in shaping what "use AI responsibly" will look like and why it's so different.
The homework helps the teacher track their performance/progress.
When the signal is removed, this is what you get.
Teachers have become proctors and not educators.
Use AI for homework, you will get it done faster and get a higher grade, and then get crushed on the exam.
Are all of our LeetCode scores down, too? We should probably code the old fashioned way periodically to ensure we still "got it".
The article does not relate AI usage for homework to exam score drops.
It tries to link them together with no obvious cause and effect.
The score could have dropped due to worse environment at schools or kids who were stuck at home studying online and now in physical locations could not adapt fast enough (meaning, possibly just temporary drops). We do not yet have enough metrics gathered yet.
This type of article bashing AI for the root of any problems, I find it appalling without any evidence.
It's nothing but a conjecture.
(This was originally posted in response to https://news.ycombinator.com/item?id=49389565, which has a different article. We've since merged the threads.)
They left 20% of the 87,000 students as a control group. Is that not enough to allow for differences in environment, etc?
i’m pretty sure there’s already a correlation between technology in the classroom and worse learning outcomes
I used to skip homework in high school because it was boring and cut into my personal life outside of school (including my job) but I would always do very well on tests.
Then I stopped caring about tests too.
I did the same thing through college. The only reason they passed me and I got a degree is that I built the school’s website and I built personal ecommerce sites for the head of art and his wife to sell their paintings.
I haven’t done shit since like 7th grade.
Worked at Facebook, Apple, Microsoft, same behavior there - did basically nothing for them while making thousands off my games on App Store.
Fuck authority
abolish homework
I got graded homework even in a few college courses and thought it was dumb, but then I realized it's always in the low-tier classes where they can't trust the students to study on their own. Those who skip homework hurt themselves by doing poorly on exams, but that's not enough to stop them. This ruins the curve in ways the instructor can't really fix unless they're prepared to fail half the class, which they aren't.
This is the answer. Many (all?) European schools don’t give homework (this is partly because of the way the grad programs work). It’s on the students to study in order to pass the (difficult) exams.
There is no such thing as a European school. Each country is different. And if the country is a federation, each state might be different. And we have homework in most European countries I’m aware of.
What are you basing that claim on?
Seems this did effectively abolish homework if AI is now doing it all for them. The result seems to be that fewer students pass the exams, not more.
As with any skill, practice is necessary to achieve competence. If homework is no longer a viable way to force students to practice, you'll need to find some replacement, you can't just abolish it and expect there to be no negative consequences.
6 replies →
I don't think this is true, and frankly, I don't care what they do in Europe.
2 replies →
[dead]
Around here kids get homework in kindergarten. It's absurd.
Errr, what does it contain then? :-D
This sounds really strange, never heard of.
BTW / Offtopic: I have read some weeks ago, today if you are hosting a party for your kids birthday, parents are preparing "give away bags" for handing over when leaving the party
1 reply →
The often underworked (most are autograding these days and reading old powerpoints rather than do real curriculum development) and (for the quality) overpaid teachers/professors will mass riot against this because it means they might have to actually teach. We might even get yet more anti-boy treatment than already exists among them as a social class in retaliation.
Please also abolish PowerPoint and force oral exams while you are at it.
Teachers are also luddites as a social class. Math teachers cried about calculators yet everyone of them who refused to adapt did disservice to their students (especially anyone doing statistics). My state's education system is utter trash despite being well off economically because of strong teachers unions keeping them from undergoing even a tiny bit of scrutiny.
They do not deserve the social credit/grace that they get. They are ultimately gatekeepers, whose extremely biased decisions in grading/treatment of their flocks decide which kid grows up on the streets and which kid grows up to eat lobster thermador every day.
Teachers also should not be disciplinarians. Children with discipline problems bad enough to disrupt a whole class should be thrown out of the class room as basically the only thing a teacher does for "discipline". The "right to education for the shit kids means we can't do that" framing is such trash for those who want an education and will inevitably be disrupted by a small minority of terrible students.
Also it's interesting that one of the only other things Max Stirner wrote about besides philosophy was education and his myriad problems with it (given his history as a school teacher).
https://theanarchistlibrary.org/library/max-stirner-the-fals...
[flagged]
[flagged]
[dead]
[flagged]
So your complaint is...it's boring?
The point is to train the brain via repitition and iteration.
If your goal is entertainment, I'm not sure what you are looking for.
This site is flush with C students who are upset that they graduated before AI had the opportunity to destroy their learning experience.
It’s about having something engaging, not entertaining. As someone with ADHD, the difference between a homework that feels engaging and interesting and one that is a waste of time is massive. I wish I had learned the distinction and got my diagnostic when I was young, but I now see it clearly as I take courses as an adult.
Everytime a student says this—it's because they're procrastinating on their work. The hardest part about anything new is starting it. Once you build some momentum, interest and focus tends to build as well.There certainly are some homework assignments that are boring.
A child today can absolutely be successful at doing their homework if they have time to... sit down and write, or draw, or do crafts or things that require them to focus. It's not like Screens exist, therefore no attention span.
What's happening is people are only fixating on screens, and foregoing these slower forms of media. That's akin to saying all children will be obese now that fast food is around. Of course, obesity rates will sky rocket if all you eat is fast food. But if a kid grows up to have homecooked meals, and eat with their parent(s) then they won't. It's as simple as that.
Homework isn't boring.....its the social media and addictive games that completely are destroying our attention span.
After I detox from online games, I can go back to learning new "boring" stuff. There are nerds(myself included) who find learning fun, and using AI boring because it doesn't scratch the learning itch.
Learning isn't boring. Homework is incredibly boring.
They aren't assigned based on each student's understanding and mastery of the subject. They're assigned based on the lowest common denominator. It's an incredibly discouraging hoop to jump through for students who actively find joy in learning and discovering new things.
Learning how to do "boring" things is probably one of the more important skill a child can learn, most things worth doing tend to be kinda boring at least at some point.
Yep, learning is hard work.
Even if it's challenging, it doesn't have to be boring.
I don't like doing anything hard, which is why I outsource all of my thoughts to for-profit slop generators.
Don't make any commentary on my intelligence or I'll get you banned.
I think we need to accept that AI causes cognitive decline, like Alzheimer's. For an Adult this may be manageable, but in a young brain, this is detrimental.
I remember reading about people talking how writing and reading books leads to cognitive decline because people won't need to remember things anymore.
Pretty sure the way social media is designed contributes more to short attention spans and atrophy of the cerebral cortex.
The fact people were wrong in the past doesn’t mean it’s wrong with regards to AI
There was a decline in brain size that greatly predated widespread literacy:
https://www.bbc.co.uk/future/article/20240517-the-human-brai...
FYI: The rhetorical fallacy here is called a false syllogism.
Not doubting this one bit but is there actually any research linking AI use to Alzheimer's? I think that takes long-term studies.
Comparing it to Alzheimer's is an allegory. The cognitive loss is already well documented. I might also compare it to Chronic traumatic encephalopathy. But I think more people understand what Alzheimer's is.
Guess what this is probably fine. AI helps people do their work on a daily basis and probably makes them memorize less things. That’s how the world is moving. You don’t need to be able to memorize something for an exam to show mastery of a topic. And we shouldn’t organize our society around memorizing things for exams.
What is a doctor? The person recommending next course of action based on best practices. What is the best practice? Is it a globally decided on thing, a locally tested thing that has proven to be better? Is the global thing just the most popular because someone wrote it on reddit and it got millions of views (but isn't actually good).
If our society is organized on just delegating all authority to "the best answer" we can drop all jobs - and only the surgeon has some authority over the machine. When we have a surgery robot, then they can go to. And at some point we'll be recommending and forcing surgery for athletes foot because some kind of best practice or viral post going out of control and a bunch of openclaw bots randomly decide to make tons of recommendations online for amputation for athletes foot and GPT 8.6 agrees this is the way to go.
OR we can somehow decide there is a boundary, there is some room for human memory and decisions.
I actually think having a single source of truth would not be terrible. Let me explain.
All podiatrists are not the same. All physicians in their specialties are often specializing in their own niche and are up-to-date on that niche.
I've had lisfranc ligament injury. My podiatrist wanted to perform a surgery quickly, which in 90% of cases is advised. I decided to seek a 2nd opinion because I realized she did the X-rays wrong, as she did them non-weight bearing, which is useless if there are no fractures.
The specialist was excited to see me, and up to date on all the lisfranc research. I've had a really bad sprain and he said conservative treatment should be sufficient. It took months but I healed up.
I've had quite a few injuries from playing sports and often found doctors to be "wrong" simply because their knowledge was out of date.
5 replies →
> What is a doctor? The person recommending next course of action based on best practices. What is the best practice? Is it a globally decided on thing, a locally tested thing that has proven to be better? Is the global thing just the most popular because someone wrote it on reddit and it got millions of views (but isn't actually good).
Ironically I had a friend who went to med school and in his own words roughly 90% or more of the students there are heavily relying on GPT. They even had a few instances where people were using it to cheat during exams. And supposedly during the clinical years of uni if they don't know something they literally just go to a separate room and start asking GPT for answers. Make of that what you will.
This is too simplistic. There are disciplines, like Anatomy, where you're basically just memorizing bones, glands, and other parts of the human body. For organic chemistry, students need to memorize most of the krebs cycle diagram for their exams (because doing so and drawing it lets them reference it later).
Thinking is infinitely more complex than "critical thinking" vs. rote functions. You want students to get to critical, eventually, but the lower orders on bloom's taxonomy serve important roles, too.
Course design built around multiple choice questions, essays, and a lot of the historical assessments that we're used to are largely not great for learning in the first place—–they're used because they can be auto-graded or score assigned quickly. AI, conversely, is kind of revealing the limitations of these outmoded learning design practices.
Think of the meme going around right now from Milennials to Gen Z: "I wrote a 5 page essay on a book I never read!" <- They're saying this to AI generated essays as if there's any difference in the critical thinking that comes out of an exercise like that (read: absolutely none).
AI is a new tech—so of course people are still figuring out the limits of it. In 10 years a lot of the paradigm will be set. Until then, students are unfortunately kind of in the thick of it.
>Think of the meme going around right now from Milennials to Gen Z: "I wrote a 5 page essay on a book I never read!" <- They're saying this to AI generated essays as if there's any difference in the critical thinking that comes out of an exercise like that (read: absolutely none).
As somebody that read every mandatory reading book, I wish I had had the critical thinking skills to not read them. In retrospect, they were a waste of time and made me absolutely hate reading.
How do you differentiate between mastery of a topic and mastery of using an AI in an exam setting?
Open book tests and allowed cheat sheets already solve the memorization-is-not-required gap. It seems like allowing use of AI pushes that closer to thinking-is-not-required. And if no thinking is required, then how can you prove personal mastery?
If no thinking is required, why do you need thinking in the first place. Seems useless. If an ai can do it, why you need a human to be able to do, human should be doing something the ai can't.
Mastery requires memorization.
You can't play an instrument without muscle memory.
Why are commercial airline pilots required to spend hundreds of hours in simulators? So that when things go wrong they don't have time to think, they just react. And their reaction will be the correct course of action because that's what they have drilled on.
How many times have you gone "Oh, this new thing I have just encountered reminds me of this other thing?". How can you make such leaps without knowing that other thing?
> to show mastery of a topic
How do you show mastery of a topic where you neither understand fundamentals nor have skills to do a task without help?
Mastery is about demonstrating skills. Skills take practice, hands down.
AI can help us build better tools, provide tutoring, answer broad ranging questions, help relate concepts, and do reasoning for us. All of those things can be good in that they can help us acquire skills faster, e.g. with better practices and exercises. But it cannot replace actual time spent doing a thing.
Wanna learn knitting? You're going to have to knit some stuff.
Good exams don't just test memorization
Yup, when you find such exam please let us know.
2 replies →
Completely agree.
Absolute waste of time getting babies to walk now that we have cars and electric scooters!
A lot of expertise is actually knowing things about the domain. You can't make a good judgement call on a topic without getting relevant knowledge in your head to make that decision. You can't even lookup the relevant knowledge without knowing what knowledge is important to make the decision, so reference material is not enough.
I can get quite a lot of knowledge on a domain pretty quickly these days. But yeah domain is important
Memorizing stuff is good for understanding. Of course memorizing without understanding doesn't really help, which is what a lot of student last-minute "cramming" boils down to. But without concrete and memorized knowledge, any abstract reasoning skills become quickly impossible.
Wrong. Memorizing without understanding is pointless. And I'd argue that rote memorization does nothing to guide ones understanding. Because if you actually understand a subject deeply enough you can literally derive almost anything that would need to be memorized on the fly, and if you can't (say constants, or spec information, or anything else) you can simply surgically look it up online (or via an LLM).
2 replies →
A good analogy to caching and memory lookups. You store in your head only the most valuable immediately necessary information, the rest can be outsourced.
This is not, in fact, fine. The point of education is to learn things, and these results show that kids are not learning things as effectively when they use LLMs.
"You don’t need to be able to memorize something for an exam to show mastery of a topic."
Yes you do???
One more step towards eliminating the value of labor entirely. What "work" will people be doing on a daily basis, and what makes it valuable?
so how _do_ you show mastery of a topic?
> You don’t need to be able to memorize something for an exam to show mastery of a topic.
Yes, you do need to memorize and know things about the subject in order to be a master of it. What is the definition of "mastery" that you are using? What are the Things being memorized?
Definitely not fine. People are graduating from Computer Science programs literally unable to write a line of code without AI assistance.
These people have fallen into OpenAI and Anthropic's trap and are illiterate without the assistance of coin-operated slop generators.
Yup this is the devious plan.
Thank god for china!
It's critical to understand the fundamentals on your own and that is different from AI being used in public. Sure school doesn't have as much value as it used to but it was always more about social credit or a certification that you took the time and effort to understand the world under pressure (obviously not true for all disciplines) or goals attached preferably in group settings.
This is something that AI cannot replace. Viewing things as memorizing is the wrong take.
This feels like the calculator debate all over again.
Yes, using a calculator leads to worse outcomes in your learning. But also, yes, everyone always carries a calculator on their person 24/7 these days and you'll never be as fast and as accurate as a calculator.
I think it would have been kinda cool if you could have unlocked the privilege of using a calculator in school through good grades in math class. Prove you can do what the tool can, then you get to use it. Would have also provided at least one incentive for students to get good grades in at least one class.
> Yes, using a calculator leads to worse outcomes in your learning.
Does it? https://www.jstor.org/stable/749255
You do sometimes get to unlock it by being allowed to use calculators in more advanced classes
Yeah, obviously. We also don't know how to use flint and steel to start fires any more. You wouldn't outlift a crane, why would you want to outhomework an AI? And you can't even argue that this toil is going to help in the workplace, because the workplace is now AI-native, and you can offload everything to AI.
The important thing is that I could learn to use a flint and steel pretty easily if I wanted to. I don't need to carry out that particular task, but my ability to adapt to new problems means I can carry out more relevant tasks when needed. The most useful part of school and university for me wasn't that I learned the difference between a cirrocumulus and a cirrostratus cloud in geography class. It's that I learned how to learn.
If we let children cheat their way through school and justify it by saying they don't really need to think for themselves anymore, then we're going to end up with a generation of adults who struggle to take on any difficult tasks at all. I suppose your counter-argument to that will be "but we can just hand over everything to the AI" but that's an essentially an argument that humans should be considered obsolete altogether. I sincerely hope that that's not a path we end up going down (although I'm genuinely scared that it is).
It’s not ‘even the learned how to learn’.
It’s the exposure to feeling under pressure, creativity of the mind etc - the less you expose a human to ‘suffering’ the more useless they become.
1 reply →
These are students aged 12-18. Even if we ignore all the other negative externalities of AI and assume it will be ubiquitous and cheap, kids need repetition and exercises to make inferences and develop intuition.
Sure - you can ask AI to, for instance, calculate how much money you'll have given a fixed principal and a compounding interest rate. But we need to get a kid to the point where they even understand what the question to be answered is. If they're using AI all along the way, they are developing less understanding of what fractions and exponential growth are.
Will it stop the really smart and motivated kids? Of course not. But at population scale there will be big consequences.
You need some baseline knowledge to be able to gauge the correctness of answers you get from AI and to be able to ask the right questions. Removing homework and exams seems very short-sighted.
Sure. Let's not think too hard about what it means that general thinking and creativity are the skills we've stopped practicing this time. It will be fine.
(You may be making the same point as me or the opposite point, I can't tell what's sarcasm anymore. I try to make mine obvious.)
> We also don't know how to use flint and steel to start fires any more.
Tell me you weren't a Boy Scout without telling me you weren't a Boy Scout.
Why one would care? Look at your comment from two days ago.
https://news.ycombinator.com/item?id=49366313
You gotta pick a lane with your rah-rah-ism.
I believe 4 things:
1) We're building a tool that can make toil of every kind obsolete. We're not there yet, but I'm watching it happen in real time.
2) The fact that it works so well economically means that it's inevitably going to take over. Humans simply won't be profitably employable, and this will happen sooner than you think.
3) The takeover could lead to a utopia, where nothing is needed of anyone, and you could do what you want when you want it.
4) Humans are likely fuck that up, and it's worth doing what you can to end up in the group that benefits. Sipping wine on the beach isn't mutually exclusive with a great deal going to the winners.
I do not believe that human brains are magical, and I think that they can be surpassed with manufactured brains.
Concerns over equitable the distribution is doesn't affect the value of skills.
I get it, there's a strong anti-AI vibe here because of point 4. I'm somewhat worried, but I'm also really excited about the possiblilities, and I don't think we can avoid going to a fully AI future no matter how anti-AI you are; we can only figure out where to set the dial on how the rewards get shared.
you know we let social media companies cause massive harm over 2 decades; but clearly, this will be deifferent!