Comment by DocTomoe
12 hours ago
TL;DR: We had one group not do some things, an later found out that they did not learn anything by not doing the things.
This is a non-study.
12 hours ago
TL;DR: We had one group not do some things, an later found out that they did not learn anything by not doing the things.
This is a non-study.
no, that isn't accurate. One of the key points is that those previously relying on the LLM still showed reduced cognitive engagement after switching back to unaided writing.
No, it isn't.
The fourth session, where they tested switching back, was about recall and re-engagement with topics from the previous sessions, not fresh unaided writing. They found that the LLM users improved slightly over baseline, but much less than the non-LLM users.
"While these LLM-to-Brain participants demonstrated substantial improvements over 'initial' performance (Session 1) of Brain-only group, achieving significantly higher connectivity across frequency bands, they consistently underperformed relative to Session 2 of Brain-only group, and failed to develop the consolidation networks present in Session 3 of Brain-only group."
The study also found that LLM-group was largely copy-pasting LLM output wholesale.
Original poster is right: LLM-group didn't write any essays, and later proved not to know much about the essays. Not exactly groundbreaking. Still worth showing empirically, though.
And how exactly is that surprising?
If you wrote two essays, you have more 'cognitive engagement' on the clock as compared to the guy who wrote one essay.
In other news: If you've been lifting in the gym for a week, you have more physical engagement than the guy who just came in and lifted for the first time.
> And how exactly is that surprising?
Isn't the point of a lot of science to empirically demonstrate results which we'd otherwise take for granted as intuitive/obvious? Maybe in AI-literature-land everything published is supposed to be novel/surprising, but that doesn't encompass all of research, last I checked.
2 replies →