Comment by blindstitch
3 years ago
I taught a 102 course last semester with 13 weekly very short writing assignments. The level of cheating was pretty demoralizing. In one week, 15 of 75 students all used ChatGPT. Luckily when you are grading that many at a time it becomes very clear who is cheating based on the overlap between them and structural similarity, though they do try quite hard to muss it up to look original.
In general I found those tools mostly useless and very easy to fool by making minor changes. I had better luck putting the original writing prompts into chatGPT, saving them all to a text file, and cross referencing when dealing with student work that didn't pass a sniff test.
Repeat after me - detectors are snakeoil. They sell you the idea you can avoid the disruption you cannot. Not only is it trivial to bypass detection, including more sophisticated not yet implemented watermarks, the technology also biases against non native speakers.
I am half tempted to show students how easy it is to fool the detection. Turn something into a parenthetical, remove red flag phrases, cut out fluff. Or better yet, just use it as a skeleton draft and rewrite it in your own words. Then I would show them how easy it is for me to see through their fudging, which I do by just noting down the similarities between papers and the information inside the GPT response. In some cases there is 100% overlap between the two. They always repeat certain words, too, so I use ripgrep to see who else used them and do cross sectional comparisons. But I also think for now it's a good idea to take the magician's code on that part of the process, at least when communicating with the students directly.
All of this is the kind of detective work that most graduate students are not qualified to perform easily.
Unfortunately this will also catch students who write clear and well structured essays. My friend was taking a summer course on her break from college. Her professor accused her of cheating because her work was a clear outlier. It was a clear outlier because she was a significantly better writer.
A better check is that there’ll be a mismatch between content and citations, or the citations will be entirely hallucinated. As far as I can tell ChatGPT is incapable of generating actual citations.
The outliers consistently write better prose but they never include nearly as much distinct points of information within their responses. They also have way less fluff. They include personal anecdotes. That professor probably didn't have enough materials of hers to work with.
As for checking hallucinations I definitely do not have the time to go that deep on 75 submissions a week. If that could be automated it'd be helpful. The students are correct that I can afford to spend only about 5 minutes per essay on grading (they're short, 200-250 words).
next up: CiteGPT
plug in your finished work, making up whatever stats and arguments and assertions suits your taste or agenda, and CiteGPT will back-solve matching snippets that support your writing, inserting all in line citations and building a reference list automatically. Maybe even suggest minor tweaks to make your arguments match the available supporting material.
This problem has such a simple solution: have the kids write some in-class essays throughout the term. Then, if cheating is suspected on a homework assignment, the teacher cross-references against the known GPT-free work to compare its quality.
It does mean that good writers may be able to get away with cheating, but then they really didn’t need the practice to begin with.
The students just use GPT within class because they are completely shameless. I'd have to sit behind them and watch their screens like a prison guard. I haven't resigned myself to that so far.
I think the more reasonable solution is to just wait for the university to take active measures against it, and accept that this is just a new development in how students cheat, which they have always done. The fact that the administrators are dumping this all on teachers and TAs is totally unfair to the actual workers.
2 replies →