Comment by sien
8 months ago
In Australia Universities that have remote study have places where people can do proctored exams in large cities. The course is done remotely but the exam, which is often 50%+ of the final grade, is done in a place that has proctored exams as a service.
Can't this be done in the US as well ?
The Open University in the UK started in 1969. Their staff have a reputation for good interaction with students, and I have seen very high quality teaching materials produced there. I believe they have always operated on the basis of remote teaching but on-site evaluation. The Open University sounds like an all-round success story and I'm surprised it isn't mentioned more in discussions of remote education.
Variations in this system are in active use in the US as well.
Do you feel it is effective?
It seems to me that there is a massive asymmetry in the war here: proctoring services have tiny incentives to catch cheaters. Cheaters have massive incentives to cheat.
I expect the system will only catch a small fraction of the cheating that occurs.
> I expect the system will only catch a small fraction of the cheating that occurs.
The main kind of cheating we need them to prevent is effective cheating - the kind that can meaningfully improve the cheater's score.
Requiring cheaters to put their belongings in a locker, using proctor-provided resources, and being monitored in a proctor-provided room puts substantial limits on effective cheating. That's pretty much the minimum that any proctor does.
It may not stop 100% of effective cheating 100% of the time, but it would make a tremendous impact in eliminating LLM-based cheating.
If you're worried about corrupt proctors, that's another matter. National brands that are both self- and externally-policed and depend on a good reputation to drive business from universities would help.
With this system, I expect that it would not take much to avoid almost all the important cheating that now occurs.
Remote proctoring programs at least are pretty rough these days. Their environment conditions are pretty exacting and then they expect you to just stare at the screen and think for basically the whole exam. Minor normal webcam problems can invalidate the entire exam through no fault of the examee or if you look around or fidget a lot it can trigger their cheat detection as well. I'm glad I finished my test taking time before it became the norm.
6 replies →
But it also only catches cheating on exams. For homework/projects, you can't really have that be in person.
3 replies →
> proctoring services have tiny incentives to catch cheaters. Cheaters have massive incentives to cheat.
If they don’t catch them they don’t have a business model. They have one job. The University of London, Open University and British Council all have 50+ years experience on proctoring university exams for distance learning students and it’s not like Thomson Prometric haven’t thought about how to do it either, even if they (mostly?) do computerised exams.
The problem is that the business model is that when you outsource compliance (in this case that might be catching cheaters), the important thing is to be able to say that everyone did their best, and you don't necessarily need to do your best to say that.
Well, if they don't catch someone. They don't have much incentive to avoid false positives. Catching someone who did not cheat but failed to follow all the draconian rules, is probably a lot easier than to catch an actual cheater.
And I daresay most of the corporate certs from companies like Microsoft and Red Hat are probably have pretty well-proctored exams too. To what degree their processes are applicable to a University environment I don't know.
1 reply →
Teachers typically also have years, sometimes decades, of experience running exams. Yet I've never seen a teacher that is any good at stopping cheating. And that's in person for the class that they are teaching.
1 reply →
> I expect the system will only catch a small fraction of the cheating that occurs.
It'll depend a lot on who/where/how is doing the screening and what tools (if any) are permitted.
Remember that bogus program for TI8{3,4} series calculators that would clear the screen and print "MEMORY CLEAR"? If the proctor was just looking for that string and not actually jumping through the hoops to _actually_ clear the memory then it was trivial to keep notes / solvers ... etc on the calculator.
It's actually somewhat of a challenge to display "Mem cleared" without access to the lowercase font. You have access to any uppercase character, spaces, and BASIC functions. With stat vars, you also get lowercase "a" "b" "c" "d" "e" and "r". And you can display text at a specific row and column.
I ended up displaying "M" "e" "min(" "c" "log(" "e" "a" "r" "e" "d". Then covered up the "in(" with spaces.
Then you lower your contrast for the full effect.
Was I at university in a small window in time when a TI-89 and TI-92 was allowed?
In the years since, I’ve only ever heard mention of older models, not newer ones which makes me wonder if this is a special case and situation where technology is frozen in time intentionally to foster learning.
4 replies →
Or just have two calculators and swap them
You can't stop people hiring someone who looks similar from sitting the exam, or messages in morse code via Bluetooth. It's hard to stop a palm card.
But it stops a casual cheater from having ChatGTP on a second device.
You can.
I did a remote proctored exam for the NREMT last year. They had me walk the camera around the room, under the desk, etc. All devices had to be in my backpack. No earbuds. They made me unplug the conference tv on the wall, lift picture frames etc. I had to keep my hands above the table the whole time, I couldn't look down if I was scratching an itch. They installed rootkit software and closed down all of the apps other than the browser running the test. They killed a daemon I run on my own pcs that is custom. They are recording from the webcam the whole time and have it angled so they can see. They record audio the whole time. I accidentally alt tabbed once and muted the mic with a wrong keyboard, those were first and second warning within 5 seconds.
When you take the test in a proctored testing center location they lock all of your stuff in a locker, check your hands, pockets, etc. They give you earplugs. You use their computer. They record you the whole time. They check your drivers license and take a fingerprint.
Those methods would stop a large % of your attack vectors.
As do the repercussions:
A candidate who violates National Registry policies, or the test center's regulations or rules, or engages in irregular behavior, misconduct and/or does not follow the test administrator's warning to discontinue inappropriate behavior may be dismissed from the test center. Exam fees for candidates dismissed from a test center will not be refunded. Additionally, your exam results may be withheld or canceled. The National Registry of EMTs may take other disciplinary action such as denial of National EMS Certification and/or disqualification from future National Registry exams.
At a minimum you're paying the $150 fee again, waiting another month to get scheduled and taking another 3 hours out of your day.
11 replies →
Both of those are so hard and so expensive that usually just learning the material is more practical.
LLMs and remote exams changed the equation so now cheating is incredibly easy and super effective compared to trying to morse code someone with a button in your shoe.
From what I've seen it works.
There is definitely a war between cheaters and people catching them. But a lot of people can't be bothered and if learning the material can be made easier than cheating then it will work.
You can imagine proctoring halls of the future being Faraday cages with a camera watching people do their test.
Local LLMs are almost here, no Internet needed!
2 replies →
Way back like 25 years ago in what we call high school in the US, my statistics teacher tried her damndest to make final exams fair. I said next to someone I had a huge crush on, and offered to take their exam for them. I needed a ‘c’ to ace the class, and she needed an ‘a’ to pass. 3 different tests and sets of questions/scantrons. I got her the grade she needed, she did not get me the grade I needed.
So to your point, it’s easy to cheat even if the proctor tries to prevent it.
I am confused by your pronouns and other plot holes.
You wanted to "ace the class", which is an "A" on your final report card? But your crush's exam tanked your grade? You passed the class anyway, right?
Did you swap Scantrons, then, and your crush sat next to you, writing answers on the dgfitz forms?
She wouldn't pass without an "A" on the exams, so her running point total was circling the drain, and your effort gave her a "C-" or something?
In what ways did your teacher make the exams "fair"? What percentage of the grade did they comprise?
Were the 3 tests administered on 3 separate occasions, so nobody caught you repeatedly cheating the same way?
3 replies →
If you've been to one of these testing centers, you'd realize it's not easy to cheat, and the companies that run them take cheating seriously. The audacity of someone to cheat in that environment would be exceptionally high, and just from security theater alone I suspect almost no actual cheating takes place.
I did a proctored exam for Harvard Extension at the British Council in Madrid. The staff is proctoring exams year-round for their in-house stuff so their motivation notwithstanding they know what they’re doing.
Where I'm studying its proctored-online. They have a custom browser and take over your computer while you're doing the exam. Creepy AF but saves travelling 1,300 km to sit an exam.
Wouldn't spending $300 on a laptop to cheat on an exam for a class you're paying thousands for make sense? It would probably improve your grade more than the text book.
You have to install an app that is a Bowser that at the same time locks the entire computer. Only this browser works. Install it, give it the needed admin permission and participate in your test or don't. This is also used in Australian schools for NAPLAN https://www.nap.edu.au/naplan/understanding-online-assessmen...
Can you tell us: Is "remote study" a relatively recent phenom in AU -- COVID era, or much older? I am curious to learn more. And, what is the history behind it? Was it created/supported because AU is so vast and many people a state might not live near the campus?
Also: I think your suggestion is excellent. We may see this happen in the US if AI cheating gets out of control (which it well).
It definitely existed before, particularly as a revenue stream for some of the smaller universities such as USQ. I think for the big ones it was a bit beneath them, then suddenly COVID came and we had lockdown for a long time in Melbourne. Now it's an expectation that students can access everything from home, but the flipside is everyone complains about how much campus life has declined. Students are paying more for a lower quality education and less amenity.
The same thing exists in South Africa, the university is called UNISA [1]. It has existed for a long time - my parents time. Lots of people that can't afford to go to university (as in, needs to earn an income) studies with them.
[1] - https://www.unisa.ac.za/sites/corporate/default
Not even just large cities. Decent sized towns have them too, usually with local high school teachers or the like acting as proctors.
Proctoring services done well could be valuable, but it’s smaller rural and remote communities that would benefit most. Maybe these services could be offered by local schools, libraries, etc.
It does feel like easy side money for local schools and teachers that will have empty classrooms after 5pm.
Depends on how many students would use the service. If its just a 1 or 2 at a time then its going to be quite expensive for those students.
nope. too much impact on profit.