Comment by math_dandy
8 months ago
I teach math at a large university (30,000 students) and have also gone “back to the earth”, to pen-and-paper, proctored and exams.
Students don’t seem to mind this reversion. The administration, however, doesn’t like this trend. They want all evaluation to be remote-friendly, so that the same course with the same evaluations can be given to students learning in person or enrolled online. Online enrollment is a huge cash cow, and fattening it up is a very high priority. In-person, pen-and-paper assessment threatens their revenue growth model. Anyways, if we have seven sections of Calculus I, and one of these sections is offered online/remote, then none of the seven are allowed any in person assessment. For “fairness”. Seriously.
I think you've identified the main issue here:
LLMs aren't destroying the University or the essay.
LLMs are destroying the cheap University or essay.
Cheap can mean a lot of things, like money or time or distance. But, if Universities want to maintain a standard, then they are going to have to work for it again.
No more 300+ person freshman lectures (where everyone cheated anyways). No more take-home zoom exams. No more professors checked out. No more grad students doing the real teaching.
I guess, I'm advocating for the Oxbridge/St. John's approach with under 10 class sizes where the proctor actually knows you and if you've done the work. And I know, that is not a cheap way to churn out degrees.
>I guess, I'm advocating for the Oxbridge/St. John's approach with under 10 class sizes where the proctor actually knows you and if you've done the work. And I know, that is not a cheap way to churn out degrees.
I could understand US tuition if that were the case. These days with overworked adjuncts make it McDonalds at Michelin star prices.
Funnily enough I only had 10-person-classes when I paid $125 for summer courses in a community college between expensive uni semesters.
1 reply →
Given that the adjuncts often aren't paid all that much better than the McDonalds workers...
Believe it or not, 300-person freshman lectures can be done well. They just need a talented instructor who's willing to put in the prep, and good TAs leading sections. And if the university fosters the right culture, the students mostly won't cheat.
But yeah, if the professor is clearly checked out and only interested in his research, and the students are being told that the only purpose of their education is to get a piece of paper to show to potential employers, you'll get a cynical death-spiral.
(I've been on both sides of this, though back when copy-pasting from Wikipedia was the way to cheat.)
> though back when copy-pasting from Wikipedia was the way to cheat
Back when I was teaching part time, I had a lot of fun looking at the confused looks on my students' faces when I said "you cannot use Wikipedia, but you'll find a lot of useful links at the bottom of any article there..."
Over here in Finland, higher education is state funded, and the funding is allocated to universities mostly based on how many degrees they churn out yearly. Whether the grads actually find employment or know anything is irrelevant.
So, it's pretty hard for universities over here to maintain standards in this GenAI world, when the paying customer only cares about quantity, and not quality. I'm feeling bad for the students, not so much for foolish politicians.
Gosh, I'm so myopic here. I'm mostly talking about US based systems.
But, of course, LLMs are affecting the whole world.
Yeah, I'd love to hear more about how other countries are affected by this tool. For Finland, I'd imagine that the feedback loop is the voters, but that's a bit too long and the incentives and desires of the voting public get a bit too condensed into a few choice to matter [0].
What are you seeing out there as to how students feel about LLMs?
[0] funnily enough, like how the nodes in the neural net of an LLM get too saturated if they don't have enough parameters.
1 reply →
After a short stint as a faculty member at a McU institution, I agree with much of this.
Provide machine problems and homework as exercises for students to learn, but assign a very low weight to these as part of an overall grade. Butt in seat assessments should be the majority of a course assessment for many courses.
>> (where everyone cheated anyways)
This is depressing. I'm late GenX, I didn't cheat in college (engineering, RPI), nor did my peers. Of course, there was very little writing of essays so that's probably why, not to mention all of our exams were in person paper-and-pencil (and this was 1986-1990, so no phones). Literally impossible to cheat. We did have study groups where people explained the homework to each other, which I guess could be called "cheating", but since we all shared, we tended to oust anyone who didn't bring anything to the table. Is cheating through college a common millenial / gen z thing?
Even before LLMs, if you walked into any frat and asked to see their test bank, you'd get thousands of files. Though not technically cheating, having every test a professor ever gave was a huge advantage. Especially since most profs would just reuse tests and HWs without any changes anyway.
To my generation, it wasn't that cheating was a 'thing' as much as it was impossible to avoid. Profs were so lazy that any semi-good test prep would have you discover that the profs were phoning it in and had been for a while. Things like not updating the course page with all the answers on them were unfortunately common. You could go and tell the prof, and most of us did, but then you'd be at a huge disadvantage relative to your peers who did download the answer key. Especially since the prof would still not update the questions! I want to make it clear: this is a common thing at R1 universities before LLMs.
The main issue is that at most R1s, the prof isn't really graded on their classes. That's maybe 5% of their tenure review. The thing they are most incentivized by is the amount of money they pull in from grants. I'm not all that familiar with R2 and below, but I'd imagine they have the same incentives (correct me if I'm wrong!). And with ~35% of students that go to R2 and below, the incentives for the profs for ~65% of students isn't well correlated with teaching said students.
6 replies →
Here's how cheating advanced since then.
1. People in the Greek system would save all homework sets and exams in a "library" for future members taking a given course. While professors do change (and a single professor will try to mix up problems) with enough time you eventually have an inventory of all the possible problems, to either copy outright or study.
2. Eventually a similar thing moved online, both with "black market" hired help, then the likes of Chegg Inc.
3. All the students in a course join a WhatsApp or Discord group and text each other the answers. (HN had a good blog about this from a data science professor, but I can't find it now. College cheating has been mentioned many times on HN).
Cheap "universities" are fine for accreditation. Exams can be administered via in-person proctoring services, which test the bare minimum. The real test would be when students are hired, in the probationary period. While entry-level hires may be unreliable, and even in the best case not help the company much, this is already a problem (perhaps it can be solved by the government or some other outside organization paying the new hire instead of the company, although I haven't thought about it much).
Students can learn for free via online resources, forums, and LLM tutors (the less-trustworthy forums and LLMs should primarily be used to assist understanding the more-trustworthy online resources). EDIT: students can get hands-on-experience via an internship, possibly unpaid.
Real universities should continue to exist for their cutting-edge research and tutoring from very talented people, because that can't be commodified. At least until/if AI reaches expert competence (in not just knowledge but application), but then we don't need jobs either.
> Real universities should continue to exist for their cutting-edge research and tutoring from very talented people, because that can't be commodified. At least until/if AI reaches expert competence (in not just knowledge but application), but then we don't need jobs either.
Okay, woah, I hadn't thought of that. I'm sitting here thinking that education for it's own sake is one of the reasons that we're trying to get rid of labor and make LLMs. Like, I enjoy learning and think my job gets in the way of that.
I hand't thought that people would want to just not do education of any sort anymore.
That's a little mind blowing.
4 replies →
There are excellent 1000-student lecture courses and shitty 15-student lecture courses. There are excellent take-home exams and shitty in-class exams. There are excellent grad student teaching assistants and shitty tenured credentialed professors. You can't boil quality down to a checklist.
No but you can observe and react to trends. Remote courses for me have me sitting directly at the Distraction 9000 (my computer) and rely entirely on "self discipline" in order for me to get anything out of it. This is fine for annual training that's utterly braindead and requires nothing from me but completing a basic quiz I get unlimited attempts for so my employer can tell whatever government agency I did the thing. If I want to actually get trained however, I always do in-person, both because my employer covers those expenses and who in the world turns down free travel, and because I retain nothing from remote learning. Full stop.
Of course that's only my experience and I can't speak for all of humanity. I'm sure people exist who can engage in and utilize remote learning to it's full potential. That said I think it's extremely tempting to lean on it to get out of providing classrooms, providing equipment, and colleges have been letting the education part of their school rot for decades now in favor of sports and administrative bloat, so forgive me if I'm not entirely trusting them to make the "right" call here.
Edit: Also on further consideration, remote anything but teaching very much included also requires a level of tech literacy that, at least in my experience, is still extremely optimistic. The number of times we have to walk people through configuring a microphone, aiming a webcam, sharing to the meeting, or the number of missed participants because Teams logged them out, or Zoom bugged out on their machine, or whatever. It just adds a ton of frustration.
1 reply →
I think this is where it's going to end up.
The masses get the cheap AI education. The elite get the expensive, small class, analog education. There won't be a middle class of education, as in the current system - too expensive for too little gain.
10 is a small number. There's a middle ground. When I studied, we had lectures for all students, and a similar amount of time in "work groups," as they were called. That resembled secondary education: one teacher, around 30 students, but those classes were mainly focused on applying the newly acquired knowledge, making exercises, asking questions, checking homework, etc. Later, I taught such classes for programming 101, and it was perfectly doable. Work group teachers were also responsible for reviewing their students' tests.
But that commercially oriented boards are ruining education, that's a given. That they would stoop to this level is a bit surprising.
Very common. Large lecture with a professor, and small "discussion sections" with a grad student for Q/A, homework help, exam review.
All of my classes with a dozen students were better than all of my classes with 2 dozen. My favorite class had 7 students.
Oxbridge supervisinons/tutorials are typically two students, and at a push three (rarely)
certainly not anywhere close to ten!
Thanks for the clarification there!
Yeah, 1:3 teacher to student ratio would make university a lot more expensive
All degrees are basically the same though and of 95% of the value is signaling nobody really cares about the education part
I see that pressure as well. I find that a lot of the problems we have with AI are in fact AI exposing problems in other aspects of our society. In this case, one problem is that the people who do the teaching and know what needs to be learned are the faculty, but the decisions about how to teach are made by administrators. And another problem is that colleges are treating "make money" as a goal. These problems existed before AI, but AI is exacerbating them (and there are many, many more such cases).
I think things are going to have to get a lot worse before they get better. If we're lucky, things will get so bad that we finally fix some shaky foundations that our society has been trying to ignore for decades (or even centuries). If we're not lucky, things will still get that bad but we won't fix them.
Instructors and professors are required to be subject matter experts but many are not required to have a teaching certification or education-related degree.
So they know what students should be taught but I don't know that they necessarily know how any better than the administrators.
I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
I attended two universities to get my computer science degree. The first was somewhat famous/prestigious, and I found most of the professors very unapproachable and cared little about "teaching well". The second was a no-name second tier public uni, but I found the professors much more approachable, and they made more effort to teach well. I am still very conflicted about that experience. Sadly, the students were way smarter at the first uni, so the intellectual rigor of discussions was much higher than my second uni. My final thoughts: "You win some; you lose some."
16 replies →
>I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
There is a lot more on the plate when you are kindergarten teacher - as the kids needs a lot of supervision and teaching outside the "subject" matters, basic life skills, learning to socialize.
Conversely, at a university the students should generally handle their life without your supervision, you can trust that all of them are able to communicate and to understand most of what you communicate to them.
So the subject matter expertise in kidnergartens is how to teach stuff to kids. Its not about holding a fork, or to not pull someones hair. Just as the subject matter expertise in an university can be maths. You rarely have both, and I don't understand how you suggest people get both a phd in maths, do enough research to get to be a professor and at the same time get a degree in education?
1 reply →
Just watch out for who is certifying how things should be taught. It’s honestly one reason education is so bad and so slow to change.
Edit: and why perfectly capable professionals can’t be teachers without years of certification
> I've always found it weird that you need teaching certification to teach basic concepts to kindergartners but not to teach calculus to adults.
I think this is partially due to the age of the students, by the time you hit college the expectation is you can do a lot of the learning yourself outside of the classroom and will seek out additional assistance through office hours, self study, or tutors/classmates if you aren't able to understand from the lecture alone.
It's also down to cost cutting, instead of having entirely distinct teaching and research faculty universities require all professors to teach at least one class a semester. Usually though the large freshman and sophomore classes do get taught by quasi dedicated 'teaching' professors instead of a researcher ticking a box.
>don't know that they necessarily know how any better than the administrators.
If someone is doing something day in and day out, they do gain knowledge on what works and doesn't work. So just by doing that the professors typically know much more about how people should be taught than the administrators. Further, the administrators' incentives are not aligned towards insuring proper instruction. They are aligned with increasing student enrollment and then cashing out whenever they personally can.
This is very different in France. Studying to be a teacher at university level is a big deal.
2 replies →
Once you're an adult some of the best lessons come from having bad teachers.
2 replies →
Nobody knows "how" things should be taught. Pedagogy is utter disaster.
2 replies →
A PhD was historically a teaching degree: that’s what the D stands for.
6 replies →
The instructors may not know the absolute best way to teach, but I think they do know more than the administrators. All my interaction with teacher training suggests to me that a large proportion of it is basically vacuous. On dimensions like the ones under discussion here (e.g., "should we use AI", "can we do this class online"), there is not really anything to "know": it's not like anyone is somehow a super expert on AI teaching. Teacher training in such cases is mostly just fads with little substantive basis.
Moreover, the same issues arise even outside a classroom setting. A person learning on their own from a book vs. a chatbot faces many of the same problems. People have to deal with the problem of AI slop in office emails and restaurant menus. The problem isn't really about teaching, it's about the difficulty of using AI to do anything involving substantive knowledge and the ease of using AI to do things involving superficial tasks.
I totally agree. I think the neo-liberal university model is the real culprit. Where I live, Universities get money for each student who graduates. This is up to 100k euros for a new doctorate. This means that the University and its admin want as many students to graduate as possible. The (BA&MA) students also want to graduate in target time: if they do, they get a huge part of their student loans forgiven.
What has AI done? I teach a BA thesis seminar. Last year, when AI wasn't used as much, around 30% of the students failed to turn in their BA thesises. 30% drop-out rate was normal. This year: only 5% dropped out, while the amount of ChatGPT generated text has skyrocketed. I think there is a correlation: ChatGPT helps students write their thesises, so they're not as likely to drop out.
The University and the admins are probably very happy that so many students are graduating. But also, some colleagues are seeing an upside to this: if more graduate, the University gets more money, which means less cuts to teaching budgets, which means that the teachers can actually do their job and improve their courses, for those students who are actually there to learn. But personally, as a teacher, I'm at loss of what to do. Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be? Nobody else seems to care. Or should I pass them, let them graduate, and reserve my energy to teach those who are motivated and are willing to engage?
I think one of the outcomes might be a devaluation of the certifications offered in the public job marketplace.
8 replies →
> Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be?
No, you should fail them for turning in bad theses, just like you would before AI.
12 replies →
You should fail them.
The larger work that the intellectual and academic forces of a liberal democracy is that of “verification”.
Part of the core part of the output, is showing that the output is actually what it claims to be.
The reproducibility crisis is a problem Precisely because a standard was missed.
In a larger perspective, we have mispriced facts and verification processes.
They are treated as public goods, when they are hard to produce and uphold.
Yet they compete with entertainment and “good enough” output, that is cheaper to produce.
The choice to fail or pass someone doesn’t address the mispricing of the output. We need new ways to address that issue.
Yet a major part of the job you do. is to hold up the result to a standard.
You and the institutions we depend on will continue to be crushed by these forces. Dealing with that is a separate discussion from the pass or fail discussion.
Fail them. Only let the ai generated text that has been verified and edited to be true to pass.
If they want to use AI make them use it right.
> Some thesises had hallucinated sources, some had AI slop blogs as sources, the texts are robotic and boring. But should I fail them, out of principle on what the ideal University should be?
I don't think you should fail them - instead, give them feedback on how to improve their thesis themselves, and how to make better use of tools like ChatGPT.
If instead of flat out failing to turn in their thesis, instead they are submitting work that needs more iteration due to bad use of AI, that sounds like a net win to me. The latter can be turned into something useful.
In Australia Universities that have remote study have places where people can do proctored exams in large cities. The course is done remotely but the exam, which is often 50%+ of the final grade, is done in a place that has proctored exams as a service.
Can't this be done in the US as well ?
The Open University in the UK started in 1969. Their staff have a reputation for good interaction with students, and I have seen very high quality teaching materials produced there. I believe they have always operated on the basis of remote teaching but on-site evaluation. The Open University sounds like an all-round success story and I'm surprised it isn't mentioned more in discussions of remote education.
Variations in this system are in active use in the US as well.
Do you feel it is effective?
It seems to me that there is a massive asymmetry in the war here: proctoring services have tiny incentives to catch cheaters. Cheaters have massive incentives to cheat.
I expect the system will only catch a small fraction of the cheating that occurs.
> I expect the system will only catch a small fraction of the cheating that occurs.
The main kind of cheating we need them to prevent is effective cheating - the kind that can meaningfully improve the cheater's score.
Requiring cheaters to put their belongings in a locker, using proctor-provided resources, and being monitored in a proctor-provided room puts substantial limits on effective cheating. That's pretty much the minimum that any proctor does.
It may not stop 100% of effective cheating 100% of the time, but it would make a tremendous impact in eliminating LLM-based cheating.
If you're worried about corrupt proctors, that's another matter. National brands that are both self- and externally-policed and depend on a good reputation to drive business from universities would help.
With this system, I expect that it would not take much to avoid almost all the important cheating that now occurs.
11 replies →
> proctoring services have tiny incentives to catch cheaters. Cheaters have massive incentives to cheat.
If they don’t catch them they don’t have a business model. They have one job. The University of London, Open University and British Council all have 50+ years experience on proctoring university exams for distance learning students and it’s not like Thomson Prometric haven’t thought about how to do it either, even if they (mostly?) do computerised exams.
6 replies →
> I expect the system will only catch a small fraction of the cheating that occurs.
It'll depend a lot on who/where/how is doing the screening and what tools (if any) are permitted.
Remember that bogus program for TI8{3,4} series calculators that would clear the screen and print "MEMORY CLEAR"? If the proctor was just looking for that string and not actually jumping through the hoops to _actually_ clear the memory then it was trivial to keep notes / solvers ... etc on the calculator.
7 replies →
You can't stop people hiring someone who looks similar from sitting the exam, or messages in morse code via Bluetooth. It's hard to stop a palm card.
But it stops a casual cheater from having ChatGTP on a second device.
13 replies →
From what I've seen it works.
There is definitely a war between cheaters and people catching them. But a lot of people can't be bothered and if learning the material can be made easier than cheating then it will work.
You can imagine proctoring halls of the future being Faraday cages with a camera watching people do their test.
3 replies →
Way back like 25 years ago in what we call high school in the US, my statistics teacher tried her damndest to make final exams fair. I said next to someone I had a huge crush on, and offered to take their exam for them. I needed a ‘c’ to ace the class, and she needed an ‘a’ to pass. 3 different tests and sets of questions/scantrons. I got her the grade she needed, she did not get me the grade I needed.
So to your point, it’s easy to cheat even if the proctor tries to prevent it.
4 replies →
If you've been to one of these testing centers, you'd realize it's not easy to cheat, and the companies that run them take cheating seriously. The audacity of someone to cheat in that environment would be exceptionally high, and just from security theater alone I suspect almost no actual cheating takes place.
I did a proctored exam for Harvard Extension at the British Council in Madrid. The staff is proctoring exams year-round for their in-house stuff so their motivation notwithstanding they know what they’re doing.
Where I'm studying its proctored-online. They have a custom browser and take over your computer while you're doing the exam. Creepy AF but saves travelling 1,300 km to sit an exam.
Wouldn't spending $300 on a laptop to cheat on an exam for a class you're paying thousands for make sense? It would probably improve your grade more than the text book.
1 reply →
Can you tell us: Is "remote study" a relatively recent phenom in AU -- COVID era, or much older? I am curious to learn more. And, what is the history behind it? Was it created/supported because AU is so vast and many people a state might not live near the campus?
Also: I think your suggestion is excellent. We may see this happen in the US if AI cheating gets out of control (which it well).
It definitely existed before, particularly as a revenue stream for some of the smaller universities such as USQ. I think for the big ones it was a bit beneath them, then suddenly COVID came and we had lockdown for a long time in Melbourne. Now it's an expectation that students can access everything from home, but the flipside is everyone complains about how much campus life has declined. Students are paying more for a lower quality education and less amenity.
The same thing exists in South Africa, the university is called UNISA [1]. It has existed for a long time - my parents time. Lots of people that can't afford to go to university (as in, needs to earn an income) studies with them.
[1] - https://www.unisa.ac.za/sites/corporate/default
Not even just large cities. Decent sized towns have them too, usually with local high school teachers or the like acting as proctors.
Proctoring services done well could be valuable, but it’s smaller rural and remote communities that would benefit most. Maybe these services could be offered by local schools, libraries, etc.
It does feel like easy side money for local schools and teachers that will have empty classrooms after 5pm.
1 reply →
nope. too much impact on profit.
> Students don’t seem to mind this reversion.
Those I ask are unanimously horrified that this is the choice they are given. They are devastated that the degree for which they are working hard is becoming worthless yet they all assert they don't want exams back. Many of them are neurodivergent who do miserably in exam conditions and in contrast excel in open tasks that allow them to explore, so my sample is biased but still.
They don't have a solution. As the main victims they are just frustrated by the situation, and at the "solutions" thrown at it by folks who aren't personally affected.
It is always interesting to me when people say they are "bad test takers". You mean you are bad at the part where we find out how much you know? Maybe you just don't know the material well enough.
caveat emptor - I am not ND so maybe this is a real concern for some, but in my experience the people who said this did not know the material. And the accommodations for tests are abused by rich kids more than they are utilized by those that need them.
As a self proclaimed bad test taker, it's not that I don't know the information. It's that I am capable of second guessing myself in a particular way in which I can build a logical framework to suggest another direction or answer.
This presents itself as a bad test taker, I rarely ever got above a B+ on any difficult test material. But you put me in a lab, and that same skillset becomes a major advantage.
Minds come in a variety of configurations, id suggest considering that before taking your own experience as the definitive.
datum: I'm ND, but I'm a good test-taker. There were plenty of tests for subjects where I didn't need to study because I was adept at reading the question and correctly assuming what the test-creator wanted answered, and using deduction to reduce possibilities down enough that I could be certain of an answer - or by using meta-knowledge of where the material from the recent lectures was to narrow things down, again, not because I knew the material all that well but because I could read the question. Effectively, I had a decent grasp of the "game" of test-taking, which is rather orthogonal to the actual knowledge of the class material.
I think the reverse exists as well. I think I am a much better test taker than average, and this has very clearly given me some advantages that come from the structure of exam-focused education. Exam taking is a skill and it's possible to be good at it, independent of the underlying knowledge. Of course knowing the material is still required.
However you are correct in noticing that there are an anomalously high number of "bad test takers" in the world. Many students are probably using this as a flimsy excuse for poor performance. Overall I think the phenomenon does exist.
Tests are just a proxy for understanding and/or application of a concept. Being good at the proxy doesn’t necessarily mean you understand the concept, just like not being good at the proxy doesn’t mean you don’t. Finding other proxies we can use allows for decoupling knowledge from a specific proxy metric.
If I was evaluating the health of various companies, I wouldn’t use one metric for all of them, as company health is kind of an abstract concept and any specific metric would not give me a very good overall picture and there are multiple ways for a company to be healthy/successful. Same with people.
There are lots of different ways to utilize knowledge in real world scenarios, so someone could be bad at testing and bad at some types of related jobs but good at other types of related jobs. So unless “test taking” as a skill is what is being evaluated, it isn’t necessary to be the primary evaluation tool.
I don't think I understand, as a terrible test taker myself.
The solution I use when teaching is to let evaluation primarily depend on some larger demonstration of knowledge. Most often it is CS classes (e.g. Machine Learning), so I don't really give much care for homeworks and tests and instead be project driven. I don't care if they use GPT or not. The learning happens by them doing things.
This is definitely harder in other courses. In my undergrad (physics) our professors frequently gave takehome exams. Open book, open notes, open anything but your friends and classmates. This did require trust, but it was usually pretty obvious when people worked together. They cared more about trying to evaluate and push us if we cared than if we cheated. They required multiple days worth of work and you can bet every student was coming to office hours (we had much more access during that time too). The trust and understanding that effort mattered actually resulted in very little cheating. We felt respected, there was a mutual understanding, and tbh, it created healthy competition among us.
Students cheat because they know they need the grade and that at the end of the day they won't won't actually be evaluated on what they learned, but rather on what arbitrary score they got. Fundamentally, this requires a restructuring, but that's been a long time coming. The cheating literally happens because we just treated Goodhart's Law as a feature instead of a bug. AI is forcing us to contend with metric hacking, it didn't create it.
> Many of them are neurodivergent who do miserably in exam conditions
Isn't this part of life? Learning to excel anyway?
Life doesn't tend to take place under exam conditions, either.
16 replies →
I don't think so? I teach maths, not survival or social pressure. If a student in my class is a competent mathematician why should they not be acknowledged to be that?
9 replies →
IMO exams should be on the easier side and not require much computing (mainly knowledge, and not unnecessary memorization). They should be a baseline, not a challenge for students who understand the material.
Students are more accurately measured via long, take-home projects, which are complicated enough that they can’t be entirely done by AI.
Unless the class is something that requires quick thinking on the job, in which case there should be “exams” that are live simulations. Ultimately, a student’s GPA should reflect their competence in the career (or possible careers) they’re in college for.
> They should be a baseline, not a challenge for students who understand the material.
You've made this normative statement but not explained why.
I think exams should not require huge amounts of computation (I agree) but should contain a range of questions - from easy to very difficult - so that the best and average students can be differentiated.
1 reply →
We have an Accessible Testing Center that will administer and proctor exams under very flexible conditions (more time, breaks, quiet/privacy, …) to help students with various forms of neurodivergence. They’re very good and offer a valuable service without placing any significant additional burden on the instructor. Seems to work well, but I don’t have first hand knowledge about how these forms of accommodations are viewed by the neurodivergent student community. They certainly don’t address the problem of allowing « explorer » students to demonstrate their abilities.
Yes I think the issue is as much that open tasks make learning interesting and meaningful in a way that exams hardly can do.
This is the core of the issue really. If we are in the business of teaching, as in making people learn, exams are a pretty blunt and ineffective instrument. However since our business is also assessing, proctoring is the best if not only trustworthy approach and exams are cheap in time, effort and money to do that.
My take is that we should just (properly) assess students at the end of their degree. Spend time (say, a full day) with them but do it only once in the degree (at the end), so you can properly evaluate their skills. Make it hard so that the ones who graduate all deserve it.
Then the rest of their time at university should be about learning what they will need.
5 replies →
I’ve had access to that at my school and it’s night and day. Not being as stressed about time and being in a room alone bumps me up by a grade letter at least.
> Many of them are neurodivergent who do miserably in exam conditions
I mean, for every neurodivergent person who does miserably in exam conditions you have one that does miserably in homework essays because of absence of clear time boundaries.
Autism vs. ADHD
1 reply →
>Many of them are neurodivergent
if "many" are "divergent" then... are they really divergent? or are they the new typical?
Many of the students I talk to. I don't claim they form a representative sample of the student cohort, on the contrary. I guess that the typical student is typical but I have not gone to check that.
I think having one huge exam at the end is the problem. An exam and assessment every week would be best.
Less stress at the end of the term, and the student can't leave everything to the last minute, they need to do a little work every week.
Too much proctoring and grading, not enough holding students' hands for stuff they should have learned from reading the textbook.
In my undergraduate experience, the location of which shall remain nameless, we had amble access to technology but the professors were fairly hostile to it and insisted on pencil and paper for all technical classes. There were some English or History classes here and there that allowed a laptop for writing essays during an "exam" that was a 3 hour experience with the professor walking around the whole time. Anyway, when I was younger I thought the pencil and paper thing to be silly. Why would we eschew brand new technology that can make us faster! And now that I'm an adult, I'm so thankful they did that. I have such a firm grasp of the underlying theory and the math precisely because I had to write it down, on my own, from memory. I see what these kids do today and they have been so woefully failed.
Teachers and professors: you can say "no". Your students will thank you in the future.
I have a Software Engineering degree from Harvard Extension and I had to take quite a few exams in physically proctored environments. I could very easily manage in Madrid and London. It is not too hard for either the institution or the student.
I am now doing an Online MSc in CompSci at Georgia Tech. The online evaluation and proctoring is fine. I’ve taken one rather math-heavy course (Simulation) and it worked. I see the program however is struggling with the online evaluation of certain subjects (like Graduate Algorithms).
I see your point that a professor might prefer to have physical evaluation processes. I personally wouldn’t begrudge the institution as long as they gave me options for proctoring (at my own expense even) or the course selection was large enough to pick alternatives.
Professional proctored testing centers exist in many locations around the world now. It's not that complicated to have a couple people at the front, a method for physically screening test-takers, providing lockers for personal possessions, providing computers for test administration, and protocols for checking multiple points of identity for each test taker.
This hybrid model is vastly preferable to "true" remote test taking in which they try to do remote proctoring to the student's home using a camera and other tools.
That’s what I did at HES and it was fine. Reasonable and not particularly stressful.
is it ok for students to submit images of hand-written solutions remotely?
seriously it reminds me of my high school days when a teacher told me i shouldn’t type up my essays because then they couldn’t be sure i actually wrote them.
maybe we will find our way back to live oral exams before long…
Business models rule us all. Have you tested what kind of pushback you'll receive if you happen to flout the remote rule?
Centralization and IT-ification has made flouting difficult. There’s one common course site on the institution’s learning management system for all sections where assignments are distributed and collected via upload dropbox, where grades are tabulated and communicated.
So far, it’s still possible to opt out of this coordinated model, and I have been. But I suspect the ability to opt out will soon come under attack (the pretext will be ‘uniformity == fairness’). I never used to be an academic freedom maximalists who viewed the notion in the widest sense, but I’m beginning to see my error.
Sorry to hear this. And thanks for sharing this warning to other educators. I hope you find a way through.
I attended Purdue. Since I graduated, it launched its "Purdue Global" online education. Rankings don't suggest it's happened yet, but I'm worried it will cheapen the brand and devalue my degree.
I remember sitting with the faculty in charge of offering online courses when I visited as an alum back in 2014. They seemed to look at it as a cash cow in their presentation. They were eager to be at the forefront of online CS degrees at the time.
Higher ups say yes to remote learning and no to remote work. Interesting to see this side by side like this.
Remote learning also opens up a lot of opportunities to people that would not otherwise be able to take advantage of them. So it's not _just_ the cash cow that benefits from it.
Yeah, the thing AI cheating is it seems inherent not in teaching but what mechanical, bureaucratic, for-profit teaching and universities have become.
Some US universities do this remotely via proctoring software. They require pencil and paper to be used with a laptop that has a camera. Some do mirror scans, room scans, hand scans, etc. The Georgia Tech OMS CS program used to do this for the math proofs course and algorithms (leet code). It was effective and scalable. However, the proctoring seems overly Orwellian, but I can understand the need due to cheating as well as maintaining high standards for accreditation.
> seems overly Orwellian
Wow.
Maybe we should consider the possibility that this isn't a good idea? Just a bit? No? Just ignore how obviously comparable this is to the most famous dystopian fiction in literary history?
Just wow. If you're willing to do that, I don't know what to tell you.
Stanford requires pen & paper exams for their remote students; the students first need to nominate an exam monitor (a person) who in turn receives and prints the assignments, meets the student at an agreed upon place, the monitor gives them the printed exams and leaves, then collects the exam after allotted time, scans it and sends it back to Stanford.
So just have test centers, and flip the classroom.
I think this is a good approach.
Thanks for sharing this anecdote. It’s easy to forget the revenue / business side of education and that universities are in a hard spot here.
Thank you for not giving in. The slide downhill is so ravenous and will consume so much of our future until the wise intervene.
why not pay for students to take the pen and paper exams at some proctored location, perhaps independent of the university.
Can't we use AI to monitor the students?
I'm not a teacher, but I came here to say the same thing. Pen and paper.
Capitalism and the constant thirst for growth is killing society. Since when did universities care almost solely about renevnue and growth?
> Since when did universities care almost solely about renevnue and growth?
Since endowments got huge.
Could you explain this more? At first glace, a large endowment should either free you from worrying about revenue or move your focus to managing an endowment with a school as a side hustle.
2 replies →
That’s a magnifier but it shouldn’t be the cause; for that you need a shift in management culture from optimizing for academic missions to optimizing for careers/influence of management and trustees.
2 replies →
With the us government now going after their funding they may have to start caring even more
When it was generally accepted by our society that the goal of all work is victory, not success. Capitalism frames everything as a competition, even when collaboration is obviously superior. Copyright makes this an explicit rule.
Hand written essays are inherently ableist. I would be at a massive disadvantage. I grew up during the 60's, but handwriting was alway slow and error prone for me. As soon as I could use a word processor I blossomed.
It's probably not as bad for mathematical derivations. I still do those by hand since they are more like drawing than expression.
> Hand written essays are inherently ableist
So is testing; people who don't have the skills don't do well. Hell, the entire concept of education is ableist towards learning impaired kids. Let's do away with it entirely.
Would you hire someone as a writer who is completely illiterate? Of course that's an extreme edge case, but at some point equality stops and the ability to do the work is actually important.
Most people would be happy to hire a writer with no consideration of how good their handwriting was.
I was a slow handwriter, too. I always did badly on in-class essay exams because I didn't have time to write all that I knew needed to be said. What saved my grade in those classes was good term papers.
Having had much occasion to consider this issue, I would suggest moving away from the essay format. Most of the typical essay is fluff that serves to provide narrative cohesion. If knowledge of facts and manipulation of principles are what is being evaluated, presentation by bullet points should be sufficient.
> Hand written essays are inherently ableist
Doing anything is inherently based on your ability to do it. Running is inherently ableist. Swimming is ableist. Typing is inherently ableist.
Pointing this out is just a thought terminating cliche. Ok, it's ableist. So?
> As soon as I could use a word processor I blossomed.
You understand this is inherently ableist to people that can't type?
> I still do those by hand since they are more like drawing than expression.
Way to do ableist math.
> Hand written essays are inherently ableist.
yes.
> I would be at a massive disadvantage.
yes.
...but.
how would you propose to filter out able cheaters instead? there's also in person one on one verbal exam, but economics and logistics of that are insanely unfavorable (see also - job interviews.)
Handwriting essays doesn't filter out cheaters though? It didn't even filter out cheaters before ChatGPT, before it was just a person writing the essay for you that you would copy