Comment by mattlondon
2 years ago
That is an interesting idea. Learn to love the bomb: mark the prompt not the essay?
"Your assignment class is to write a comprehensive 250 word prompt that covers the main topics we've been discussing about the causes for WW1. You should assess the resulting essay to ensure that it covers X, Y, and Z in it's output, being sure to discuss foo and bar. Write a brief 500 word analysis of what you tried and your prompt's effectiveness and be prepared to discuss in class what worked well and what you needed to change in your prompt."
What's stopping students from using an LLM to write the prompt and analysis? Mentioning the topics discussed in class and any key points could easily be part of the initial prompt.
The in-person discussion is key, but at that point why do you need a written essay?
These tools radically change the concept of knowledge. If information can be looked up at any moment, why does it need to exist in our brains?
We can argue that knowledge enhances our reasoning capabilities, but our entire education system is not built for assessing reasoning, but how good we are at recalling information. So we need to make core changes to how we approach education if we ever want to coexist with this technology.
Well the point is to make them think about the prompt. I don't care if someone prompts to come up with a second prompt to get the answer, they need to input and refine to make it fit the goal. So by setting fairly stringent objectives for what the AI needs to generate, you need to be quite specific to engineer the prompt and think about what it needs. And then you need to look at what it said and qualify it somehow to decide if it is good enough - sure you could use a LLM for that too, but what is the prompt for that? You'll need to teach the second LLM what to analyse in the output of the first LLM to meet the requirements of the assignment, so the prompt(s) becomes the essays of yesterday in terms of academic work/thought for the student.
That's the point of what I was getting at - they're going to use AI regardless, so at least work with it rather than against it.
Right. But what I mean is if the final evaluation will require an in-person discussion, then couldn't we skip the text generation step, and just have the discussion? Homework as a tool to make students think needs to be drastically different, since students will always find loopholes to avoid it, and we can't evaluate it in isolation anymore (i.e. a teacher can no longer grade it on its own).
Doing live discussions obviously requires much more time and effort, but it is also much more engaging than doing homework, so the chance of actually learning something is higher.
There's also an argument to be made that homework in general isn't conducive to learning, and that education should take place primarily in schools. When you consider how much future generations will rely on AI in their everyday lives, this argument becomes even more compelling.
So I think this is our chance to drastically change how we think about education, rather than try to shoehorn a groundbreaking technology into our outdated methodologies.
>If information can be looked up at any moment, why does it need to exist in our brains?
Because creativity involves making connections between seemingly unrelated knowledge. For a good example, consider Richard F. Lyon's 1980 optical mouse design:
http://bitsavers.trailing-edge.com/pdf/xerox/parc/techReport...
One important innovation here is lateral inhibition between neighboring sensor elements, which gives fast and cheap contrast enhancement and automatic gain control. This was inspired by the behavior of cells in biological eyes. Lyon needed both biology and electronic engineering knowledge simultaneously to make the connection. Looking things up wouldn't have helped because it would have only provided one field of knowledge at a time.
> Because creativity involves making connections between seemingly unrelated knowledge.
Sure. But don't you think that AI can make those connections for us, or even be creative on its own? Why should we value pure human creativity over human+machine creativity, or pure machine creativity?
This technology poses these fundamental questions, and we're far from being able to answer them as a society. How it impacts education seems like a core concern, and might be a good starting point.
I think you are thinking in the right direction. Don’t fight gravity (AI) - gravity will win. Instead, change the system so that we (and the students) work with gravity rather than against it.