Comment by davisp
14 hours ago
> On correctness: LLMs may make mistakes. But these videos are code sitting in our repository, and we can maintain them. Every report and review becomes a fix in the source, and everyone who watches after that gets the corrected lecture. Every recorded lecture on the internet is stuck with its mistakes. Ours will continuously improve.
Models change. Models are unpredictable. The assumption that we can constrain them between prompt bumpers has not been proven and seems unlikely to be provable given how these model works.
Beyond that, how do you know if the reports and reviews are even correct? Are you just gonna rely on the kindness of others to provide that value? Of all things that LLM’s have killed, I’m pretty sure the open sharing of knowledge and understanding are the first on the chopping block. I don’t care whether or not you’re pro or anti LLM, publishing knowledge for reputation is now a dead end for anyone wanting to make a living.
Wrapping others’ knowledge in a black box that may or may not accurately represent that knowledge is basically enshitification on steroids. The reason we hold folks like Lovelace, Sagan, or Feynman in high regard is because they weren’t wrong when they shared their understanding and knowledge.
But the best of luck to everyone involved! I’m sure everything will work out!
> Wrapping others’ knowledge in a black box that may or may not accurately represent that knowledge...
You wrote it well. This aspect irks me, too.
> Humans change. Humans are unpredictable. The assumption that we can constrain them between syllabus and tenure review has not been proven and seems unlikely to be provable given how these humans work.
That's such an excellent argument against letting humans teach other humans! Especially unpredictable ones like Feynman. You should start a petition.
Sarcasm off. Now let me stick up for the guy I just made fun of you for arguing with, because his reply to you got flagged to death and most people here can't even see it.
Here's what he said, since it's the only place they actually engage your objection. They're not claiming the LLM is the authority on what's correct. Sina used to make these videos by hand on YouTube and it was slow, so now the LLM drafts and they review: "watch it, review it, correct it, improve the explanation, and put our name behind the result." A user report is a signal to go look at something, not truth. Weeks of production become hours of review. And STEM is a good target because linear algebra isn't going obsolete, so a fix stays fixed, and a lecture can accumulate improvements instead of freezing one teaching performance forever. (Item 49504080, if you have showdead on.)
That last part is the best thing anybody has said for this project, including them. But it doesn't answer what you asked. Who audits the audit? Who writes the bug reports they plan to inspect, now that publishing knowledge for a living is, as you put it, a dead end? "Put our name behind the result" is exactly right and it's the entire argument, and there's no way to check it while the sources and the lecture language are private -- especially with one lecture in this thread starting mid-sentence and ending abruptly, another springing an 8GB memory budget on you halfway through Bloom filters, and the emphasis landing on the wrong words at the crucial moments. Publish the repo and "we reviewed it" stops being a promise.
Somebody with showdead should vouch that comment back. It didn't deserve to die.
[flagged]