Comment by sho_hn
1 day ago
"Make a Bartosz-style website about $topic" seems like a fun benchmark idea. Maybe more so than pelicans on bicycles.
To be honest, though, this seems like ideal content for an LLM to produce. It's basically fact regurgitation.
> To be honest, though, this seems like ideal content for an LLM to produce. It's basically fact regurgitation.
You're trolling us, right? "Basically fact regurgitation" is all that teachers do after all. Have you ever noticed the difference between an inspirational teacher and a not-so-inspiring one in terms of effectiveness of communication and the "ah ha!" or lack of moments in your own understanding? If you can honestly say "no", then I might be able to understand your statement above, but really?
TBF; the infamous MS report 'Working with AI: Measuring the Occupational Implications of Generative AI' had Teacher/Professor pretty high up there.
> It's basically fact regurgitation.
This page wasn’t a regurgitation of facts. It was filled with custom interactive applets that let you explore the effects of physical changes. The core value proposition here is not the facts but the ability to explore and intuit the physics.
I do understand the contention is that an LLM would be less thoughtful in editorializing which bits to make interactive, reasoning about the progression in understanding and delight by the user.
I'm not so sure it's that far out of reach, though. From what I've seen the reasoning models do, they're not too far away from being able to run a strategy of figuring out interesting increments of a problem, parameterizing them, making an interactive scene for those parameters, ... it feels within reach.
I said nothing about LLMs. I said this page was not simply regurgitation of facts.
I personally doubt LLMs are close to producing anything like this, but that wasn’t the point. You indicated that this should be easy for an LLM because it’s just a fact dump. Regardless of whether some future LLM can generate something like this, it’s much more complicated and interesting than a simple fact dump.