Comment by kuschku
5 days ago
If only 5-10 people have ever tried to solve something in programming, every LLM will start regurgitating your own decade-old attempt again and again, sometimes even with the exact comments you wrote back then (good to know it trained on my GitHub repos...), but you can spend upwards of 100mio tokens in gemini-cli or claude code and still not make any progress.
It's afterall still a remix machine, it can only interpolate between that which already exists. Which is good for a lot of things, considering everything is a remix, but it can't do truly new tasks.
What is a "truly new task"? Does there exist such a thing? What's an example of one?
Everything we do builds on top of what's already been done. When I write a new program, I'm composing a bunch of heuristics and tricks I've learned from previous programs. When a mathematician approaches an open problem, they use the tactics they've developed from their experience. When Newton derived the laws of physics, he stood on the shoulders of giants. Sure, some approaches are more or less novel, but it's a difference in degree, not kind. There's no magical firebreak to separate what AI is doing or will do, and the things the most talented humans do.
That highlighted phrase "everything is a remix" was for a good reason, there's a documentary of that same name, and I can certainly recommend it.
At the same time, there are things that are truly novel, even if the idea is based on combining two common approaches, the implementation might need to be truly novel, with new formulas and new questions that arise from those. AI can't belp there, speaking from experience.