Comment by molf
14 hours ago
> unless you'd call cargo-cult chaining magic numpy incantations you've taken from Karpathy's tutorials "exploring"
That's what I would call exploring.
I too started exploring programming as a teen by cargo culting. Fooling around and getting results is what made it fun. Understanding came later.
> That's what I would call exploring.
Then it is not "building llm from scratch" in my book. Just mindlees following instructions. Could be educational yes, but only trivially useful, if you have no bloody idea what you are doing.
> Fooling around and getting results is what made it fun. Understanding came later.
It is not how LLMs are "built from scratch".
No it's not useful, no you don't know what you're doing, and no it is not 'proper'.
But you're learning.
Maybe just accept that not everyone starts from fundamental theory, and there are lots of people who start learning by fooling around.
> No it's not useful, no you don't know what you're doing, and no it is not 'proper'. But you're learning.
It is sold by PG as something special though.
> Maybe just accept that not everyone starts from fundamental theory, and there are lots of people who start learning by fooling around.
Even then LLMs are strange thing to advice to play with, when there are so much more interesting and theoretically accesible for a 17 y.o. so I wonder why would you'd particularly single it out.