Comment by islewis

I resource I found invaluable in demystifying LLM's was Andrej Karpathy's Youtube series. In particular, his "Makemore" video built a basic LLM in the same vein as OP's.

It's long (2h), but well worth the time if anyone is curious about the technical workings of LLMs.

https://www.youtube.com/watch?v=PaCmpygFfXo&list=PLAqhIrjkxb...