Comment by simplegeek
14 hours ago
As an aside, for someone interested and who's an absolute beginner, can someone please recommend good resources on how to build LLMs from scratch? Thank you in advance.
14 hours ago
As an aside, for someone interested and who's an absolute beginner, can someone please recommend good resources on how to build LLMs from scratch? Thank you in advance.
1. Build an LLM from Scratch by Sebastian Raschka (https://sebastianraschka.com/llms-from-scratch/)
2. LLM from 0 to Hero, and nanoGPT by Andrej Karpathy
I second 1. I'm a newbie in neural networks and I think it's an excellent book! One of my barriers in ML is the resources, I find them overcomplicated or too simplistic without a mid term. It's not the case of this book, everything is well-explained. Neural Networks aren't fun for me, but this book makes it very interesting.
If you are serious about ML, you can email me to get recommendations on resources.
(Fully free, of course.)
Stanford CS336 is up on youtube from Spring 2026.
I think this is the best structured class out there that teaches how to scale LLMs . Hope the 17 year old knows linear algebra. Building an intuition for the shape of the matrices is important. A lot of understanding the 'building from scratch' means understanding choices like why RoPE instead of the original frequency based positional encoding. Start with Karpathy and then go to CS 336
Check the front page of this god forsaken website a few times a day and you'll get about 10 different posts a day about it.