Comment by torben-friis
15 hours ago
I'm OK going through the learning itself (will definitely check out the book though, since it's so well liked!). The question is more, what do you do with that once the book or course is done?
For a junior coder, there's usually a path to recommend, joining some open source project or doing some home scripting, etc. Something where you can be useful even if it's not your full time job.
I'm wondering whether there's some similar hobbyist path available here, or whether there's too much of a jump between the educational/pop sci stuff and the professional realm. If it's the second, that's also fine, I just want to know if I'm missing something.
One practical-ish thing you could reasonably do with this (beyond just better understanding how models work so you can make better choices/tradeoffs), is post-train some of the small-to-medium models, fine-tune them on your own data or priorities.
You don't need to change many weights, relatively, to add capabilities or modify behavior. Look up LoRas for instance.
I'm of the opinion that post-training existing models is an area with a lot of benefit and fun in the near future (it's already being done here and there of course).
I doubt you _really_ have to know everything in this course to know how to do that, but I'd want this as background personally for it.
You included machine learning in with LLMs in your root post. Try training a supervised model for some task, like an image classifier or a detector. LLMs have not replaced convolutional models, and even transformers can be used for non-NLP tasks. The idea this is somehow not professional is not true. Despite what people would have you believe, ResNet50 fine-tuned on a good dataset is still competent. In many environments there isn’t enough compute for bigger models, and large language models aren’t always the best choice.
You also don’t have to go straight into LLMs. You can learn most of the training mechanics via classic deep learning. It isn’t much different with LLMs except the training data and losses are specific and the models are much larger.
The comment below/above is right. Pre-training isn't the path (but you really do need to learn how it's done). Learn how it works, learn the architecture, then take models and do post training. Fine tuning and RL on them. There was a fun/clever/useless project recently posted here about building a terminal emulating model thing, as an example.