Comment by mips_avatar

A single 3090 will train qwen 0.8B just fine. While it’s not a very capable model any training technique you would want to master can be used to make real progress. And all the skills you need to learn how to do this can be learned watching Andrej Karpathy’s zero to hero series (shame he quit educational content and went to anthropic)