← Back to context

Comment by CamperBob2

17 hours ago

The prereqs aren't as extensive as people fear, but they are important. Starting out by building a scratch-trained GPT3-scale model is always an option, but anyone who's going to succeed at that doesn't need to watch any YouTube videos.

For the rest of us, the sense of discovery that we get from training and running toy models is a big part of the reward. This stuff is not an exact science, at least not yet, so time spent building intuition with small-scale models is an incredibly important part of the learning process. Not pointless at all.