Comment by throwup238

Yeah and that’s why we call it “standing on the shoulders of giants.” Humans went through tons of trial and error in every facet of life to get where we are today. We kept the stuff that worked and taught it.

But before humans can understand enough language to ingest that synthetic data, they do a lot of their own discovery based training where they learn about the world physically and absorb the language people around them use, kind of like throwing random internet data at an LLM.