Comment by andai

Most of the recent gains in LLM quality came from improving the quality of inputs (i.e. recognizing that raw unfiltered internet is not the ideal diet for growing reason).

I don't know how good the filters are though, since they're mostly powered by LLMs...