Comment by withinboredom

Could always train them on data up to 2015ish and then see if you can rediscover LLMs. There's plenty of data.