Comment by radicalbyte

Specifically we have American models which were built on extremely crappy data in insane quantities. What happens when you use same models to build a corpus of extremely high quality training data. Say for Math, coding etc. Then use that to train models. Can you get the same performance from models 10% of the size? Or 1%? Or 0.01%?

From what I've been seeing we're clearly getting to a position where models are getting "good enough" for some tasks to be really cool assistants to skilled people. And they're limited more by being extremely slow and expensive to run. What happens when they're not?

I can't see the model providers winning enough to make their valuations real.