Comment by manquer

Lower models typically use distilled outputs from higher model to improve .

all training runs have to at some level share the compute capacity ; while they have more than enough to run many large runs concurrently, they don’t have enough compute to run everything concurrently.