Comment by andai
3 days ago
OpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
3 days ago
OpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
Models are trained in highly distributed and parallel ways these days.
It is entirely possible to run multiple concurrent post-training runs. When a frontier lap deploys a 1M RL gym rollout, these 1M environments are absolutely not talking to each other or interconnected. They individually generate traces and movements that can be then combined for post training.