Comment by comex
2 hours ago
It's unclear how big a role distillation plays, but it may be a big one. There's also a law of diminishing returns. To get a meaningful increase in model quality you seemingly need an exponentially larger model. And most people don't think K3 is actually on par with top closed models.
Google says 1 million blackwell gpus are being delivered monthly I'm really curious if it's companies just hoarding chips/memory/servers awaiting to be deployed in data centers not ready yet for months or years, or everything built is actually deployed upon delivery. Plus google and amazon have there own chips in the mix.