Comment by oblio
16 hours ago
I'd argue we've already hit the ceiling. Can you truly tell the different between SOTA models from 9 months ago and those from today? There are some improvements but they're mostly marginal.
Plus there is a chance the actual scaling that matters is beyond our reach. Think instead of TB models, PB or ZB models. We don't even have that kind of information. Humanity in its entire history hasn't generated 1ZB of information.
> Can you truly tell the different between SOTA models from 9 months ago and those from today
On a task that corresponds to the benchmarks, yes absolutely.
> On a task that corresponds to the benchmarks, yes absolutely
The very definition of diminishing returns.