Comment by naasking
3 days ago
> We are quite far into the development cycle of LLMs.
No, we've barely scratched the surface. Billions of dollars have been poured into the stupidest possible thing that could work + scaling, and we're only now trying more clever things. Fine-tuning on specific tasks will yield considerable productivity benefits in those domains.
I'm not only skeptical of your claim on the "rate of improvements over the last 6-12 months", but it's not even a compelling time horizon to infer any kind of trend at this stage.
The needle hasn’t meaningfully moved on these types of tasks though. There was a class of problems that LLMs could nearly immediately help with that brought us quickly to the current rate. But then we stalled and it hasn’t meaningfully moved with any of the major releases