Comment by LPisGood
19 hours ago
Do we really care about this gap? If open models are 6 months to a year behind frontier models, does it really matter that much?
19 hours ago
Do we really care about this gap? If open models are 6 months to a year behind frontier models, does it really matter that much?
This is the first time in terms of model progress where my personal response is: It does not matter to me because the models 6-12 months ago were already good enough for most everything I need to do. I think 95% of dev work is perfectly fine 6 months behind, if that is truly where we are at now with these open models.
I’ve seen analyses pointing to the fact that the gap is growing, which would be worrying. I think all the benchmarking and whatnot is not reliable so who knows, but we’ll definitely have a good feel in a couple of years.
It depends on how far your own development tasks are beneath the higher levels of complexity and sophistication. The trade-offs for most ordinary developers will be minimal or nothing.
I believe my development tasks are some of the most complex and sophisticated out there (high performance mathematical optimization and modeling), and to me Opus 4.6 is pretty good.
Which sort of development tasks do you believe are still out of reach?