Comment by msdz
6 hours ago
Great article, even if it will be interesting to see whether things continue to develop in such a direction or not.
> There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks.
Future?
Even just recently I’ve read of two approaches to this problem:
Cactus have come up with Needle [0][1], which is their tool-calling focused 14 MB model (still an LLM!) – no world knowledge engrained.
And instead of say, tool call structure, VibeThinker [2][3] focuses on reasoning over world knowledge.
Combine these two approaches with a reliable search tool/a safe way of accessing the internet for the model, and you’ve got a probably slightly slower model for factual questions, which on the upside however doesn’t hallucinate.
That kind of setup is super dependent on a search engine, and search keeps getting worse.
Generally I agree, with two caveats/thoughts:
Providing not just any a baseline, but a correct and useful one, is ever more important the less the model is grounded in world knowledge – misunderstandings probably compound faster if there is no general grasp of (broadly) “life on earth”, or computers, or whatever.
And secondly, I think (consumer-oriented) search becoming worse and worse is a challenge that’s mostly solvable (but far from solved!) for the big labs: (Mostly) trusted or even editorialized/reviewed sources like published work, Wikipedia, etc. is something they could index internally, it doesn’t need to come from a random blog site on the public internet. Furthermore, there’s a whole slew of companies specializing in crawling-for-LLM (i.e., bypassing bot protections) now as well.
search isnt that super hard to solve. the problem is that incentives are misaligned - which means companies who sell something have an incentive to share their index with the corporation that pays to be the default search provider. for a simple search engine that just delivers facts there's very little incentive to keep its index up-to-date: because that company wont get the apis to retrieve the content efficiently, so it needs more compute, bandwidth and storage to run.
the technicals of search did not get worse. the economics of the search provider did. If government wasn't stuck in the 50s mindset, search provider would have been nationalized just like a utility.
I think the techinicals of search did get worse - the ai sloppification of the internet made the needle harder to find in the haystack.