Comment by redox99
14 hours ago
Search results suck because the web sucks these days. The big models from OpenAI/Anthropic have every book in existence baked into them
14 hours ago
Search results suck because the web sucks these days. The big models from OpenAI/Anthropic have every book in existence baked into them
I don’t think that’s the right way to think about LLM ‘knowledge’. They don’t have absolute recall of everything in the training set. They have been trained so that they have weights that can predict what those books might say - that is, if they read them they would find the contents unsurprising. That doesn’t mean it wouldn’t be helpful to pull relevant passages of text directly into context for a particular task.
Does it really matter? What about including all relevant and up-to-date literature as skills for local models? I have no experience with this but I am pretty sure someone has already thought about it.