Comment by TJTorola
13 hours ago
Perhaps we'll get to a point where believing any un-sourced information from an LLM will feel crazy. I don't want my model to know more than it needs to perform logic and use tools. Once it is capable of using tools I would much rather it looked up information or sourced it from existing context rather than just divine it from it's weights.
The problem is it needs world knowledge to know what to lookup. This puts a floor on how little it can know while being able to look up what it doesn't know. Maybe its better if it knows a lot but has a good instinct for verifying that.
I empathize, and I have the same preference, but I wonder how this interacts with other people (many of them being our coworkers) using LLMs. There is no authoritative source for the models to pull info from, so either people will have to exercise good judgement and double check important claims, or they will trust too blindly and fall close to the level of whatever LLM they use. In that case, I prefer my coworkers to use an LLM that does have world-knowledge -- I will still hear them spout ridiculous claims, but at least it should be less frequent. It strikes me there's a sort of prisoners dilemma here, where if nobody trusts others to critically evaluate info, it's in our interest to make the tooling do it instead, to whatever degree that is possible. Maybe I'm too cynical about working with others though.