Comment by gruez
1 year ago
So you're saying whatever the model doesn't have to be tethered to reality at all? I wonder if you think the same for chatgpt. Do you think it should just make up whatever it wants when asked a question like "why does it rain?". After all, you can say the words generated are also semi-random sequence of letters that humans give meaning too.
I think going to a statistics based generator with the intention to take what you see as an accurate representation of reality is a non starter.
The model isn’t trying to replicate reality, it’s trying to minimize some error metric.
Sure it may be inspired by reality, but should never be considered an authority on reality.
And yes, the words an LLM write have no meaning. We assign meaning to the output. There was no intention behind them.
The fact that some models can perfectly recall _some_ information that appears frequently in the training data is a happy accident. Remember, transformers were initially designed for translation tasks.
> Do you think it should just make up whatever it wants when asked a question like "why does it rain?"
Always doing that would be preferable to the status quo, where it does it just often enough to do damage while retaining a veneer of credibility.