Comment by riazrizvi
4 days ago
Natural language is ambiguous. It needs to be. I think the approach here of trying to figure out how to make circles into squares, and argue why circles should be squares, is misguided.
Discussions of this type are going to eventually morph into better understanding of how to accept ambiguity and randomness in language, and further shape it with other larger sub-patterns beyond the little proto-grammars that the QKV projection matrices extract.
Yes, but determinism != ambiguity, because determinism means: for this exact input the same exact output needs to follow.
If I ask the same model the same question I should be able to deterministically get the same answer.
Now if we phrase the same question slightly differently we would expect to get a slightly different answer.
> Now if we phrase the same question slightly differently we would expect to get a slightly different answer.
You wouldn't get this from an LLM though, a tiny change in starting point gets a massive change in output, its a chaotic system.
Maybe predictability is what is meant?
Me: What’s an example of a dice roll?
LLM: 1
“Language ambiguity with determinism”? Sure I can juxtapose the terms but if it’s semantically inconsistent, then what we mean by that is not a deterministic, definitive thing. You’re chasing your tail on this ‘goal’.
Ambiguity: The request/prompt leaves a lot of room for interpretation. Many qualitatively different answers may be correct, relative to the prompt. Different or non-deterministic models will return highly variance results.
Determinism: If a model is given the exact same request/prompt twice, its two responses will also be identical. Whether or not the consistent response qualifies as correct.
The two concepts are very different.
(Ambiguous vs. precise prompt) x (Deterministic vs. Non-deterministic model) = 4 different scenarios.
A model itself can be non-deterministic without being ambiguous. If you know exactly how it functions, why it is non-deterministic (batch sensitive for instance), that is not an ambiguous model. Its operation is completely characterized. But it is non-deterministic.
An ambiguous model would simply be model whose operation was not characterized. A black box model for instance. A black box model can be deterministic and yet ambiguous.
2 replies →
If you really want that to work while being reproducible, maybe give it a random number tool and set the seed?
> LLM: 1
A perfectly acceptable answer.
If it answers 1 every time it's still a perfectly acceptable answer.
1 reply →