Comment by Dylan16807
10 hours ago
Even then it's deterministic in the way a hash function is deterministic. Change one letter and you can get a completely different output. What people actually want is something continuous.
10 hours ago
Even then it's deterministic in the way a hash function is deterministic. Change one letter and you can get a completely different output. What people actually want is something continuous.
Agreed on the desire for continuous behavior. That said, in a modern LLM, is this hash analogy accurate? I would be surprised if a single letter changed most zero temp force ranked outputs.
E.g:
“Where is the Eiffel Tower Located? One word only.”
“Where is the Effel Tower located? One word only.”
“Where is the Eiffel Tower located? One wor only.”
I’d be very surprised if those got different answers from even a small local model at temp 0.
For a single word response, perhaps.
But for anything else I wouldn't.
The entire chain will be affected from the different tokenization on down. Even if it lands in roughly the same semantic area, it doesn't mean it will land there with anything like the same syntactic selections. Anywhere there were multiple near-tokens could easily select a different route based on even minor fluctuations in the starting conditions. It's chaotic.
I don't know about single letters, but single words?
"Score this resumé. Applicant: Jim ..."
"Score this resumé. Applicant: Greg..."
Is it obvious to anyone that these will have the same modal response?
"Your are a helpful/less assistant"
Give it a try. 4 letter difference. Add a few 100 tokens describing the task, such that the change becomes a tiny fraction of the input.
Discontinuities everywhere.
This is it. People mistake deterministic for precise/exact/correct. It's not.