Comment by anon291
1 year ago
Keep in mind that the system always chooses randomly so there is always a possibility it commits to the wrong output.
I don't know why openAi won't allow determinism but it doesn't, even with temperature set to zero
1 year ago
Keep in mind that the system always chooses randomly so there is always a possibility it commits to the wrong output.
I don't know why openAi won't allow determinism but it doesn't, even with temperature set to zero
Nondeterminism provides an excuse for errors, determinism doesn't.
Nondeterminism scores worse with human raters, because it makes output sound even more robotic and less human.
Would picking deterministically help through? Then in some cases it’s always 100% wrong
Yes, it is better if for example using it via an API to classify. Deterministic behavior makes it a lot easier to debug the prompt.
Determinism only helps if you always ask the question with exactly the same words. There's no guarantee a slightly rephrased version will give the same answer, so a certain amount of unpredictability is unavoidable anyway. With a deterministic LLM you might find one phrasing that always gets it right and a dozen basically indistinguishable ones that always get it wrong.
My program always asks the same question yes.