Comment by fmbb
2 years ago
> temperature=0.5
Why not 0?
I’m not a prompt engineer so I never worked with the API, but I thought the “temperature” was a little knob they added for variety in responses. Is that what you want in a CLI?
2 years ago
> temperature=0.5
Why not 0?
I’m not a prompt engineer so I never worked with the API, but I thought the “temperature” was a little knob they added for variety in responses. Is that what you want in a CLI?
Good question! I used 0.5 out of habit, but I do need to do some more experimenting with this parameter. But yes, intuitively it should probably be low. I'll do some experiments in the morning and see if it works well at 0.
Non-zero temperature does have a load-bearing function, allowing escape from states where the next token is always the same. You can kinda think of it as a dither.