Comment by potatoman22
2 days ago
It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled (temp > 0) and input is varied. Using the right prompt makes the model perform better on average, so it's not completely dumb.
I like task vectors and soft prompts because I think they show how prompt engineering is cool and useful.
https://arxiv.org/pdf/2310.15916
https://huggingface.co/docs/peft/conceptual_guides/prompting
> It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled
Are you not aware the random sampling makes something non-deterministic?
I'm saying LLMs are deterministic and because of that, prompt engineering can be effective. You knew what I was trying to say, but chose to ignore it.
You should follow the HN Guidelines. I'm trying to have a discussion, not a snarkfest.
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
I don’t believe that when he wrote that, he was using his own intelligence
At least I'm adding to the discussion.
1 reply →