Comment by chpatrick
4 days ago
Could someone explain how Jev is different from using any old model and constraining the output to "My choice is a/b/c..."?
4 days ago
Could someone explain how Jev is different from using any old model and constraining the output to "My choice is a/b/c..."?
The argument the article makes is it's not that different. Jev's argument is that they have trained the model to better output probabilities (which is not necessarily a training object of LLMs but we don't actually know that)
Ultimately Jev claims to have a data advantage which is likely where the future lies. They'll have a unique edge in improving general purpose classification / decisioning.
The model can spend more “mental energy” on the decision because it doesn’t have to spend any on phrasing the output.
LLMs are constant time per token, if you only let it choose a/b/c/d then there's nothing else it can "spend".
If you believe the marketing, constraining the output this way can make the model much faster and much more type-safe (the model didn’t give you a fifth choice not present in the choices).
How is that different from just not letting it output option 5 when you sample from the probabilities?