Comment by fragmede
15 hours ago
it's bad enough how to deal with people that don't think before they speak now we gotta make the computers do it as well‽
15 hours ago
it's bad enough how to deal with people that don't think before they speak now we gotta make the computers do it as well‽
Huh, the grandfather was suggestion to have the computer think while you speak.
That's different from banning the computer from thinking before they speak, ain't it?
Thinking while I'm speaking means it isn't listening to everything I've said before thinking what to say. If I start my reply with "no, because...", and it's already formulating its response based on the "no" and not what comes after the because, then it's not thinking before it speaks.
The model can have a reasonable good guess of what you are trying to say, and use 'speculative' thinking. Just like CPU's use branch prediction.
In the common case, you say what the model predicted, and thus the model can use its speculative thinking. In the rare case where you deviated from the prediction, the model thinks from scratch.
(You can further cut down on latency, by speculatively thinking about the top two predictions, instead of just the top prediction. Just costs you more parallel compute.)
This is also all very similar to a chess player who thinks about her next turn, on your turn.