← Back to context Comment by danielmarkbruce 2 days ago They aren't predicting the next token. It's quite literally not a prediction. 5 comments danielmarkbruce Reply gwerbin 2 days ago They're estimating a probability distribution over the next token, from which a sample is taken. Close enough. danielmarkbruce 2 days ago It's not an estimation of something. It's a policy. gwerbin 1 day ago Sure, you're right.But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence. 1 reply →
gwerbin 2 days ago They're estimating a probability distribution over the next token, from which a sample is taken. Close enough. danielmarkbruce 2 days ago It's not an estimation of something. It's a policy. gwerbin 1 day ago Sure, you're right.But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence. 1 reply →
danielmarkbruce 2 days ago It's not an estimation of something. It's a policy. gwerbin 1 day ago Sure, you're right.But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence. 1 reply →
gwerbin 1 day ago Sure, you're right.But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence. 1 reply →
They're estimating a probability distribution over the next token, from which a sample is taken. Close enough.
It's not an estimation of something. It's a policy.
Sure, you're right.
But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence.
1 reply →