← Back to context

Comment by squidbeak

4 days ago

Reading stuff like this (as a layman), diminishing these things as 'Next token predictors' seems absurdly reductive. At some point we'll need to concede that 'selection' is a better term for this than prediction.

7 comments

squidbeak

Reply
chrisjj  4 days ago

> diminishing these things as 'Next token predictors' seems absurdly reductive.

This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.

Regardless, if all you want is a next-token selector, save your money and roll a die.

  • squidbeak  4 days ago

    > This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.

    No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.

    > Regardless, if all you want is a next-token selector, save your money and roll a die.

    Tell me, where is the emergent symbology guiding that dice?

    • chrisjj  4 days ago

      > No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.

      The paper claims no symbolic richness beyond that evident from the undisputed next-token prediction.

      > Tell me, where is the emergent symbology guiding that dice?

      There's none. That's my point.

  • xg15  4 days ago

    > which in no way challenge the established view that these bots are next-token predictors.

    I mean, of course they are, that's literally what the inference loop does. You can look at the source of your favorite model runner and you'll see exactly that.

    What I find misleading about this term is that it focuses attention on the "next token" part and glosses over the "prediction" part as some sort of unspecified "statistical algorithm" - even though this is where most of the work happens and where the interesting questions are.