Comment by TZubiri

>Stop Thinking of LLMs as Next-Token Predictors

>Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong,

If it isn't wrong, then I will continue thinking of them as such, thank you.

Here's a much more formal definition I can come up with (which is more complete but compatible with 'next-token predictor')

LLMs are a set of functions of the type:

>typedef int Token;

>char* token2utf8(Token token)

>Token next(Token* context)

>(Token,void*) next(void* hidden_state)

Where the second next token function's runtime is O(n), and the latter is O(1). All are constant memory.

Object paradigm is more appropriate than functional definition, as the "Hidden state" coincides with private object state rather than a function paramter and return value.

>LLM.next(system_prompt) # O(len(context))

>LLM.next(user_prompt) # O(len(user_prompt)) not of system_prompt+user_prompt

That's it, that's all LLMs are, that's the interface, the rest are implementation details.