Comment by bob001
2 days ago
>The details matter (training set, full context).
But you said:
>Training shifts the probability - but doesn't change the fact that the output is a sampling based on input and the model?
So which is it?
>You're essentially suggesting we play Turing's Guessing Game; if your LLM can guess my response to any prompt - then it can be said to have modeled how I would reply to those prompts. Expand the prompts far enough, and you could reasonably argue the LLM models me.
I'm merely noting that your argument that "training+input=>output is the problem" applies to humans. There's other arguments for and against LLMs however I'm simply pointing out that your argument isn't a good one.
> I'm merely noting that your argument that "training+input=>output is the problem" applies to humans.
I'm not sure I ever said that it was a problem.
I also never meant to say that sampling across the model (formed by training) based on input couldn't produce novel token sequences.
As for human vs LLM - my argument would be that while trained on a large corpus, the models are poor in experience and "sensory input" - their training so different from growing up - that even if we could model humans as LLMs - the gap between current LLMs and "human" LLMs remain a wide one.
More to the point - I believe the current generation of LLMs are way too sensitive to input (prompt, system prompt, context/RAG) - precisely because they're still too narrowly trained, and also aligned to weigh input heavily.
What we have today are models that work well with language (generate, transform, translate) - not so great on complex tasks - because they're still generating next probable token.