Comment by red75prime
1 day ago
It doesn't explain why it doesn't make "I'm not paid enough for this shit" more statistically likely.
LLMs' processing that reproduces statistical patterns of the training data is modified by post-training. That's why we have LLMisms, for example.
LLMs aren't simple patter-matchers/pattern-predictors. They are incredibly complex systems that capture some aspects of the systems that produce the training data.
Maybe it does. Need to run evals to see if it does or doesn’t.
Point was - everything in the context window affects the output. Including “silly” things like “it is known AI can do this”. And that has nothing to do with superstition, as the poster above me seemed to imply.