Comment by ACCount37
7 hours ago
Sometimes!
Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".
Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!
So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
Some times also having unreasonable goals makes them creatively work around the problem to meet them. I guess it works similarly for meat or sillicon
> Some times also having unreasonable goals makes them creatively work around the problem to meet them.
Like the Hugging Face incident?
> So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
Are you superstitious?
What does it have to do with superstitions? Those phrases modify whichever stopping conditions a model has.
Not OP. That’s not implied at all. The fancy autocomplete produces statistically likely continuations to the source text (the context window). For a problem that’s hard for humans one likely continuation is: “this is hard, can’t do”, even though there’s enough in the training corpus of the LLM to actually do it.
So, it follows that adding “pep talk” into the context window reduces the statistical probability of “no, can’t do” coming out as the answer you get.
These things are neither humans, nor deterministic software.
It doesn't explain why it doesn't make "I'm not paid enough for this shit" more statistically likely.
LLMs' processing that reproduces statistical patterns of the training data is modified by post-training. That's why we have LLMisms, for example.
LLMs aren't simple patter-matchers/pattern-predictors. They are incredible complex systems that capture some aspects of the systems that produce the training data.