Comment by angusturner
2 hours ago
What does this even mean. There is strong evidence of LLMs doing in context learning.
Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference
2 hours ago
What does this even mean. There is strong evidence of LLMs doing in context learning.
Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference
> What does this even mean. There is strong evidence of LLMs doing in context learning.
1. Is this learning persistent?
2. Do they verify these new lessons against core principles?
3. Do they and protect themselves/ignore requests if these new lessons contradict those core principles?
Humans do that from the time they're 3 years old (not that well, but they do do it).