Comment by eurekin
9 hours ago
The needle benchmarks show, that models extended context works for the part, that can be explained as: "I can access/adress that part of the input".
I have no idea, why in that context, the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time, which has to put an upper bound of long context support in some way. There's just more things to lose focus to (or, mismanage the limited attention heads resources - per token)
There are no "attention heads" or fixed number of things a model can pay attention to ... or at least not exactly.
After every prompt the model decides "I have a weight of 1 to distribute between every token in my context". If you have ten tokens, each gets a weight of 0.1 ...
... except it's not that simple, because the LLMs don't distribute that "attention budget" equally. If your prompt was "where is Paris", then any tokens in context it can associate with Paris will get a greater share. If the word Paris is in your context, it might get 0.3 or 0.4 weight, and close by tokens might get 0.2, while other (unrelated) tokens get 0.03 or something.
Now, add lots of context, and you start to see the problem: more context = greater distribution of the attention budget. Even if the prompt is about Paris, and the Paris tokens get higher weights, they are only getting (say) 0.002 ... while unrelated tokens are getting 0.001.
All LLM "answers" are just math, computation, based on the context and those weights. If it can't "focus it's attention" because it's distributed among too much context, it's far more likely to miss the relevant tokens (eg. the Paris ones) and give you an answer that ignores key parts of context.
But again, there's no fixed number of things it can pay attention to: it's a gradual degradation of the chance of it seeing key info it should, based on the amount of context.
EDIT: Ok, I guess there are an internal thing called "attention heads"; TIL. They work exactly as I described (essentially I described a single "head").
Still, what I just wrote remains true: whether you have 1 head or 96 or ... however many your particular model has ... the fundamental issue isn't the number of heads. It's not like the LLM can pay attention to exactly (say) 96 things at once: each one of those heads has all of the available context "competing" for attention as described, and then the heads themselves "compete" to produce the answer (although not through the same mechanism; it's more like the weighting between heads is baked-into the LLM).
At the end of the day, the problem isn't "there aren't enough heads to pay attention" and the LLM hits a fixed limit. The problem is each head is trying to weight the relevant stuff, and the more stuff there is to distribute weights through, the less likely it will get it right (and then when all the heads are combined to produce the final answer, it misses key context).
> There are no "attention heads"
Yes there quite literally is internally in an LLM.
https://bactra.org/notebooks/nn-attention-and-transformers.h...
Just because something is called an "attention head" doesn't mean the terminology makes sense.
If I'm reading the article right, hungryhobbit is accurately describing a single attention head.
And... what having multiple attention heads means is that you do the "single attention head" thing several times, and average the results. There is no part of hungryhobbit's comment that changes if you make this adjustment.
In particular, he is clearly responding to this clause in the comment above:
>> the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time
Those are the "attention heads" that hungryhobbit is saying don't exist. You'll notice that this definition of "attention head" does not correspond to any part of an LLM; I fail to see the problem with saying that "attention heads", as defined by this clause, don't exist.
3 replies →
Llm's have come on quite a bit beyond the fundamentals nowadays. Attention heads really are a thing.