Comment by dcrazy 4 days ago Isn’t the purpose of self attention exactly to recognize the relevance of some tokens over others? 2 comments dcrazy Reply kimixa 4 days ago That may help with tokens being "ignored" while still being in the context window, but not context window size costs and limitations in the first place. melecas 4 days ago [dead]
kimixa 4 days ago That may help with tokens being "ignored" while still being in the context window, but not context window size costs and limitations in the first place.
That may help with tokens being "ignored" while still being in the context window, but not context window size costs and limitations in the first place.
[dead]