Comment by XenophileJKO
1 month ago
Context is the information you give the model, attention is what parts it focuses on.
And this is finite in capacity and emergent from the architecture.
1 month ago
Context is the information you give the model, attention is what parts it focuses on.
And this is finite in capacity and emergent from the architecture.
So attention is based on a smaller subset of context?