If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
Cowan (2001) is an oft-cited paper proposing three to five "chunks" of capacity in human attention: https://www.cambridge.org/core/journals/behavioral-and-brain...
The full PDF is worth a read (Figure 1 may be of interest to many here): https://www.cambridge.org/core/services/aop-cambridge-core/c...
If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
updated the comment. i meant four "concepts". i dont reason about my own thinking in terms of parameters.