Comment by hinkley

Maybe PIM will push this forward, but I still think we're doing something fundamentally wrong by not just embracing NUMA and trying to do something Sun tried decades ago, which is have number of cores share 4GB of semiprivate working memory.

We've kind of half-assed it with DDR memory banks, but it mostly introduces mysterious slowdowns that are difficult to reason about and I think we would be better served I think by making a formal thing. Instead of introducing an L4 cache we could do this instead, and reduce the size of the L1-L3 caches, which shortens lookup time and thus latency.

For legacy apps, you could provide facilities for the OS to 'page' blocks in from main memory, but the speed would come from managing the workload imperatively, starting loads in the background before the data is actually needed, and dumps after it is last touched.