Comment by vessenes

Interesting read. The paper calls this a “roadmap” and says 3d HBM is still figuring out what it can be, and what it will look like - seems right.

Hyperscalers are dealing with a pretty complex Pareto envelope that includes power (total), power (density), volume of space available, token throughput and token latency.

My guess is that there’s going to be some heterogenous compute deployed possibly forever, but likely for at least the next six to ten years, and exotic fragile underclocked highly dense compute as imagined in the paper is likely to be part of that. But probably not all of it.

Either way as a society we’ll get the benefits of at least a trillion dollars of R&D and production on silicon, which is great.