Comment by monocasa
9 hours ago
Yeah, their cache work is undeniably really cool.
That being said I think it's a natural consequence of the difference between a mainframe and commodity servers. A mainframe is going to be running pretty disparate workloads simultaneously, so it makes sense to steal from your neighbor if they aren't using their cache. Whereas it's more likely that a commodity server is just running the same server on each core, and if you have a different workload, you pick a different shape of server to run it on. There are pros and cons to both.
I do wonder about the spectre consequences of borrowing cache lines from other cores though.
Even though their CPUs are insanely fast, the real power of mainframes is in their IO. The amount of data you can push through those machines is absolutely mind blowing.
Historically this was because each I/O "channel" was a separate computer that handled the actual communication with the device, be it a terminal, disk, tape drive, card reader, printer, etc. and exchange data with the CPU via DMA. This allowed mainframe CPUs, which in the past weren't particularly fast, to handle huge workloads involving hundreds or thousands of users. These days, even commodity computers get blazing fast I/O to bus mastering devices. Where the mainframes win today is on reliability, built-in redundancy, hot-pluggability and expandability of components (you can just plug in CPUs, memory, disks, and network interfaces as long as you can afford them with the machine still running), and service and support. (Mainframes phone home immediately if they detect problems and an IBM service person will be on site the same day to fix it.)
What's unique about their cache ?
Basically for Telum II, (I don't know what changed here from Telum III, the core under discussion with the ARM decoders) each CPU core has a giant 36MB L2 cache. Then, rather than a discrete L3 cache, the cores keep track of L2 residency needed for that core's working set, and allocate the rest of their L2 to a shared pool that is the L3 cache. Then the same thing with L4 being the same pools in all of the other chips on the same drawer (which you can kind of think of as close to a single server).
https://chipsandcheese.com/p/telum-ii-at-hot-chips-2024-main...
Wow that is a massive cache. Pretty cool. Guessing it would great for data throughput !