Comment by teraflop
5 hours ago
> Software isn’t the only place where big-O scaling is relevant.
> Fully connected graph topologies are great on paper, but the number of connections scales quadratically. For a 64-core fully connected CPU topology you would need 2,016 separate data buses.
Nitpick: I don't think the comment you're replying to is proposing a fully-connected graph. It's proposing a hypercube topology, in which the number of connections per CPU scales logarithmically. (And with each node also connected to its diagonal opposite, but that doesn't significantly change the scaling.)
If my math is right, a 64-core system with this topology would have only 224 connections.
This is what I meant. I also like the idea of optical, line-of-sight connections. If you do the hypercube topology everything a node connects to has a different parity, so you lay them out on two panels facing each other.