Rollie calculation, like the online softmax in FA, implies a centralized computing unit that does the compute and stores the intermediate results in its registers. With PIM you have no centralized compute unit, you have a bunch of memory, and a bunch of MACs all over the place.
How would you do map-reduce across multiple DIMMs w/o extra reads/writes?
PIM implies some sort of distributed compute, which can work for some cases, but I am not sure LLMs are one of them.
Rollie calculation, like the online softmax in FA, implies a centralized computing unit that does the compute and stores the intermediate results in its registers. With PIM you have no centralized compute unit, you have a bunch of memory, and a bunch of MACs all over the place.
How would you do map-reduce across multiple DIMMs w/o extra reads/writes?
PIM implies some sort of distributed compute, which can work for some cases, but I am not sure LLMs are one of them.
Re-arrange x+(y+(z+w)) into (x+y)+(z+w) to accomplish a distributed sum. Shared bus, interconnect fabric. This is a supercomputer on a chip.