Comment by ACCount37

11 hours ago

If what we want to do with this is make cheap QKV sweeps, then "a weak NPU with a lot of mem bandwidth" seems good enough? Exactly the tool for that job, and nothing else.

Also spares us the trouble of dealing with weights. By the time we're in QKV realm, the weights have already weighted.