Comment by Lerc
8 hours ago
AI is not intrinsically matmul. Current models are a framing of the problem as matmul to enable the bulk operations necessary. There are many calculations that can (and have been) reframed into matrix form to achieve similar results. More than once I have seen a matmul version of something that I thought couldn't have been representable that way. I'm not sure if the greater benefit is the performance or the ability to conceptually manage the problem in you head.
Not all algorithms can be converted of course, but a lot of that is because there are some inherently unparallelizable problems.
Many algorithms can be reasonably converted into matmul. Not many algorithms can be reasonably converted into very low precision matmul. And that's all that these things are going to do.