Comment by ActorNightly

Not if you have duplicates of rows on the first matrix, which can be done very efficiently if you build specialized hardware. Then its all just forward in parallel.