Comment by felineflock
1 hour ago
That is awesome!
I am curious about the decision to not use GPU since this is for Apple Silicon.
Wouldn't the GPU potentially accelerate the DeltaNet/attention layers and matrix multiplication in general?
1 hour ago
That is awesome!
I am curious about the decision to not use GPU since this is for Apple Silicon.
Wouldn't the GPU potentially accelerate the DeltaNet/attention layers and matrix multiplication in general?
It will, but the process at this point is SSD bound rather than compute bound. On a bigger machine, Apple silicon must help but I don't have a bigger machine. I can think about this more and will make changes if that helps.