Comment by dharma1

4 days ago

yes GPU is significantly faster, but cpu only lets you do it anywhere - wasm in the browser, any server etc.

NPUs - definitely a good use case for at least part of it, there are ports of whisper that use coreML/ANE with less power and 3x speed of CPU only