Comment by HDBaseT
4 days ago
Wouldn't it still be more efficient to do GPU transcriptions anyways? is this something we could actually put the effectively useless NPUs to use in modern laptops?
4 days ago
Wouldn't it still be more efficient to do GPU transcriptions anyways? is this something we could actually put the effectively useless NPUs to use in modern laptops?
yes GPU is significantly faster, but cpu only lets you do it anywhere - wasm in the browser, any server etc.
NPUs - definitely a good use case for at least part of it, there are ports of whisper that use coreML/ANE with less power and 3x speed of CPU only
Possibly, but you may want to use the GPUs for other things, or have under-utilized CPU-only servers lying around.