Comment by BeetleB
1 day ago
I suggest trying whisper-cpp if you haven't. It's probably the fastest CPU only version.
But yeah, NPUs likely will be faster.
1 day ago
I suggest trying whisper-cpp if you haven't. It's probably the fastest CPU only version.
But yeah, NPUs likely will be faster.
Depending on the part, it's likely the iGPU will be even faster. The new panther lake has iGPUs with either 80% or 250% the performance of the NPU when at the higher end. But on lower end models, it's lower but still within the same performance class
faster-whisper can be faster in many cases, even on CPU.
Looking at that page, it doesn't seem particularly faster than whisper-cpp, except when using batches - but I'm not clear on what that means.
Does it have a command line utility I can embed into my scripts?
Batching is essentially running multiple instances at once, ie bundling 8 segments and running them simultaneously on the processing unit, but which obviously takes more RAM to do. Notice, however, that if you drop the precision to int8 from fp16, you use basically the same amount of RAM as whisper.cpp yet it completes in a fraction of the time using batching [0].
Yes, if you check their community integrations section on faster-whisper [1], you can see a lot of different CLIs, GUIs, and libraries. I recommend WhisperX [2], it's the most complete CLI so far and has features like diarization which whisper.cpp does not have in a production-ready capacity.
[0] https://github.com/SYSTRAN/faster-whisper#benchmark
[1] https://github.com/SYSTRAN/faster-whisper#community-integrat...
[2] https://github.com/m-bain/whisperX