Comment by zamadatix

The run and/or troubleshooting steps for Windows should probably include the note you need to install https://developer.nvidia.com/cuda-downloads?target_os=Window... if you have an Nvidia GPU (and probably something similar if you have an AMD GPU?). As it is right now the steps happily get you benchmarking your CPU and I'd say that might even be worth adding a "Warning: The benchmark is operating in CPU only mode, press y to continue if this is intended" type message to the program.

Edit: And for the same prompt and generated token counts it runs ~4x slower than `ollama run hf.co/bartowski/Qwen2.5-14B-Instruct-GGUF:Q4_K_M --verbose`. It's possible I'm mixing up a few things there but my results also post in the same ballpark slower than others with the same GPU so it seems something is up with the application in either case.