Comment by aziis98

I just tried the Q4_K_M variant of this [] and this is one of the first models that run at ~20tps on my laptop. I also tried it with some "hard" maths questions and it clearly knows much. Can't wait to try some local coding agent harnesses with it (I recently discovered kon [1] and dirac [2] and wanted to try them out)

The only thing I'm not sure about is if this model supports thinking or not.

[1]: https://github.com/0xku/kon

[2]: https://github.com/dirac-run/dirac