Comment by dwa3592
3 hours ago
Its really easy to argue against local models because when it comes to quality, you can argue using the tokens/sec. and when it comes to speed, you can argue using the parameter count. This is not compared to the frontier stuff but it is the frontier of last year that now runs on a local machine. It was impossible to do this last year.
the local models open source harnesses are really improving quite fast; just a few months ago i couldn't get any tool calling to work, and responses were very slow (thinking going on too long etc) but now with some newer models on my macbook air, tool calling works and depending on the model, it returns from thinking fairly swiftly....