Comment by aziis98

For me, at least for agentic use, you need at least ~40tps. Less might be good only for tasks you could run in the background (like at night maybe).

Instead of coding agents like tool for local models I would like to see more "docker ps" like tools where you can queue up tasks that get processes incrementally (maybe when the pc i idle) and are specialized for doing retries and caching as much work as possible to work decently with slow local models.