Comment by nylonstrung
5 hours ago
I don't really understand how async tool calls translate to token savings
It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?
5 hours ago
I don't really understand how async tool calls translate to token savings
It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?
I think a running agent periodically checks if a process it started has finished.
"More tool work per model turn" could reduce the number of cache reads (or even cache misses) and associated cost?