Comment by ttoinou

3 hours ago

Lots of small details are taken care of so it runs smoothly. For example ds4-agent is append only, never rewriting history of messages, keeping KV cache prefix reusable. Huge benefit

0 comments