Comment by mrgaro
12 hours ago
Because everybody has the same system prompt, the KV caching will make this a non-issue. The only cost is the reduced max context length.
12 hours ago
Because everybody has the same system prompt, the KV caching will make this a non-issue. The only cost is the reduced max context length.
No comments yet
Contribute on Hacker News ↗