Comment by billyjobob

2 days ago

If you send the full 200k tokens on every request you will get very few requests before you hit the token limit. Caching reduces the number sent but I don't know how much they can cache?

0 comments