Comment by winwang
12 hours ago
It's because the cache comes from the prefix. `f(f(f(A,B),C),D)` is quite different from `f(f(A,B),D)`.
12 hours ago
It's because the cache comes from the prefix. `f(f(f(A,B),C),D)` is quite different from `f(f(A,B),D)`.
Yes, that's my understanding- though I realize that OpenAI has made changes to what you can do with caching over the API recently, so you may be able to do some fancy stuff to get around this.