Comment by fcarraldo
1 day ago
Does a mid-session provider switch result in loading the entire context into the new model, inflating session cost?
I don't think I understand the token/cost implications of this feature
1 day ago
Does a mid-session provider switch result in loading the entire context into the new model, inflating session cost?
I don't think I understand the token/cost implications of this feature
Yes you pay a big burst right after switching. After that, everything is cached and it's smooth sailing.
Its nice if you used local, but needed å beefier modell, or more context Window. It will eat input tokens, but you do that all the time unless you have input caching.