Comment by coldtea
18 hours ago
It's not like they have a poweful all-knowing oracle that can explain it to them at their dispos... oh, wait!
18 hours ago
It's not like they have a poweful all-knowing oracle that can explain it to them at their dispos... oh, wait!
They have to know that this could bite them and to ask the question first.
I do think having some insight into the current state of the cache and a realistic estimate for prompt token use is something we should demand.
If there was an affordance on the TUI that made this visible and encouraged users to learn more - that would go a long way.