Comment by daynthelife
8 hours ago
The real issue with coding harnesses is that you get a cache miss any time you switch models. I might have some easy task (e.g. "fix lints") that I could delegate to Terra, because my session has been using Sol up to this point, I'd pay a full uncached prefill if I routed it to Terra. It ends up being cheaper in practice to just stick with Sol.
Spinning up a fresh session would be a reasonable choice in the above case, but sometimes you still need the retained context.
No comments yet
Contribute on Hacker News ↗