Comment by brianwawok
15 hours ago
And I keep using Opus to like, make git commits. Really just need a smart router that is actually smart, vs having to micromanage model
15 hours ago
And I keep using Opus to like, make git commits. Really just need a smart router that is actually smart, vs having to micromanage model
the problem is managing the contexts. your session might fit in Opus, but will that smaller model you dispatch the git commit to fit? even so, will it eat too much on prefill? do you keep compactions around for this, or RAG before dispatch or something? how do you button back up the response?
all doable but all vaguely squishy and nuanced problems operationally. kinda like harness design in general.