Comment by cruffle_duffle

Create a “token guy” that others can consult with to figure out the best model to dispatch something to. Token guys role can be defined as an expert in the current LLM landscape. Not sure whose role this is analogue to in medical (somebody here said insurance agent… that is probably getting close).

Token guy’s role gets even better when you let them recommend entirely different hospitals for a given step in patient care (OpenAI models, random shit on openrouter, etc).

I have a “token guy” in my workflow and the dude has looked at underlying tool call patterns, cache use, etc and made all kinds of recommendations to the clients calling it all on their own and completely unprompted—just following a simple role.

I’d experiment with expanding their role into prompt engineer or some kind of specialist on that domain.