Comment by sdesol
2 days ago
> I think the argument is that the extra value provided is a small enough context window for working with an LLM.
I'm not sure moving something that could work as function to a microservice would save much context. If anything, I think you are adding more context, since you would need to talk about the endpoint and having it route to the function that does what you need. When it is all over, you need to describe what the input and output is.
Oh certainly. I was arguing that if you need more isolation than a function gives you, don't jump to the conclusion that you need a service. Consider a library as a middle ground.