The self-hosted environment is just the backend for shell calls the agent wants to make. The inference bits, thread persistence, and (optionally) mcp/tool calls happen from the API.
Some people/companies/whatever might like the convenience and scalability of managed solutions, especially if you're say, just building something simple like a Slack bot with your custom workplace tools/data.
Yes, the lock-in is real and only good for OpenAI, but there's absolutely demand for managed services where you defer the responsibility of security patching; scaling; uptime, etc to a third party provider. Just like why people use AWS/GCP/etc over bare metal in a colo.
This is every enterprise service at this point. Let’s look at my enterprise search provider. We self host so pay for the infrastructure. We provide/pay for the models for embeddings or any LLM stuff. We still pay them consumption based prices.
Can potentially have private networking advantages, e.g. internal api's.
The self-hosted environment is just the backend for shell calls the agent wants to make. The inference bits, thread persistence, and (optionally) mcp/tool calls happen from the API.
Some people/companies/whatever might like the convenience and scalability of managed solutions, especially if you're say, just building something simple like a Slack bot with your custom workplace tools/data.
Yes, the lock-in is real and only good for OpenAI, but there's absolutely demand for managed services where you defer the responsibility of security patching; scaling; uptime, etc to a third party provider. Just like why people use AWS/GCP/etc over bare metal in a colo.
Going to be GitHub self hosted runners all over again, you pay for the API and also pay for your own self hosting.
This is every enterprise service at this point. Let’s look at my enterprise search provider. We self host so pay for the infrastructure. We provide/pay for the models for embeddings or any LLM stuff. We still pay them consumption based prices.
Native integration with their SDK.
For the lock in