I still think you're looking at this from a very small perspective (and to be clear, to begin with that may be the right call, it'll just bite badly at some scale).
It isn't about the cost of processing one message, it's about the cost of processing a thousand a second, or a million a second.
Fundamentally, Cloudflare make their money by taking raw infrastructure, slicing it into a million tiny pieces, paying 2-5x overhead to be able to do that, but then pricing each at 10-100x the cost to serve. It's cheap per request, per message, etc, but it's an incredibly expensive way to rent the infrastructure.
It's also a false dichotomy to say that the alternative is provisioning for peak load. There are many points on the spectrum, from Cloudflare's offering which is about as "serverless", high level, and expensive as you can get, through to buying your own servers.
Chat isn't really "bursty" at scale, it's more seasonal on a daily/weekly basis. That's pretty easy to scale over time on any cloud provider. Autoscaling groups were pretty much designed for this. GKE Autopilot is a more modern version, but there are lots of ways to do it.
I know as a startup it's important to optimise for right now, and CF might well be the best option for you. But I stand by this being your Achilles heel, it's a very expensive way to run the infra, and when you're delivering 1m messages a second, that's approaching a $1m/month bill just for request processing, let alone storage, indexing, etc.
But the per-request cost is not the only one. You're also paying $0.02 per million CPU-microseconds. If you do the math, that's easily double or triple the equivalent hourly charge for VM instances from the various major cloud providers.
For now, you're benefiting from the zero idle charges and Cloudflare's generous free allowance, because you're at a small scale. As you get bigger, the effect of those factors -- as a fraction of your total spending -- will decrease by a lot, and you'll still be paying the inflated unit costs.
This sounds reasonable (and the 30 microcents per request is pretty cool), but I wonder how do you handle storage and indexing. It still costs you, even when no one is typing.
I still think you're looking at this from a very small perspective (and to be clear, to begin with that may be the right call, it'll just bite badly at some scale).
It isn't about the cost of processing one message, it's about the cost of processing a thousand a second, or a million a second.
Fundamentally, Cloudflare make their money by taking raw infrastructure, slicing it into a million tiny pieces, paying 2-5x overhead to be able to do that, but then pricing each at 10-100x the cost to serve. It's cheap per request, per message, etc, but it's an incredibly expensive way to rent the infrastructure.
It's also a false dichotomy to say that the alternative is provisioning for peak load. There are many points on the spectrum, from Cloudflare's offering which is about as "serverless", high level, and expensive as you can get, through to buying your own servers.
Chat isn't really "bursty" at scale, it's more seasonal on a daily/weekly basis. That's pretty easy to scale over time on any cloud provider. Autoscaling groups were pretty much designed for this. GKE Autopilot is a more modern version, but there are lots of ways to do it.
I know as a startup it's important to optimise for right now, and CF might well be the best option for you. But I stand by this being your Achilles heel, it's a very expensive way to run the infra, and when you're delivering 1m messages a second, that's approaching a $1m/month bill just for request processing, let alone storage, indexing, etc.
> We pay roughly $0.30 per million requests
But the per-request cost is not the only one. You're also paying $0.02 per million CPU-microseconds. If you do the math, that's easily double or triple the equivalent hourly charge for VM instances from the various major cloud providers.
For now, you're benefiting from the zero idle charges and Cloudflare's generous free allowance, because you're at a small scale. As you get bigger, the effect of those factors -- as a fraction of your total spending -- will decrease by a lot, and you'll still be paying the inflated unit costs.
This sounds reasonable (and the 30 microcents per request is pretty cool), but I wonder how do you handle storage and indexing. It still costs you, even when no one is typing.