← Back to context

Comment by chatmasta

12 days ago

This is how you build a dominant company. Good for you ignoring the whiny conventional wisdom that keeps people stuck in the hyperscalers.

You’re an infrastructure company. You gotta own the metal that you sell or you’re just a middleman for the cloud, and always at risk of being undercut by a competitor on bare metal with $0 egress fees.

Colocation and peering for $0 egress is why Cloudflare has a free tier, and why new entrants could never compete with them by reselling cloud services.

In fact, for hyperscalers, bandwidth price gouging isn’t just a profit center; it’s a moat. It ensures you can’t build the next AWS on AWS, and creates an entirely new (and strategically weaker) market segment of “PaaS” on top of “IaaS.”

Yup. Bingo. We've had to pass the cloud egress costs onto our customers, which sucks.

With this, it'll mean we can slash that in half, lower storage costs, remove "per seat" pricing, etc

Super exciting

  • How do bandwidth costs work now? do you pay the ISPs a flat fee, or is it still usage-based? how much cheaper is it compared to cloud providers?

    • in my experience (Eu, datacenter/ISP space) connectivity is either sold based on 99percentile commitments (aka, you pay for sending us this amount of traffic, anything over this gets you billed extra) or based on a minimum commitment for the traffic you send. (atleast X amount of bandwith) or it is based on a flat free principle where you pay an upfront cost for the setup and the rest is a base price for X amount of bits per seconds.

      It depends a lot on what kind of connection you require, and things like oversubscription and congestion control also come into play.

      Peering ports with IXP's are usually flat rate, while ports in datacenters to end customers usually have more complex constructs.

      Hyperscaler bandwith is notoriously expensive. for instance, a 100Gbps port on the AMS-IX is 2500$[1].

      Now, you need to account for extra costs to actually use this port (some IP space, ASN number etc) but even with all that added up i think you will not get much more expensive then 400$ per month averaged in total over a year.

      Now what makes comparing difficult is that hyperscalers are not transparant when it comes to connectivity costs. Looking at egress fees for example:

      AWS seems to charge 1 cent per Gigabyte transferred for egress fees.

      If we send data at line rate across our 100Gbps for an entire month we get the following:

      100gbps = 12.5Gigabytes per second. 12.5 * 2 629 743 83 (number of seconds in a month) = 32871797875 Gigabytes 32871797875 / 0,001$ = 3.287.179,7875

      thats 3,2 million dollars... compared to roughly 4000!

      AWS also seems to offer "dedicated connections". at roughly 22 dollars per hour [3] (no clue if this is even comparble to an IXP port, but the comparison would still be fun to make).

      22$ x 720 (hours per month = 15.840$, or roughly 3times the IXP port price.

      In both cases, you are getting absolutely shafted by egress prices at cloud providers compared to doing it yourself.

      [1] https://www.ams-ix.net/ams/pricing-us [2] https://aws.amazon.com/ec2/pricing/on-demand/ [3] https://aws.amazon.com/directconnect/pricing/

      1 reply →