Our support team has reached out to the user from the thread to let them know they're not getting charged for this.
It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
Apologies that this didn't come through in the initial support reply.
One additional feedback, for consideration: to me, your Pricing page[1] doesn’t make it sufficiently clear that the “Starter” plan may incur costs at all (let alone in this ballpark). It’s now more apparent when looking at it in hindsight, but you have to either read very carefully, or go to the separate “View Features” page to understand this.
“0$ to get started, then pay as you go” reads to me: “0$ to get started, and then you can order add-ons and extra features as you need them”, not “$0 to get started, but we may start charging you virtually unlimited amounts at any point without prior notice”.
When signing up for the “Starter” tier initially, I completely misunderstood this. I didn’t have to enter any credit card or invoice details, so I thought as long as you don’t have that info from me, you can’t and won’t bill anything.
How on earth could I, as a customer, be sure that netlify hadn't paid someone to DDOS me? If I were in charge of a business like that, I would have that thought constantly...
> 0$ to get started, then pay as you go” reads to me: “0$ to get started, and then you can order add-ons and extra features as you need them
I think I disagree with this, but maybe I'm misunderstanding you.
Pay as you go sounds strongly to me that you pay based on your actual usage, not that it's free except for add-ons. A pay as you go phone, for example, does not imply you need to buy a telephony add-on, an SMS add-on, etc.
PAYG phones, however, were always prepaid, so I think I would expect PAYG hosting to be similar. That said, if my site was publicly accessible without my prepayment, I think it would be clear that it works the way it apparently does.
It's potentially misleading, but I don't think it's intentionally dishonest.
1. Yes. We've forgiven lots and lots of bills over the last 9 years and they haven't gone viral
2. While I've always favored erring towards keeping people's sites up we are currently working on changing the default behavior to never let free sites incur overages
Thank God for social media that the user was able to get attention about this on Reddit which he was then advised there to post this on HN. It must have been stressful to see a six-figure bill and then get told that that, no worries, you’d ‘only’ be charged $5k instead for a static site. It’s just ridiculous to me to be sent a 6-figure bill in the first place.
You don't see VPS providers like Vultr forgiving bills like this, nor do they make the news. Granted they are not the same scope as Netlify, but still.
if only i had $1 for every time for every time someone asked this exact question on HN. yes, we all get it: easy question is askable and not answerable. you want a gold star?
I’ve been a netlify user since 2017 and I just deleted all my sites. I can’t risk receiving a $100k bill for toy projects. Your “current policy” is not good enough.
Same, as it stands you the user are legally liable for the full bill unless netlify graciously forgive it.
Even in op's case, they didn't (still charging 5k!).
If there was an option to cap billing, or at least some legally binding limit on liability, then I can countenance using netlify.
Until then, it's just not feasible nor worth the risk.
the fact that once it arrives to the limits does not display an error page.
At this point I honestly do not care about they changing their policy, they should have thought that a normal person receiving a 100000$ bill on a free tier shall not been at all on the table in any circumstance, even if they forgive the bill, nobody needs to stress out like that.
Same. I will (almost certainly) never incur a $104k bill, but switching to Cloudfare Pages looks free and I don't want to depend on unwritten policies of goodwill to mitigate the potential risk.
Same here. Will I ever get a level of traffic that would cause this problem? Extremely doubtful. Is it worth the risk when Cloudflare Pages is a similarly easy offering, and took 5 minutes to switch to? Hell no.
The only "fix" here is to act like Hetzner and null route upon DDoS, price cap the thing, or offer unlimited bandwidth on the free tier like e.g. Cloudflare Pages.
Uncapped but paid is a recipe for disaster and you'll always be subject to the will of the support staff when something happens. If they can grasp to a straw leading to suspicions that it's not in fact a DDoS attack, you can for example be sure they'll do just that. Just no.
Did exactly the same, moving everything over to Cloudflare took me less than 15 minutes. “We’ll forgive those cases, pinky swear” is not a valid excuse when putting (even opt-in) hard limits in place is technically viable.
"Current policy?" So, you will retain a right to change such fees when you feel like it.
This is a serious matter. We are building a new site for our company with Netlify, but we can't open ourselves to this predatory practice. And even if you do not mean to be predatory, even the option of such is enough.
If not resolved with a clean, legally binding promise, our company (and probably quite a few others) must move our business to Cloudflare, Amazon, or some other competitor of yours.
Why are you asking this question here? Any actual company would have reviewed all the legal documents prior to choosing a provider. The promises you seek are the exact reason "enterprise-grade" providers can (and do) charge so much.
« Apologies that this didn't come through in the initial support reply. »
"Didn't come through" doesn't actually match the user's report of having support explicitly offering 20% and then 5% payment. It sounds like maybe you have a training problem? That seems like one of the important points to speak to.
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
That doesn't square with the 5% fee on the original $104k that your company told the OP to then pay.
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
Well, giving the option to users to plan ahead would be best, no? Like a setting to choose whether they want a potentially unlimited bill versus downtime.
Instead of that, you are choosing to stress and make people scared/anxious/homeless even (if they don't think of raising the issue on HN).
Seriously, this is not rocket science. This must have been discussed before in your company, and someone actually made this decision to stress people about such bills.
Frankly the only reason I can even come up with that Netlify wouldn't have such controls in place is exactly if they do _not_ simply forgive these sorts of jumps in costs (as the CEO here seems to be claiming). I'm pretty sure if they'd be left holding the bag, they'd manage to find some way to cut off these kinds of jumps in usage.
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
The legitimate mistake sounds to be on _your_ side if anything. You failed to match the attack pattern after all.
> Apologies that this didn't come through in the initial support reply.
The support email said you normally discount the attacks to 20%, but in this case it would be discounted to 5%. Are you here publicly claiming that your policy is to in fact to forgive (i.e. discount 100%) these bills? Was the support reply totally incorrect in claiming that you normally discount the attacks to 20% or are you lying when saying that your policy is to forgive the bills? You might want to clarify your position here.
To be fair, these days, things can become viral literally overnight.
That said, instead of depending on unreliable heuristics, they should just allow an option to change the behavior. The "current policy" to charge small sites on the free tier thousands of dollars instead of just throttling/shutting down the traffic is really predatory.
I understand that you need to pay bills, but auto-billing over the bandwidth budget just isn't OK, or at least not unless the user specifically configures that that's OK. I for sure didn't understand your bandwidth tiers that way.
You can avoid this sort of bad press and disgruntled users and your support cost by just giving users the option to shut down the site once the bandwidth budget is up.
Lol this deescalated pretty quickly, went from $104K to $20K to $5K to $0
Which basically means you almost scammed the customer for $5K or $20K. Super negative practices. I for one could never trust a company operating in that manner. It would be much more honest to say "unlimited bandwidth" and set a hard-limit for maximum budget, then people know they won't be charged, than to go through all this crap and then pretend you're doing a favor to the customer (you're not). If I'm normally spending $10/month any idiot out there would know for sure that I'm not going to spend $104K instantly. That's a very basic filter to have. But you don't place such filters because obviously you're working on the principle to scam people many thousands of $ if they fall for that. Heck, for all we know you might send that amount of traffic to your customer and the try to scam them and if it doesn't work then pretend you're doing them a favor.
Heck, at that point, why not "send some traffic" to your customer? It's not like they have any way of verifying its source. Hmm... why even send traffic at all? Just add a multiplier to their metrics!
This is very weird take. I'm struggling to understand why this is incident as a reflection of "super negative practices" or is somehow a "scam". The CEO came here and publicly apologized for the mistake and mis-communication, and the issue is resolved for the user with no charges. What am I missing?
You can't rely on such a policy if it is not part of the actual contract. This doesn't address the enormous uncertainty and risk that is present here when using Netlify.
This is what sticks out to me about the situation. I would much rather a site go offline due to service overage triggering at some limit that I set - simply relying on the good faith of a host to subjectively waive fees is not reliable nor does it instill confidence that I won't be financially ruined by malicious third parties (like nearly happened here). I would imagine that the good faith of Netlify in this case would mean very little to a court when there is a contract that stipulates costs for services, and the worst case scenario for a user is that Netlify could take the issue to court with the contract the user agreed to and demand full payment. Even the possibility for this situation to occur without any tools existing to prevent it is terrifying and is a terrible value proposition for a service.
By the time you forgive the bill you may have caused significant psychological distress, maybe even irreparable. This doesn't feel like a responsible approach.
This is the way most companies work unfortunately. Paypal limits your account and makes you wait 6 month to (maybe) give you a way to get the money back.
I’ve already migrated my two sites off Netlify after reading about this incident, and seeing other replies where folks said they were stuck with large bills.
This large bill doesn’t look like a legitimate mistake, it looks like everything worked as intended until things got escalated via Hacker News.
This leaves all your other small business users potentially on the hook and at the mercy of your mercy.
Not only should this stuff be capped rather than the dam allowed to flow, but your systems should have picked this up immediately and known it for its nature.
Thus must have been a nice little earner for you over the years.
> traffic spikes that doesn't match attack patterns
I interpret this as "we always charge for traffic served, but we attempt to block illegitimate traffic" which means of course that the worse their traffic discriminator performs, the more money they make!
I assume you'll be offering this user a good amount of credit on their account for having to deal with this BS and the stress of being told they owe you $100k?
So this one got attention due to some good Samaritan on Reddit who told OP to post here. Now, to the real question here: have others not received as good advice and just paid up?
I'm moving my domain name and personal site off Netlify (already deleted the sites, DNS transfer requested), probably moving to Cloudflare pages.
It may only move a few MB a month, but I just can't risk if I put anything more substantial there that I might get hit with a bill for $100k and you maybe will forgive it. And that this has apparently been policy for nearly a decade makes it even worse.
Sorry, but there is a lot more going on here than you addressed, these charges were incurred on your "starter tier", which has no mention of additional costs.. I've noticed a lot of "sponsored content" by netlify, and again no mention of this possibility.. Also, no comment on not having ddos protection, or at least a spend limit?
Sure, this instance was resolved, but it's also the top post of the last month. Who honestly things it would be the same outcome if not for going viral...
I’ve been a Netlify users on the Pro plan for a few years now. Moving from Netlify to CloudFlare after this; “this didn’t come through in the initial support reply” doesn’t cut it for a $100k bill.
But you do see how _not_ addressing this in the initial support reply is going to cost you all in the long term, right? The real lesson here seems to be for small projects, it may well be worth the investment to handle my own hosting. All I see here is that getting you to do the right thing required publicly shaming you, which means you can be trusted about as far as I can throw a piano.
I’d rather be shut down than have a heart attack from a $100k bill. That could literally kill me from stress, even if you pinky swear to refund any oopsies.
That is an outrageous and inhumane policy. People get panic attacks when they get told they owe 100k they don’t have. People will be terrified your internal process wrongly determines the bill is legitimate. Imagine you have to study for an important exam or that you have a paper due. How can you possibly focus with this nightmare at your doorstep?
The most bizarre thing is that this is a known issue that folks have asked them for ways to mitigate, to no avail. The reddit thread even links to an extremely weird dialogue where Netlify's response boils down to, "if you're hosting a small site that gets DDoS'd, don't."
https://www.netlify.com/security/ sez “Active DDoS mitigation — Netlify monitors for traffic pattern anomalies and spikes, and effectively controls for them as needed” and now I'm curious about what that actually means.
It means they protect themselves from layer 3 and 4 DDoS. For layer 7 you're mostly on your own. That's what most companies mean when they talk about DDoS anyway.
Playing ”devil’s” advocate: tracking spend in real-time is not trivial. It adds complexity to stack. Bugs in the feature can cause sites to go down (for long time) without a reason. Larger online businesses likely rather sort out the problems later than risk shutting down in the middle of unexpected success.
true. I have a 9€/mo vps at Contabo for my blog and once boasted on HN that my small VPS is able to handle reddit/hn hugs which one user seemed to take personally and they started a DDOS against my VPS.
I only realized this after Contabo contacted me and said the traffic is so high that other clients service is also degraded and they will have to take my VPS down if its much longer (which was understandable). Gladly the ddos stopped soon.
But never was there any talk about any cost, they were very supportive
To some extent, that answer is fair enough, assuming they make this clear up front. If their service is "we'll keep your site up no matter what, for a price" that's a fine service to offer. It's not what the vast majority of people want, of course.
If their advertising is targeted to small businesses and individuals who could never afford this type of service, they could be guilty of false advertising, at least morally guilty. I haven't seen their marketing so I wouldn't want to say.
I don't fully understand Netlify, but it seems though it tries to be a one-stop solution for everything it doesn't have to be - you could put free Cloudflare in front of it and probably mitigate this kind of thing?
Especially since they admit it was a DDoS attack. What I find outrageous is first that they charge for incomming traffic (which is often free with other providers), but also 55$ per 100GB. For comparisson, Hetzner charges you 1€ per 1TB of outgoing traffic while incoming is free.
So even a reduction to 0.2% would habe been possible. Honestly don't understand why anyone feels comfortable overpaying so much. Especially when there is no configurable spending limit.
Eh, I wouldn’t say that’s necessarily the case. AWS support, for example, tends to be really good about waiving charges for things that are clearly your mistake, like an unused instance that you forgot to turn off for a couple months. That’s not because hosting instances doesn’t actually cost Amazon anything! It’s because they want to keep you as a customer even if it loses them a bit of money right now.
In the Netlify case, though, insisting that this person still pay 5% is downright insulting. I’m sure they’re taking a hit already - just waive the whole thing.
AWS support, for example, tends to be really good about waiving charges for things that are clearly your mistake, like an unused instance that you forgot to turn off for a couple months.
This is an admission that their UX sucks and makes it hard to know what state your account is in and what you're paying for. They waive the fees because a few high profile cases of people paying thousands due to the AWS console being awful would drive a lot of customers away.
That’s not because hosting instances doesn’t actually cost Amazon anything
Except it doesn't cost them anything. The marginal cost of keeping your single instance running is $0 (unless they were 100% out of capacity and they could have sold that instance to someone else either at full price or spot price)
>it shows how disconnected this is from their real bandwidth cost
It's a value added service, they don't trade bandwidth as a commodity. Therefore unfair characterisation.
Plus, if you dive deeper: Bandwidth doesn't cost anything because bandwidth is just about pulsing some light in some glass fiber and applying some minuscule voltage on some metal fiber.Okay, maybe it costs some amount of electricity but all this is just a business model for paying on capital expenditure through time share arrangements. People can have all kind of models for this, for example you can come together with others or pay it all by yourself to install the equipment and have free bandwidth for the lifetime of the equipment.
It's all just arrangements to cover the capital investment and earn something on top of it. That's not a scam. A scam would be if they didn't account correctly for the timeshare usage or induce usage to boost payments.
I really don't get your point. If you're a hosting provider, the very thing you're selling is bandwidth (and disk space). Everything else is a value added service.
This is true for all businesses but maybe more so for tech:
Don't have a business model that charges customers for your mistakes.
This customers bandwidth usage jumped from free tier to $100k in very short time. To be honest, this shouldn't even be possible. Any "free" tier that allows for a surprise $100k bill is not a free tier.
This bandwidth usage is the result of a mistake on Netlify's part. That much seems clear.
To go and suggest that the customer is responsible for any portion of the bill is where things really went sour imo. Don't do this. Ever. Unless you want your company to go viral for all the wrong reasons.
If you want another good example of how badly this can backfire, look at what happened when Unity announced their new pricing scheme. Unity's new pricing scheme also allowed for unbounded bills. At first they didn't even deny this. Later they said it was a customer misunderstanding. I.e., they blamed the customer for their mistake.
Thankfully, the CEO of Unity was fired.
The lessons are very straightforward:
1) Don't implement predatory pricing schemes (this can even be done unintentionally, but the intent doesn't matter).
2) If you do implement predatory pricing, the worst thing you can do is put on your surprised pikachu face when the customer asks why their bill is bigger than their annual income.
Since the author has gone viral I expect some netlify exec is going to take over and write this bill off to $0. In the words of Kramer “these big companies, they just write it off!”
A moment of silence for the people who got DDoS-ed, didn’t go viral and still had to pay $5k.
The very fact that you expect that making front page of hn will make them cancel that bill means that it will soon be over.
These kind of stories (alongside cancelled accounts) repeat over and over again and will soon become so not newsworthy that they will either not end up in front page, nor people will check on the eventual outcome which means these companies will get away with not moving a finger.
Nah. I think the dev community has long memory. Events like this is damaging for years. Whenever Netlify is mentioned, someone will inevitably point to this thread for a few years.
When I received a message from the bank saying my account was in the red I discovered that AWS had been billing me 1100 / month for 5 months before I even noticed. It was for something I'd set up one night while bored and then forgot about it. They drained my account :( Even had the nerve to say I had to pay for premium support only to get a "lol, pay" response.
If you bend over and pay in such circumstances, you are part of the problem. Twilio tried to pull this crap on me and I simply created another account with an email forwarder and left them holding the bag on the previous account
Here is my shiny new super business plan for a startup that will profit thousands from a supposedly non-paying customer:
1. Offer “free static website” with lots of templates and guides to help you build one
2. The first 100GB is free and beyond that it’s $0.01/MB. But no worries! Very few customers actually use up that free bandwidth and in case you need more you can purchase packages for $100/TB. Also we offer a free service that will help you get your site more visible by advertising it, it’s included by default.
3. After a month or so, randomly help a customer bump the website and make it popular by putting it in some list that is frequently crawled. Secretly hire someone else to crawl these websites and make lots of download requests
4. Once the customer suddenly gets 10TB of traffic, bill them for 9900GB which is $99000
5. As long as 1 out of 100 customers pay, you are profiting $990 per customer! For the rest of customers, offer a 5% discount so they only have to pay $1980. Threat taking them to collections if they refuse.
Become the next millionare by just selling free static websites to 1000 customers! Anyone join us?
There's no way in hell anyone should ever under any circumstance use a free service that might, for reasons entirely outside your control, suddenly bill you 5k, or 104k... or any non trivial amount really.
Yeah. Elsewhere in these comments there's a link to a support thread[0] where Netlify support essentially says you shouldn't ever need an option to suspend your site at a certain point because:
#1. If you think there's any chance of getting DDoSd, you should already be on a business plan instead of a starter tier.
#2. If you think there's any chance of your site going viral, you're going to want to pay the cost anyway to let all those people visit.
I agree that's ridiculous and that the lack of any option of capping costs would mean I'd never sign up for the service. But that's the official response, for what its worth.
Yeah Like wtf is wrong with that? Are people just to lazy to check what the conditions are when exceeding traffic? I'd never ever sign up for anything that just keeps charging...!?
I understand that some businesses might want to take the hit from a cost surge because they get an even higher revenue surge. But a large fraction of sites aren't like that and would prefer a loss of service to a cost overrun. Service providers should always offer a "maximum out-of-pocket cost" service option. Those that don't aren't suitable vendors for most customers and customers should be warned about them.
As I recall, you have to actively sign up for the paid plan (Blaze) to get pay-as-you-go billing. Otherwise, you get free quota, and if it's up, it's up.
I think it also integrates into all of Google Cloud's billing management stuff, but I've never had to bother with that.
Use a virtual card with just a small amount of money on it to limit your liability. Won't work if you've entered a contract, but for a lot of these providers, including AWS it works.
This "one crazy trick" does nothing to limit your true liability.
If you go to a restaurant someone at your table orders 5,000 plates of mozzarella sticks, the fact that your credit card only covers $5 doesn't mean you are magically absolved from the rest of the bill.
For $100k, a debt collection firm would be more than happy to get a judgement against you. Credit card or no.
I once got a $65,000 water bill from the city for one month. I laughed and called them and asked them to re-read the meter and correct it, and expected a quick resolution. But no, they insisted it was correct for some time and that I needed to pay it. They said I probably had a leaky faucet or running toilet.
There was no awareness on the part of the customer service people how ridiculous that was. It would be physically impossible for my service pipe to deliver that volume of water even if it had been running full open for the entire month. I kept escalating until I reached someone who agreed, and they sent someone out to re-read the meter. And my bill was reduced to about $35.00, the normal amount.
Front line customer support isn't always very in tune with what is sensible for a given customer's account.
Water is a regulated utility. Anyone in a similar situation can contact the government authority who will gleefully tell the company to go to hell and possibly implement fines for inappropriate billing.
I wish that was so straight forward. You can google this incident where an empty lot got a 35K water bill and the water company said it was an error, then backtracked on that and still saying 35K is due.
> Towards the end of 2023, the DWM seemingly corrected the issue. Revive received an email stating: “The prior balance on the account reflected water leakage that was the result of Department of Watershed actions. Once the leak was addressed and the account properly adjusted, the corrected balance for the property is $219.24.” However, DWM soon backtracked and claimed that the $219.24 quote was made in error and that the nearly $30,000 balance still applied.
Although I am a very small customer of Vercel, I have been advising larger organizations on IT and data infrastructure for the last decade or so.
I can say with very high certainty that spending limits are a critical discussion point in every large organization when making IT decisions. I've observed multiple instances where a potentially better solution was not selected due to the risk of overspending.
This is my worst nightmare as a bootstrapped founder. And that there's no way to put a limit on spend is ridiculous. Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere.
Just went through Vercel's docs:
---
"Vercel helps to mitigate against L3 and L4 DDoS attacks at the platform level. Usage will be incurred for requests that are successfully served prior to us automatically mitigating the event. Mitigation usually takes place within one minute.
Usage will be incurred for requests that are not recognized as a DDoS event, such as bot and crawler traffic.
You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP, User-Agent header value, or other identifiers."
---
That doesn't help me sleep well.
I feel that by now, these hosting providers should simply adopt best ddos protection practices and take responsibility for failure to protect.
"You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP" - there should be some really good defaults for this right?
Clearly it's possible - Cloudflare's ddos protection is worded more strongly.
I'm willing to pay more for the service for peace of mind. Like, even $10/mo more to insure against getting smacked out of nowhere.
Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.
CloudFlare pricing is indeed positively ridiculous.
At OpenTofu[0] we’re using CloudFlare R2 to host the providers and modules registry[1]. Bandwidth is free, you only pay for requests.
This already would be great, but there’s more - you only pay for requests that actually hit R2. So with an almost 100% cache hit ratio, we barely register any billable requests.
Recently someone decided to load test us and generated ~1TB of traffic over 1-3 days. All but a few of these requests were cached, so the whole situation probably cost us less than a cent.
> Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.
Yeah that's how Cloudflare can reach total control over the Internet. With thunderous applause by people that should know better.
I know that my position is outright blasphemous in this day and age, where even self-hosting a static site has become black magic and we need a third party to do it for us.
> Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0
It's not really ridiculous if you think about what you're giving them.
You are massively benefiting their platform by providing them data which they use to train their services and then sell those services to other customers.
I'd make a case that the data they collect is the most important part of their business and the free tier is a major component of this.
I don't think it's fair to call it their free tier - it's their discretionary tier, there are numerous cases of the rug being pulled as and when it suits their business requirements to do so. Being left homeless vs. urgently coughing up is exactly the wrong problem to be dealing with mid-attack, I can't see any way to consider it free by any practical definition
I know that putting all eggs on one basket and giving it all to Cloudflare is not a good idea, if they have an outtage then I would also have it to. But when they are down, one third of the internet is down with them too. With 240$ a year for CDN, 60$ a year for serverless and $0.015 / GB-month for S3-compatible storage with free egress, I don't think anyone could find a better alternative than CF.
I'm mixing with AWS, CF and self-hosted machines and the infra cost is less than 5k$ a year. Now I can spend the remain hard earned money for some fresh marlboro cigarettes.
Use a token bucket on your web server to catch abusive IPs and then blackhole them using `iptables -t raw -I PREROUTING -s ip -j DROP`. I know. I run https://ipv4.games/ which invites hackers to unleash their botnets, and the service runs on a small VM with only a few cores. It's been attacked by botnets with 49,131,669 IP addresses. There's no Cloudflare frontend or anything like that, because back when I used Cloudflare, the people who attacked the service would actually bring down the Cloudflare nodes before they brought down my web server. I doubt I've ever paid more than $100/month to operate the service. Please note that your service provider needs to have free ingress in order for this strategy to be effective.
This strategy may work for a (D)DoS that is targeted to an application layer, but won't work if the attack is designed to exhaust your bandwidth.
Once you're receiving more traffic than you network cards can handle, it does not matter if you'll drop the packets with iptables or not.
I was the target of attacks that caused Hetzner to terminate my contract. I was leasing physical servers there, so I assume the attacks were overwhelming their infrastructure.
If you want to sleep tight just get a dedicated server or VPS from something like Hetzner and/or combine with CDN providers like BunnyCDN - set up alerts just in case though. It takes more time and resources to manage it but you could save a lot on it in this case.
This so much. My hetzner (best choice for a media server within Europe) has 0 downtime in 1.5 years. And exactly as you said I am using bunny as well, which costs me a few $ per year.
For most of the new web projects, setting up your brand new server is pretty well documented process and should not take more than couple of hours.
It get complicated when you grow and add more servers or components. But at that point, you should be able to afford a part-time consultant to handle complicated tasks or just use Cloud then.
Yeah, I'm already using Cloudflare because of Google Domains got de facto killed by Google via transferring it to Squarespace. Why not Cloudflare Pages, CDN, and R2 (S3-compatible storage) too? I'm even considering paying for the paid tier in the future if I ever go above the limits of 20 000 files per static site and the 25 MiB single file size limit [^1] (more than enough right now or in the near future).
I was looking for a static site hosting option recently and tried out cloudflare pages. Fit my need perfectly. The generous free tier and the reasonable pricing model were the big factors.
Oh, and the ability to put some authentication in front of it was a big feature for me.
Host on a provider which bills per hour. This caps your cost. It also makes your users pissed because you will go down, but if you’re small, you can afford that. If you’re big, you already have scaling options and should have a team to handle ddos.
My experience is that customers don't really care that much about small amounts of downtime no matter what size you are, people mostly get that unexpected stuff happens as long as you don't get hacked or misplace their data. Customers might complain a bit but seldom leave because of a few hours downtime.
This seems to mostly hold true to developers also, GitHub manages to survive just fine after all.
> Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere.
An interesting story that expands on the above concept but a different vector entitled, "Illegal Life Pro Tip: Want to ruin your competitors business?" :
https://news.ycombinator.com/item?id=36566634
Imagine you lost your job. So you are here enjoying creating and hosting your hobby projects in theses services. Now, suddenly one fine morning you get slapped with $104K bill because someone decided to randomly ddos your one page dog lover website.
Now, who in the would would be thinking of having ddos protection for their hobby project? This is just absurd thinking.
No. This is absolutely common. I remember well how shared hosters 10 years ago already put caps on cheap packages and took the websites offline in case of traffic. And today it's Amazon who bills small players into dept.
This may seem weird, but I believe ToS ae the real problem here. I call it the "car rental" problem.
When I rent a car in person, I am often given a contract. And this contract is filled with tiny print, and pages of it.
There are often people behind you, waiting, and bored/annoyed people behind the counter, waiting. This is beyond unreasonable.
A point of sale contract should be short, in readable text, and understandable. For example, renting a car? Under a page, easily parseable, and if the person behind the counter cannot explain it, it is null and void.
From a legal side, you can do this. And you can explain legal terms. Of course this means you are describing intent, which limits one in court, oh boo hoo Mr Lawyer. Cry me a river.
Well the same should be true of any retail contract. Sign up for a service? One page with costs listed.
At least then, there is hope of an end-user sort of understanding. And as one could claim that a DoS was actually targetting the provider, and not the website, that should be described too.
So back to the topic at hand. I would write a demand letter, insistong Netify explain the charges, and ask them if they and their IP ranges were DoS, and if so that the charges be reversed.
Because you shpuld not be paying, if someone attacks Netify.
This letter should also be sent by mail, sig required, to the corporate address too.
Unfortunately, in today's world, DDoS protection is the equivalent of basic hygiene, foid and road safety. It's just a travesty that the hosting providers don't feel like it's their responsibility to address it.
“We leave your safe deposit box unlocked. You might want to forge your own lock and key. If we happen to notice someone stealing out of your box, we will let them grab as much as they can for one minute, then maybe install our own lock if our revenue is close to target.”
More reasons why I avoid clouds with outrageous bandwidth fees, and prefer Hetzner's low cost fixed pricing with Cloudflare R2's 0 egress fees.
Even if the DDoS wasn't caught by Cloudflare, the total cost for 192TB bandwidth on Hetzner would be €172. Although even after 10 years on Hetzner I've never paid for any bandwidth, always well within their generous 20TB free bandwidth.
First of all Hetzner would't let your server to be DDOSed for 192TB if it's not your normal usage. They'll likely just null route your IP if serious attack hit.
They also likely drop any charges if you escalate via support in case it was actually DDOS. E.g if you normally have 100GB / month and now you magically have 50TB / day.
I really hate Clouflare and at the same time love them. Love them for their free, generous 500k req./day pages and workers, hate them for not having spending limits (or at least I can't find them). I get that they probably don't care about individuals and small businesses paying them peanuts, and corps can afford to pay extra if something blows up, but for me it just means I will only use a free account, and they get none of my money.
I also take advantage of their free services and only pay for R2 on Cloudflare since it's the best value managed provider I could find.
I prefer to keep my App's stateless and running in Docker containers which means storing all uploaded files and generated assets in R2 managed storage - which is also used for Litestream backups of our SQLite databases.
Blowouts are minimized when using low cost services, e.g. we had a rogue process that ended up causing 1.5M writes to R2, which only ended up costing us $4.50 in that month.
I recently rewrote a website/small backend API for a non-profit organization. I could've gone with a serverless architecture for our forms handling API and reduced spending to nearly the free tier, but I had no good way to protect against a scenario like this. There was just not good enough documentation about how to completely cut off spending in the scenario of an attack, and I wasn't comfortable leaving the organization open to a cost attack like this.
So we're using Github Pages for static hosting and a $5 box from OVH now. Unmetered bandwidth, plenty resources for our purposes. Cheap enough, and we will never, EVER, have to worry about an attack like this. Well worth it imo.
Imo, serverless is great for internal jobs where you can control spending. For public facing things, you have to be a lot more careful.
Yeah I'm doing the same, a combination of OVH, Scaleway and Hetzner.
Sure there is no such thing as "free unlimited" bandwidth but I much prefer unlimited with a fixed cost until they decide it's not worth it and shut me down vs unlimited risk with no ability to cap it.
The lack of cap is the worst part and it's 100% a business decision. Every provider who tracks bandwidth could add a cap but they just choose not to because it's too profitable and the risk is mostly* on the customer anyways.
*there is of course a tiny chance they the customer goes bankrupt and they get almost notning, but usually they just need to pretend to be nice and forgive all or most of it
I just checked pricing on hetzner (not affiliated with them). First 20 TB are included with VM price, after which it's 1.19 euros per TB. If this happened to someone hosting there it would cost them additional 50 euros. Can't believe the difference in traffic pricing. Netlify is over 1000x more expensive per TB.
People will throw up all sorts of excuses for not just renting a VM for a static site - scalability, ease of use, security.
But in the vast majority of cases you could just take a $5 VM, apt-get install nginx and be absolutely fine. A tiny bit more effort and you can make sure it's always up to date and very secure. Plus you get a VM you can use for other things when needed.
Just regular managed web hosting is enough for simple websites for people who don't actually want to manage a whole server. You don't have to manage anything with such a plan, just put your files where you're told and that's all. It's how simple websites have worked for decades.
Hetzner's plans start at 1,76€/month with a domain name included and unlimited traffic. OVH is slightly cheaper.
I'm probably too old (?) to understand the appeal of Netlify or other similar services, but I really don't understand why they get used.
By the way, Netlify says "100 GB bandwidth" is included with the free plan which I thought mirrored Hetzner's number of "30 TBit total bandwidth", but you have to click the details to see that it's 100 GB per month. So not bandwidth at all, but traffic.
Problem is if you get 10x that traffic, which would already be 500€ in Hetzner, right? Of course its much cheaper and it's very unlikely that you'd get a huge bill but with them, but after all, it would be great if you could just say "Shut my machine down when my monthly spending gets to 100€"
That sounds like a company that is happy to ignore customer pain. Agree, default-on telemetry that cannot fully be disabled is a red flag that on their side, the people with development experience have no say.
This seems like a lot of brouhaha about nothing. They just recorded the fact that a user opted out of telemetry, they didn't stealthily send telemetry against user wishes.
A few years ago I taught an introduction to website developmment module at a university where students built jekyll sites. I got them to to host on netlify. Now that I no longer teach at that uni and their email accounts will have expired I'm wondering if there's any way I can contact them to tell them to take their sites off netlify... Disaster if they get hit by charges like this.
Not necessarily, the majority of them probably wouldn't have ended up in anything tech related. But they probably did end up back in China, so hopefully it would be hard to demand payment at least?
Netlify charge $55 per 100GB for overages on their starter plan. Keep in mind that they’re being billed by their provider at 95th percentile billing. That means they have ports running at say 100 gigabits per second and have a commit of say 50 gb/s and they’re billed a flat rate for that as long as they don’t go over. Because they’re billed 95th percentile they can spike their connection traffic massively for 5% of the time and not get billed more. So Netlify themselves have a safety net and don’t need real-time monitoring to immediately cut their usage. And of course any spikes from a single user are massively diluted among their entire user base. So, yeah, they’re gouging big time.
What they should have is monitoring per user and a default that 503s the site with no overages that has to be proactively disabled by the user. Instead they’re just letting it ride and trying their luck by negotiating down the overage charge to what they think the user can stomach.
It's also worth pointing out that Netlify's pricing of $550 per 1TB is 5x more than AWS, Google Cloud, and other massive cloud providers charge, and those are already known for extortionate data transfer fees (50-100x more than more reasonable providers).
And it is, I remember cases where telcos were prevented from charging thousands for roaming expenses because they were not licensed to make large loans.
Oh God roaming is one my biggest nightmares and it’s the worst UX to not get charged, too. ‘You want to go on vacation? Make sure to hunt down an option hidden 3 levels deep and enabled by default because we really like money and you didn’t read the fine print? Too bad, we really like money’
looks like it is not just defaulting to unbounded liability. it is forcing unbounded liability as there is no way to turn it off and limit your exposure.
I have been using Netlify for years, for my own projects, but also recommended it to all my freelance clients to host the projects that I was building for them.
Going forward I will move all my static pages to other hosting providers.
The Netlify team must think:
we waive the fees, because in this instance we noticed the negative press and want to avoid this from blowing up. When this happens to other users, we don’t care, as long as it does not go viral.
Such a pity, Netlify has great UX and I was so happy hosting static pages on their service. But without spending limits, this is not an option for me any more. I could not sleep well when there is a possibility of a $10.000 invoice reaching my inbox.
There is a lot to be said for just hosting your own VPS and learning how it works especially for smaller sites. The effort to host is small and you're not going to get smashed with a bill like this.
This and that similar story recently about Vercel makes me feel iffy about these services at this point. I guess I’ll either use Cloudflare pages or simple dedicated machines from Hetzner going forward.
I was searching for 'cloudflare spending limits' based on comments here, at first glance they (like Vercel) seem to have a notify don't terminate policy.
Have you tested it end to end? When was the last time? I'm slightly worried about solutions like that. I like when it's all host's responsibility. (Having your own system on top is nice though)
This happened to me on a much smaller scale about a year back. I was never happier to have stuck to my guns, building my whole site with a single `hugo` command - it made it very easy to migrate off that platform for good.
If anyone has a solid bash one liner to stress test a website, so that I can test whether my cloud billing cap will work correctly if I accidentally try to egress 100 MB of data or something, I would seriously appreciate it. There was one on a blog post here like a year back, using apache iirc, but I forgot to bookmark it.
apachebench (cli command ab), is an Apache licensed stress testing utility, not much relation to the web server, which can be easily used to stress test a webserver.
the binary should be easy to install with your package manager, you may have it installed already
I am a business user with netlify. I have unlimited functions calls on that plan (fair use of course!) and use their JWT protection that redirects to a login site at the cdn level, so you can rate limit. Not a solution for public static sites though! You are metered on the starter and pro plans after the starting limits.
They seem to have dropped that plan now.
I was starting to move back to traditional hosting as these platforms are convenient, but you do lose control and get hammered for their addon services and simple things like static ips are beyond them, even if you offer to pay.
Also, if their cdn is naughty listed, corporate networks may block your site as you are sharing pro and business plans with free sites that maybe serving malware etc.
Hearing this story has pushed me to move.
I hope they sort that for you, they really should have the ability to protect a site and let you choose what to do if you are exceeding your limits.
It's impractical because residential internet speed is very asymmetrical and behind NAT. You will still need external service to get around the NAT thing and the fact you don't have a static IP.
You do not need a lot of speed to host a personal or hobbyist blog, and NAT is just something you configure once and it's done. I use my domain providers API to update the DNS records as soon as my public IP changes (with a tiny shell script on my server). I even host some semi-commercial sites from home and it is fine :)
I would much rather run the chance of some occasional downtime in exchange of being in control of the infrastructure and owning my own data. I really like the idea of a inter-connected net that is kinda spread all over, not super concentrated to a handful of data hubs.
That could be a nice backup for when I experience downtime, but I would rather run a simple webserver at home for stuff that does not require extreme scale or a lot of resources. Hosting some websites at home is not that complicated and it keeps the dream of a distributed Internet alive!
This is the reason I've never used all of these user-facing serverless services. The price depends on the usage, but if anything goes wrong they are the ones to decide what you pay. It's not comfortable thinking that you could screw up, or get DDoS'd, and the remedy is hoping they wave the bill.
We need a separate ‘shame’ tab at the top for crap like this. There must be so many more of these asshole bills being sent out that aren’t lucky enough to get voted up.
What the hell. Netlify has implemented all these customer-hostile billing changes. Literally the worst. I migrated my company's site off of Netlify when they tried charging for per user on one of my repos (a documentation site). Glad I left them and never looked back.
Being able to set a soft limit (email me when I go beyond it) and a hard limit (shut down my service) seems like a very simple solution. How is it not a thing in 2024?
Also, being at the mercy of a CEO and their understanding is a no go, whether you're a company or an individual. We want to be 100% sure we're not going to get bankrupt when using a service. "Don't worry, our policy is to usually erase the debt" is not an appropriate answer.
Really uncomfortable with how many services like this are “we scale to your needs” and end up here. I guess capacity planning from oversubscribing bandwidth is its own can of worms but surely that is a bit… less surprise generating.
A successful distributed denial of money is likely much more devastating to anyone except really large companies than a successful DDoS. For a personal accounts it is entirely devastating with no upside, but even for a startup it would probably be better for your site to just go down instead of having huge bills generated.
I guess I never want my personal blog/site to be successful. Can't trust that I will wake up to a huge bill from EC2 or my CDN provider. Feels like health insurance in that even when you have insurance, you are never 100% confident you will not get some huge hospital bill. The pessimistic part of me says this is all deliberate to prevent the "little guy" from competing these days.
If you run on a single EC2 instance and you aren’t running an auto scaling cluster or anything of the sort, it would be pretty hard to get a huge bill. I much prefer that and the chance that it goes down then autoscaling or severless. Most serverless solutions have also gotten so config heavy or complex to make changes that most projects feel much better to me on an instance I can ssh into and poke around without having to call up support.
Thanks, I use a "small" EC2 instance for my personal stuff (about $35 a month), with the Cloudflare free version, but I honestly don't know and what would happen if something went viral or a DOS. Would the bill be double? Triple? Would the site simply crash. What does Cloudflare do? I honestly have no idea.
Cloudflare. You will get a call if on a free or pro ($25/month) plan [1] if your bandwidth usage is so high it would warrant increasing your plan. Worst case, they turn you off. Preferred over denial of money attack based on your use case. Set and forget after pointing at your origin (time is money).
You'll get a call on any of their plans if your bandwidth usage exceeds certain thresholds, I am assuming your median usage is relatively tame.
Disclosure: Cloudflare enterprise customer, no other affiliation. I don't get anything for saying nice things.
You typically can't replace Netlify with Cloudflare. You need something like Github actions with some storage, S3 or something and then one can put Cloudflare in front of it all for caching, DDOS protection and so on.
For less than 1% of OP's monthly bill, you can build or obtain a more-than-enough server, drop your static files on it, and serve through nginx. And you get to keep it forever; there's no monthly subscription fee!
Seriously, maybe I'm just old, but I look at the pricing of these hip and modern SaaS products for dead simple software and I cannot believe my eyes. The "old fashioned way" works just fine (and has always worked just fine) and is orders of magnitude cheaper.
Hetzner or DigitalOcean with Coolify [0] works great, it's like an open source Heroku that runs on any host, you get git push to deploy, and a bunch of other features built in. It only works on one machine at a time though so it's not like a CDN but for small sites, it's great.
Love Bunny too, wonderful service and great team. I wish there'd be an easy way to set up auto-deploy to Bunny Edge Storage on GitHub commit (to avoid doing so manually), but I guess it's not to hard to do through GitHub Actions.
I switched from Bunny to BlazingCDN, with same performance...
1000TB - $0.0025 per Gb
And the support helps a lot, they even made the config for Asia for free
Just something to keep in mind: this also has about half of additional bandwidth costs (above 100GB) of the reddit post, so in your case you'd be billed for ~$57k or so with similar DDOS. At least they seem to provide monitoring/alerts based on their Security page.
For personal servers you can use the dynamic DNS feature on your modem in combination with Cloudflare if you like doing it at home, if it's for your business you can see if you can afford the €2 per month for Hetzner managed hosting or your local equivalent.
If it’s really pure static, github is great because it’s impossible for you to be billed. If you want some functions, Cloudflare free plan is nice because you can configure it to stop operating when the usage limit is reached, or pay $5 a month for more than you’ll likely ever need for a hobby project. Also bandwidth is free.
There's a reason price plans are 'pay as you go' - so that they don't take on any risk for things like this. Moral of the story: never go for pay as you go by usage plans if this scenario is not mitigated through a third party.
By no means am I trying to suggest a get out for Netlify here - more a rule of thumb for anyone worried the same could happen to them.
I don't see why people are surprised by this or why people are calling it a scam. Netlify and others are extremely transparent about the fact that there are no limits. I completely understand not liking it and can see why the lack of limits would make it a bad option for plenty of people, but I don't see how it can possibly be called a scam.
Not to mention the strong conflict of interest for netlify, who stands to gain from their customers being attacked. Netlify is getting paid for something criminal in nature having occurred.
It's like who is responsible for credit card fraud? If customers are responsible for credit card fraud, and it's their responsibility not to get scammed, then who implements fraud prevention measures and what effect would that have on the volume of fraud?
Companies like these give out ridiculously huge free tiers in the hopes that very few users end up using the high free bandwidth limits. In most cases, they do. However, they do need to make their money back somehow.
I don't really get why people put their tiny static sites on hosts designed to never fall over no matter the traffic generated, no matter the situation. You're running a blog, not a government service. You don't need AWS or Netlify.
The ability to withstand almost any DDoS attack for a high price is a valuable service. It's not a scam. The people who get these huge bills just picked a hosting service that doesn't fit their requirements. I can promise you that the $3 shared hosting providers won't charge you $5k, five minutes after the DDoS starts your site just goes down.
> Not to mention the strong conflict of interest for netlify, who stands to gain from their customers being attacked. Netlify is getting paid for something criminal in nature having occurred.
I think you could argue that Netlify is guilty of racketeering in OP's case.
1. They admit illegal activity happened (a DDoS attack).
2. They demand money to be reimbursed for the illegal activity. However, the reimbursement they ask is several hundred times higher than the actual damages incurred.
My previous understanding was that service would be stopped once you hit past the free tier.
Upon review, it does not look like this is the case. I have several very low traffic projects on which would have never been anywhere close to the free limit. However, if I get involved in a random spam attack, it seems I could be on the hook for several thousand dollars.
This is incredibly dangerous. Netlify is often used as a beginner friendly free tier for static hosting. Not as something that is cheap, but as something that is free. This is just an overall dangerous position to put people in.
It does say it's pay-as-you-go on their pricing page. However they probably should have a giant warning page for new users who don't know that this is how this kind of service works if they want to target the beginner web-dev market. As far as I know, no other similar service has this though.
I looked up the definition of scam and the goal is to make money out of the naivety of the victim. It involves a crisis, the illusion of shared exposure to a risk.
The price of cdn bandwidth is about 0.01/gb on low volume (cloudflare, aws, azure…) so op should be billed around $500 with 40TB. Netlify probably buys this for way less. He was presented a bill at $104k, « generously » reduced to $5k, still a x10 margin. Vercel and Netlify are outrageously expensive for what they do.
>Netlify and others are extremely transparent about the fact that there are no limits
Are they also transparent about the fact that they
1. Won't do anything about a DDoS, and
2. In case there's a DDoS (or some other unusual traffic spike), you'll only get notified waaaaay after the fact when you get the $100K bill, instead of getting a timely alert that would allow you to shut your site down to prevent getting extreme charges?
The primary purpose of these services is to be able to scale up and continue working under heavy load, shutting the site down when this occurs would defeat the entire purpose of the service. I would say that they are transparent about both of the things you have listed by virtue of being one of those scaling serverless hosting services.
I understand the lack of limits but I haven't accounted for DDoS attacks on Netlify infrastructure to impact me. I was assuming this only included real, "organic" traffic.
What I think Netlify needs on their Plans page is to include "DDoS attacks is included in your traffic" as well as their 20%/5% charge system.
> and can see why the lack of limits would make it a bad option for plenty of people
Just out of curiosity, can you see any scenario where it WOULD be a decent option to use a free tier where you may be hit by a $20,000 or $5,000 bill out of the blue and outside of your control? You say "plenty" so I assume you consider this a reasonable system to some?
It smells like a scam, because they can suddenly bill any user they want for a scary number like $100,000, then when the user complains they "generously" reduce that to only 5%, or $5,000, hoping the user will just pay the massively reduced cost. This kind of thing - showing a huge number upfront then reducing it to a "small" number - is a classic scam.
Who controls the DDOS bots? Are they truly a separate entity? There is no direct evidence to link them together, but you would think that an honest company would be more proactive in preventing problems like this for their customers.
According to the linked reddit story, this is a known issue with Netlify and their response to past incidents is basically to pound sand. It all adds up to them purposefully trying to find ways to generate a high bill for their customers and hoping a small amount will pay for it.
If it’s a static website; use GitHub to host and put behind cloudflare.
If you need a nice front end, spin up a Wordpress instance with a provider like digital ocean or vultr or any other number of places. It’s like $5-10/ month and has terabyte bandwidth without issue typically.
Then put the site behind cloudflare or at least configure a plugin for ddos protection.
Netlify doesn't offer alerts or budget-triggers? This is my nightmare every time I put up my card details with a cloud service provider.
The funny thing is - such platforms "scale" easily with the underlying assumption that scale equals profits, enough to justify increased cost. Needless to say, inaccurate assumption.
The test for federal class action lawsuit includes 4 prongs, all of which must be satisfied: numerosity, commonality, typicality, and adequacy. [1]
I would think (as a former lawyer with only passing familiarity with class actions) that 'typicality' would be the key question.
> to determine typicality the courts consider to what extent plaintiffs’ claims are markedly different or are generally the same (for instance arising from the same event or pattern) as those of other class members with respect to the relevant legal theory and factual circumstances of the case. [1]
The defendant would probably claim that each plaintiff's issues are quite unique. However, this prong is apparently not based on the typicality of the specific facts giving rise to the lawsuit, but rather the typicality of the nature of the claim or defense. And it's apparently hard to 'win' (defeat a class action) via this prong. [2]
Been using Netlify for 7 years now and I'll be moving all of my projects to cloudflare this week. That's a hell of a lot of risk to host a few hobby sites that virtually nobody visits. Cant imagine running a bootstrapped business and having this happen. Seems extremely predatory.
Free plan can be used with some iamnobody@gmail.com email and without any credit card info. It looks like good idea because I thought too that Starter plan is 100% free and if you would use all yours resources they should shut down your page and wait for improving plan/adding credit card details.
I would not like to pay for something which was advertised as free (don't have a nervous for paying without hard limit for home project, I can host it even on rasberry pi + pay just for address). That's why I find it is better to not use your real name as long you don't want to make real business :(
And that’s how you lose loyal customers who have evangelised your products for years. I’ve helped local mom-and-pop stores, bootstrapped startups etc setup on Netlify and now I fear that Netlify might send them a hefty bill.
I’ll remove all my websites from Netlify and moving to Cloudflare. Auf Wiedersehen.
"...Top to bottom, our infrastructure redundancies make sure we keep traffic flowing, so there's no need to add more redundancy with Cloudflare. ...
You don't need Cloudflare when you use Netlify
As you can see, we already offer what Cloudflare does, and more. If your site is not on Netlify, perhaps consider us for your one stop solution for hosting, SSL, DDoS protection, DNS load balancing, and continuous deployments. ..."
Just switched my site to Cloudflare Pages. It took all of 5 minutes including futzing around with DNS stuff through the (ever-annoying) Google Domains settings page.
I'm usually pretty chill, and can't help but feel like I'm overreacting - but the risk of being sent an unexpected bill of even a few hundred dollars for something outside my control is unacceptable. Trusting that Netlify will do right by me and forgive the debt in a similar situation is not an acceptable strategy for managing that risk, so epiccoleman.com now lives at a provider without this risk.
This is the beauty of static sites, btw - switching to a new host is the work of mere minutes.
Your comment with it taking only 5 minutes made me move my sites. And indeed it was super fast. Thanks for the needed motivation!
I don’t think you are overreacting. I had Netlify in pro mode at work and their pricing tactics are predatory. It was really hard to figure out why I got charged a seat, even though nobody had rights etc. And I couldn’t then remove these seats, I had to wait a month and hope I figured out where those contributors came from.
I have a bunch of pet projects on netlify free tier and I could never afford to pay this amount of money. What are some good alternatives that don't have this issue? I've already noticed cloudflare pages mentioned in these comments.
Static: Github Actions to build and deploy to BunnyCDN
Non-static: selfhosted Dokku on Hetzner
Neither is free, if you're looking for free, Github Pages or Cloudflare for static sites. Free non-static, I'm not sure there are solutions that don't have the same problem Netlify has.
I’m stunned there is no way to limit spend and have warning and a cap on a service like that. You should be able to specify billing thresholds and have sensible defaults when the downside is basically unlimited. Basic customer UX.
Does this issue only occur if you have billing info on file?
I'm using the free tier and have no billing info set. According to this https://github.com/netlify/ask-netlify/issues/6#issuecomment...
> if you have an event that puts you over the free-tier limits, Netlify will ask you to update your billing information and add a CC
Although worryingly
> We just had this happen and our site didn't stop working.
Is there any way to ensure if you hit the limit sites just stop working and you don't get billed?
I'm also interested to know this. I have a couple of static sites running on the free tier for friends/family and now I'm planning on moving them all to a VPS as soon as I can.
It is beyond ridiculous that serverless providers don't offer a way to cap spending. The idea that it might cause your site to go offline is a complete non-argument. That what I _want_ to happen. I want to be able to say sure, I'm happy to sustain 10x traffic for a few hours, and maybe 3x sustained over days, but after that take it offline. I don't want infinitely scaling infra precisely because of the infinitely scaling costs.
No, and this is by design. If you go over the limits (can also happen if a build machine times out, ask me how I know), you will be billed without any recourse. If you have no billing information and refuse to set it, at the very least they'll permanently ban you from their platform.
Which, if it remains the only consequence, seems like a blessing now.
Wow, insane. I would not have guessed that they simply charge you for additional bandwidth instead of just shutting your site down for the rest of the month.
This is not something you would assume from the pricing page. On the pricing page, they show you:
Add-ons:
Additional bandwidth
Additional build minutes
Additional teams
In my understanding, an Add-on is something I need to enable, not something THEY enable for me if they see fit. When I am logged into Netlify, they tell me:
No currently enabled site add-ons with fees
Which apparently means nothing, as they will just automatically enable the add-on for me!!!
I have seven static sites on there, deleted 3 of them right now, will migrate the other 4 soon. Unbelievable bullshit.
I had a couple of orphaned static sites on Netlify. Just deleted them, and won't put anything else on there now.
Now I get that this is their 'product'—selling hosting to high traffic customers, and I don't particularly begrudge them charging whatever margins they think are suitable for their product and let people make their decision.
But no mechanism to cap maximum spend is completely ridiculous. Even if you require a minimum cap of say $5/$10 dollars, as long as it's clear and transparent, I think that would be reasonable.
Anyway, this has scared me into never trusting Netlify with anything any more.
I must add that at Netlify are consistent across disparate systems.
They consistently missed someone that doesn't get more than 10GB traffic a month now maxing out a 5Gbps line (60TB/day stated in TFA) for several days in a row.
And consistently missed someone with a tiny bill's now racking up 10s of thousands of dollars per day. Even if they do run a mainframe and batch process at the end of each day, that still went a few days. If extending lines of credit of tens of thousands of dollars is legal, that's very generous.
It feels even worse than that if you consider with this being widely known, competitors could make a point to attack Netlify customers knowing it may bankrupt them.
Just to toss this out there: There are too many services that don't really let you cap your spend. AWS, for example: You can set alarms, but as far as I know, you cannot sent a spend limit.
I had a couple of small businesses hosted there, but this always worried me, so I moved them to a local provider, who provides what they need for a flat fee.
Is the entire idea of paying for bandwith not absurd ? You can't really control it
I have some websites with milions + autogenerated webpages and it's flooded by bot activity that I don't particularly care about.
I've blocked some through cloudflare but some look exactly like he describes: old machines with old versions of some OS and browser scattered around the globe that seem to be scanning my entire website, maybe for AI training purpose ?
A lot of today's managed service solutions turn scaling problems into billing problems. While that's a useful tradeoff for many business use-cases, there should still be a way to set limits so you don't wind up with insane cost overruns.
You just posted it on Reddit and I already see few comments of people say they will never use it. Stuff like this can cause PR damage worth millions to a company. I'm pretty sure that it will go viral and they won't charge you.
I can't get past why Netlify's infrastructure continued serving a 3MB file to a botnet 55,000,000 times? I would've assumed their system was smarter than the ubuntu/apache install in my basement.
That it why I always refuse to use hosting services with on-demand pricing, especially for personal projects: in the worst case scenario I'd rather see it down for a few days than be indebted for whatever amount.
That's approximately one month of transit through a gigabit connection, which indeed could be pretty expensive. Even at IP transit prices it would be something like $300/mo for a 1GB connection via HE but you would need something bigger to handle this traffic. I pay about $2k at bulk for that amount of bandwidth. Any way you cut it, you're getting some sort of bill (or kicked off) for that without some sort of ddos forgiveness.
That said, $104k feels excessive to me for static hosting ($0.57/gb, did I do that right?).
I'm a huge Netlify fan, but stories like these are scary. Just spent the last hour building a custom serverless function that sends email / SMS at various bandwidth usages - potentially could be improved with an auto shutdown of the offending site beyond 75 GB:
It’s crazy to me that platforms like these don’t have a reasonable spend limit set by default or easily configured.
I understand it can be difficult to stop spending exactly at a certain amount to the cent or event to the dollar or, hey, tens of dollars if traffic is really crazy. But hundreds? Thousands? Tens of thousands? Your system should be able to measure that and stop it in time.
The lesson to be learned is to pay attention to pricing before signing up, and don't buy really expensive services. Assuming a gigabit connection, you have the potential to pay $13000 per day.
Posted 4 hours ago with 958 points and it's already sitting at 68th place? Was the story too inconvenient for Y Combinator given that Netlify has bought 2? of YC backed companies?
I hate that a lot of times the only solution is
to post on HN/reddit/Twitter etc publicly. Just imagine all the other people who don’t use social
media and ended up paying…
There's good discussion on this here, and on Reddit.
In addition to all of that, the real moral of the story is to not use anything with dynamic billing for personal projects, even if there is a free tier. Always ensure that the free tier does not automatically turn into a paid tier.
That said, I've never used Netlify, so I don't know how they present this service at the time of signing up.
GitHub Pages is great for static sites, and included Jekyll build support is a bonus.
I am running my static sites on a VPS for only 1€ per month. It includes unlimited traffic, DDOS protection, and IDS/IPS at the service provider level.
It’s probably worth wrapping the entity that contracts with these type of services in a separate anonymous LLC structure so a DDoS attack doesn’t bankrupt you.
This is going to be a dumb question. I am new to coding ( 3 months in ). I am building a simple static personal website with GitHub pages. I am worried since I had to input billing data to GitHub for my global campus application since I am a student.
Is this something I need to worry about? Does cloudflare provide a service that is cheap that can prevent something like this for my GitHub pages site?
Getting DDoSed is rare but highly destructive when it does happen if you happen to use something like Netlify. GitHub Pages is under fair use and only for personal projects (no commercial uses). For commercial use, something like Cloudflare Pages would be better. Cloudflare can also help with their whole suite of tools with mitigating attacks and they will call you if you do exceed the free tier according to this comment: https://news.ycombinator.com/item?id=39520894.
People deploying low-traffic side projects - to VPSes or similar that allows price capping - and who do use a database: what deployment stack do you use these days?
I assume Docker + something is most popular, but what something? Does terraform work sanely for cheap virtual hosting? Ansible? I don't want to manually install any more stuff than the minimum I can get away with!
For my own projects I do Terraform/Pulumi + Ansible. I use Hetzner & DigitalOcean and this setup works great with both.
I don't use docker for my projects, as I deploy on RHEL like systems which I'm intimately familiar how to configure (and have snippets I mix and match).
You use both terraform and ansible in the same project? I always thought of them as competitors filling more or less the same role, do you find it useful to use them together? Is it that hetzner and digitalocean TF providers do a good job but provide limited functionality, and ansible fills in the gaps for you?
Is it just that no one would pay for it? I'm well aware of how terrible a customer developers make, but has to be nearly a non-issue with Hetzner in the USA now, with how much free traffic they give you (or any other provider, DO, etc). There's even Cloudflare R2 nowadays.
Your blog probably doesn't actually need sub 100ms serve times.
> This feels more like a scam to me. Why do serverless platforms like Netlify and Vercel not have ddos protection, or at least a spend limit?
Normally it should just stop serving the bandwidth once the limit hits, just like when you max out your credit card, it just gets declined, they don’t charge you forever just because someone else is abusing it..
Typically, it is not even a Distributed Denial of Service (DDOS) attack; instead, it is traffic from (SEO) bots that crawl through every possible combination of the page, such as query parameters for web shop filters. I hate those down to the guts, because you have to design a rate limiter that will limit them and let other traffic through.
> Don’t pay the bills, post it on HN! They have good reach.
This line scares me so much, the fact that we have to rely on an external forum to sort things out. What if my HN post doesn’t gain any attention? Is that it for me?
I understand that someone would pull all their strings in hope for solution but still, companies should be available directly.
When this occurs does the company count that as revenue and record the discount or do you adjust the accounting to reconcile?
I've seen in the thread that bobfunk (netlify CEO) and raiyu (cofounder at digital ocean) say the bill forgiveness is the right strategy. It got me curious on the above question though.
If we take 60TB / 24 hours we get roughly 5.5Gbits/second of traffic.
You can get a 10gbps connection, per month, for under $4K a month to a datacenter. For full usage for one month, not one day. If buying at the scale of a much larger bandwidth user, the price is much less.
If architected properly having many people request the same file should make things work better and make it cheaper.
The internet and www are obviously marvelous technologies with many people to praise, webtorrent is nice but in a way a hack replicating things we already had.
I'm wondering whether this story has been manually downranked.
Posted only 3 hours ago and 800+ votes but it's suddenly dropped from top 3 (on page 1) to 38th (on page 2).
Is it worth noting that Netlify bought two Y Combinator startups or is that a crazy conspiracy theory?
> On May 19, 2021, Netlify announced the acquisition of FeaturePeek, a Y Combinator and Matrix Partners backed startup that enables developer teams to preview frontend content.
> On November 17, 2021, Netlify acquired Y Combinator and SignalFire-backed OneGraph to allow for the composition of apps with APIs and services using GraphQL.
If this happened to me, joke would be on them, I'm grandfathered in with no credit card details attached to my file. Still, I had a mild panic attack reading this and am now thinking I shouldn't use them anymore.
I'm curious what would've happened had the bill been for 10k or 1k? Would it still have been reduced, would OP have paid instead of posting on reddit/hn, would it have gotten as much attention?
Same spiel as all the big clouds...not allowed to protect yourself. Unlimited risk all on you and afterwards you may beg for forgiveness from support and when that fails go to the other support page aka hn.
How come a post with so many votes and comments, submitted just 4 hours ago, is already on the 3rd page? Would have missed an interesting discussion haven't I seen a link to it on Reddit
I love it that the reddit post asks him straight away to make a post on HN. Its like everyone knows HN is the default customer support for these 10x web 2.0 hyperscale companies...
Just use Hetzner and never worry about getting into debt and having to beg some company's customer service to be so kind and reduce your imaginary "debt" with them.
That makes sense, but is the Netlify subdomain visible from your custom domain? How would they be able to figure it out, other than humans leaking it somehow?
I can't read it because I block reddit, but I assume it's a DDOS? With a bill that large, it would actually make business sense for them to team up with DDOSers.
(I work at Vercel) We do have DDoS protection, and spend limits. We're imminently launching improvements to both of these as well Importantly, OP was on a free tier on Netlify which does allow you to pay for additional usage on demand. Vercel's free tier does not – if you are on a hobby plan and you go over the included usage, your site will be paused automatically.
Also, DDoS or not DDoS it's very reasonable to believe that an individual (or even a company) isn't ok for a 100K USD bill.
Billing amounts should go through quotas requests, so you can explicitly ask to be migrated to the upper level, but by default have an active safety net.
I feel like unless a customer very explicitly opts in saying "I NEVER want my site to go down" then the host should just shut the site down if traffic spikes to high levels.
>Since Netlify charges 55$/100GB for the exceeding bandwidth
Absolutely absurd fees, there is no basis in reality for that. Sounds like a very scummy company.
I hope you don't live in the US buddy... If you don't, you're probably fine. If you do and Netlify decides to go after you, it's probably gonna be tough and costly.
Which is crucial Netlify is called out for this. Hiding behind the "fine print" is pathetic, not a way to do business.
I mean I get it but you get notifications and emails every time you trigger extra traffic. Don't get me wrong it should be automatic or there should be a safety net but this is not only on netlify.
I'm assuming they have DDoS protection by default, being a Cloudflare product? Or is the reasoning simply that they allow you to set a bandwidth/usage cap?
Yes, that was my understanding. Also unlimited bandwidth even on the free tier. So you get protection from DDoS and asshole business practices ;) Although there might be other caveats
Netlify CEO here.
Our support team has reached out to the user from the thread to let them know they're not getting charged for this.
It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
Apologies that this didn't come through in the initial support reply.
One additional feedback, for consideration: to me, your Pricing page[1] doesn’t make it sufficiently clear that the “Starter” plan may incur costs at all (let alone in this ballpark). It’s now more apparent when looking at it in hindsight, but you have to either read very carefully, or go to the separate “View Features” page to understand this.
“0$ to get started, then pay as you go” reads to me: “0$ to get started, and then you can order add-ons and extra features as you need them”, not “$0 to get started, but we may start charging you virtually unlimited amounts at any point without prior notice”.
When signing up for the “Starter” tier initially, I completely misunderstood this. I didn’t have to enter any credit card or invoice details, so I thought as long as you don’t have that info from me, you can’t and won’t bill anything.
[1]: https://www.netlify.com/pricing/
How on earth could I, as a customer, be sure that netlify hadn't paid someone to DDOS me? If I were in charge of a business like that, I would have that thought constantly...
5 replies →
> 0$ to get started, then pay as you go” reads to me: “0$ to get started, and then you can order add-ons and extra features as you need them
I think I disagree with this, but maybe I'm misunderstanding you.
Pay as you go sounds strongly to me that you pay based on your actual usage, not that it's free except for add-ons. A pay as you go phone, for example, does not imply you need to buy a telephony add-on, an SMS add-on, etc.
PAYG phones, however, were always prepaid, so I think I would expect PAYG hosting to be similar. That said, if my site was publicly accessible without my prepayment, I think it would be clear that it works the way it apparently does.
It's potentially misleading, but I don't think it's intentionally dishonest.
4 replies →
That was my understanding as well, since I signed up for MetLife years ago up until this very moment.
[dead]
There are only two questions everyone have:
1. Would Netlify forgive the bill if this didn't go viral?
2. How do you plan to address this issue so that it never happens again?
Everyone here knew someone from Netlify would come and say OP wouldn't have to pay. That was a given. Now we want to know the important answers.
1. Yes. We've forgiven lots and lots of bills over the last 9 years and they haven't gone viral
2. While I've always favored erring towards keeping people's sites up we are currently working on changing the default behavior to never let free sites incur overages
112 replies →
Thank God for social media that the user was able to get attention about this on Reddit which he was then advised there to post this on HN. It must have been stressful to see a six-figure bill and then get told that that, no worries, you’d ‘only’ be charged $5k instead for a static site. It’s just ridiculous to me to be sent a 6-figure bill in the first place.
5 replies →
From the 5% reduction it seems (1) was less likely.
To bobfunk, the response needs more empathy and explanation around the obvious frustration around why there is no slider for cost limitation.
As it is, it feels like the minimum viable corpspeak apology and damage control.
You don't see VPS providers like Vultr forgiving bills like this, nor do they make the news. Granted they are not the same scope as Netlify, but still.
OP said they agreed to reduce the payment, which means they acknowledged it was an attack but still wanted payment
if only i had $1 for every time for every time someone asked this exact question on HN. yes, we all get it: easy question is askable and not answerable. you want a gold star?
I’ve been a netlify user since 2017 and I just deleted all my sites. I can’t risk receiving a $100k bill for toy projects. Your “current policy” is not good enough.
Same, as it stands you the user are legally liable for the full bill unless netlify graciously forgive it. Even in op's case, they didn't (still charging 5k!).
If there was an option to cap billing, or at least some legally binding limit on liability, then I can countenance using netlify.
Until then, it's just not feasible nor worth the risk.
Same boat here.
the fact that once it arrives to the limits does not display an error page.
At this point I honestly do not care about they changing their policy, they should have thought that a normal person receiving a 100000$ bill on a free tier shall not been at all on the table in any circumstance, even if they forgive the bill, nobody needs to stress out like that.
Same. I will (almost certainly) never incur a $104k bill, but switching to Cloudfare Pages looks free and I don't want to depend on unwritten policies of goodwill to mitigate the potential risk.
Same here. Will I ever get a level of traffic that would cause this problem? Extremely doubtful. Is it worth the risk when Cloudflare Pages is a similarly easy offering, and took 5 minutes to switch to? Hell no.
Starting to wonder if this whole thing was an elaborate ploy by Netlify to cull the herd of longstanding, non-paying accounts.
Same. Toy project and it’s not worth the risk of using netlify. What’s a good, simple alternative for a VueJS app?
6 replies →
I agree and also delete my account.
The only "fix" here is to act like Hetzner and null route upon DDoS, price cap the thing, or offer unlimited bandwidth on the free tier like e.g. Cloudflare Pages.
Uncapped but paid is a recipe for disaster and you'll always be subject to the will of the support staff when something happens. If they can grasp to a straw leading to suspicions that it's not in fact a DDoS attack, you can for example be sure they'll do just that. Just no.
3 replies →
Couldn't have worded it any better.
I did the same last night from my phone. My personal site and a project docs site are just going to not be online for a couple days. Easy choice.
Same. I'm looking at alternatives to get off netlify ASAP.
6 replies →
Did exactly the same, moving everything over to Cloudflare took me less than 15 minutes. “We’ll forgive those cases, pinky swear” is not a valid excuse when putting (even opt-in) hard limits in place is technically viable.
"Current policy?" So, you will retain a right to change such fees when you feel like it.
This is a serious matter. We are building a new site for our company with Netlify, but we can't open ourselves to this predatory practice. And even if you do not mean to be predatory, even the option of such is enough.
If not resolved with a clean, legally binding promise, our company (and probably quite a few others) must move our business to Cloudflare, Amazon, or some other competitor of yours.
Presumably your company’s site won’t be on their limited free tier.
1 reply →
> "Current policy?" So, you will retain a right to change such fees when you feel like it.
Is that unreasonable?
Why are you asking this question here? Any actual company would have reviewed all the legal documents prior to choosing a provider. The promises you seek are the exact reason "enterprise-grade" providers can (and do) charge so much.
edit: hey guess what, Netlify offers an enterprise plan, I'd bet they will be happy to offer you a "clean, legally binding promise": https://www.netlify.com/pricing/?category=enterprise
1 reply →
« Apologies that this didn't come through in the initial support reply. »
"Didn't come through" doesn't actually match the user's report of having support explicitly offering 20% and then 5% payment. It sounds like maybe you have a training problem? That seems like one of the important points to speak to.
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
That doesn't square with the 5% fee on the original $104k that your company told the OP to then pay.
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
Well, giving the option to users to plan ahead would be best, no? Like a setting to choose whether they want a potentially unlimited bill versus downtime. Instead of that, you are choosing to stress and make people scared/anxious/homeless even (if they don't think of raising the issue on HN).
Seriously, this is not rocket science. This must have been discussed before in your company, and someone actually made this decision to stress people about such bills.
Frankly the only reason I can even come up with that Netlify wouldn't have such controls in place is exactly if they do _not_ simply forgive these sorts of jumps in costs (as the CEO here seems to be claiming). I'm pretty sure if they'd be left holding the bag, they'd manage to find some way to cut off these kinds of jumps in usage.
7 replies →
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
The legitimate mistake sounds to be on _your_ side if anything. You failed to match the attack pattern after all.
> Apologies that this didn't come through in the initial support reply.
The support email said you normally discount the attacks to 20%, but in this case it would be discounted to 5%. Are you here publicly claiming that your policy is to in fact to forgive (i.e. discount 100%) these bills? Was the support reply totally incorrect in claiming that you normally discount the attacks to 20% or are you lying when saying that your policy is to forgive the bills? You might want to clarify your position here.
How does a 60 TB in a day peak for a site that previously never crossed the free tier threshhold not qualify as "attack pattern"?
This is a static site. To reach that sort of bandwidth out of nowhere you'd need to publish the blueprint for a teleportation machine
To be fair, these days, things can become viral literally overnight.
That said, instead of depending on unreliable heuristics, they should just allow an option to change the behavior. The "current policy" to charge small sites on the free tier thousands of dollars instead of just throttling/shutting down the traffic is really predatory.
6 replies →
I understand that you need to pay bills, but auto-billing over the bandwidth budget just isn't OK, or at least not unless the user specifically configures that that's OK. I for sure didn't understand your bandwidth tiers that way.
You can avoid this sort of bad press and disgruntled users and your support cost by just giving users the option to shut down the site once the bandwidth budget is up.
That customers must seek forgiveness at Netlify's discretion is not comforting. What's comforting is dependable spending controls.
Lol this deescalated pretty quickly, went from $104K to $20K to $5K to $0 Which basically means you almost scammed the customer for $5K or $20K. Super negative practices. I for one could never trust a company operating in that manner. It would be much more honest to say "unlimited bandwidth" and set a hard-limit for maximum budget, then people know they won't be charged, than to go through all this crap and then pretend you're doing a favor to the customer (you're not). If I'm normally spending $10/month any idiot out there would know for sure that I'm not going to spend $104K instantly. That's a very basic filter to have. But you don't place such filters because obviously you're working on the principle to scam people many thousands of $ if they fall for that. Heck, for all we know you might send that amount of traffic to your customer and the try to scam them and if it doesn't work then pretend you're doing them a favor.
The fact that the CEO had to step in after this blew up online otherwise they were going to try to extort that poor dude for thousands of dollars!
Moving my sites off of netlify ASAP.
14 replies →
Heck, at that point, why not "send some traffic" to your customer? It's not like they have any way of verifying its source. Hmm... why even send traffic at all? Just add a multiplier to their metrics!
This is very weird take. I'm struggling to understand why this is incident as a reflection of "super negative practices" or is somehow a "scam". The CEO came here and publicly apologized for the mistake and mis-communication, and the issue is resolved for the user with no charges. What am I missing?
11 replies →
You can't rely on such a policy if it is not part of the actual contract. This doesn't address the enormous uncertainty and risk that is present here when using Netlify.
This is what sticks out to me about the situation. I would much rather a site go offline due to service overage triggering at some limit that I set - simply relying on the good faith of a host to subjectively waive fees is not reliable nor does it instill confidence that I won't be financially ruined by malicious third parties (like nearly happened here). I would imagine that the good faith of Netlify in this case would mean very little to a court when there is a contract that stipulates costs for services, and the worst case scenario for a user is that Netlify could take the issue to court with the contract the user agreed to and demand full payment. Even the possibility for this situation to occur without any tools existing to prevent it is terrifying and is a terrible value proposition for a service.
So what's the policy?
Do you forgive 100%, 95%, or 80% of the bill?
Is the 100% only available when a story about a bill goes viral?
By the time you forgive the bill you may have caused significant psychological distress, maybe even irreparable. This doesn't feel like a responsible approach.
This is the way most companies work unfortunately. Paypal limits your account and makes you wait 6 month to (maybe) give you a way to get the money back.
1 reply →
I’ve already migrated my two sites off Netlify after reading about this incident, and seeing other replies where folks said they were stuck with large bills.
This large bill doesn’t look like a legitimate mistake, it looks like everything worked as intended until things got escalated via Hacker News.
This leaves all your other small business users potentially on the hook and at the mercy of your mercy.
Not only should this stuff be capped rather than the dam allowed to flow, but your systems should have picked this up immediately and known it for its nature.
Thus must have been a nice little earner for you over the years.
I'm moving all my netlify sites elsewhere, bob.
I'm probably not the only one.
Good on you for reaching out, but getting a bill like this in the first place is enough to send someone to the psych ward, lol.
Email:
> "You've got room to grow!"
ohfuckohfuckohfuck
Can you respond to the allegations that Netlify has inadequate spending limit controls? Are there plans to improve this situation?
> traffic spikes that doesn't match attack patterns
I interpret this as "we always charge for traffic served, but we attempt to block illegitimate traffic" which means of course that the worse their traffic discriminator performs, the more money they make!
Hello bobfunk, thank you for acting on this.
One question though, what is Netlify gonna do to ensure this doesn't happen again?
I understand it's a hairy question, but the general consensus seems to be some policy must be changed or at least some line should be drawn.
I assume you'll be offering this user a good amount of credit on their account for having to deal with this BS and the stress of being told they owe you $100k?
[flagged]
Made an account here to also let you know, I too am removing my websites from netlify ASAP. Thank you for bringing this to light.
How long has this been the "current" policy? 2 hours?
ex employee here, left 4 years ago. was policy back then too.
6 replies →
So this one got attention due to some good Samaritan on Reddit who told OP to post here. Now, to the real question here: have others not received as good advice and just paid up?
> instead forgiving any bills from legitimate mistakes after the fact
That's terrible for marketing.
I'm moving my domain name and personal site off Netlify (already deleted the sites, DNS transfer requested), probably moving to Cloudflare pages.
It may only move a few MB a month, but I just can't risk if I put anything more substantial there that I might get hit with a bill for $100k and you maybe will forgive it. And that this has apparently been policy for nearly a decade makes it even worse.
I'm so grateful that Cloudflare has Pages, and I was able to move my hosting needs there. Netlify has been expensive for a while now.
Sorry, but there is a lot more going on here than you addressed, these charges were incurred on your "starter tier", which has no mention of additional costs.. I've noticed a lot of "sponsored content" by netlify, and again no mention of this possibility.. Also, no comment on not having ddos protection, or at least a spend limit?
Sure, this instance was resolved, but it's also the top post of the last month. Who honestly things it would be the same outcome if not for going viral...
You guys see a lot of traffic. Why not offer DDoS protection for the free tier by default?
I’ve been a Netlify users on the Pro plan for a few years now. Moving from Netlify to CloudFlare after this; “this didn’t come through in the initial support reply” doesn’t cut it for a $100k bill.
But you do see how _not_ addressing this in the initial support reply is going to cost you all in the long term, right? The real lesson here seems to be for small projects, it may well be worth the investment to handle my own hosting. All I see here is that getting you to do the right thing required publicly shaming you, which means you can be trusted about as far as I can throw a piano.
How about a button that says "put down my website if it suddenly starts getting charged"?.
Never used "netlify", but to me a product is broken if you are using the words free and bill together.
I wont touch a fake free service if it requires a payment method. Want my money, give me a reason to pay you, dont trick me into paying you.
Temped to go fuzz your product and document other dark patterns...
So netlify is a major scammer organization now!? Uh oh time to look elsewhere
I’d rather be shut down than have a heart attack from a $100k bill. That could literally kill me from stress, even if you pinky swear to refund any oopsies.
See the Robinhood user who committed suicide after misunderstanding his liabilities from selling options.
2 replies →
You should rethink this policy. Someone could panic and do something unthinkable, then you'd wish bad press was the only thing on your conscious.
Is the support employee going to be fired for making such a traumatizing mistake? Or was 5% ok until this went viral?
That is an outrageous and inhumane policy. People get panic attacks when they get told they owe 100k they don’t have. People will be terrified your internal process wrongly determines the bill is legitimate. Imagine you have to study for an important exam or that you have a paper due. How can you possibly focus with this nightmare at your doorstep?
Truly shameful.
This is predatory and you know it.
Your support was going to charge him 5% as a "sign of good fate". How kind.
If it hadn't gotten traction, you absolutely would have charged him.
How many other people have you strong armed into paying ridiculous bills?
The fact that you have no usage limits is clear indication that this is intentionally left open to abuse.
Extremely shady and downright criminal.
The most bizarre thing is that this is a known issue that folks have asked them for ways to mitigate, to no avail. The reddit thread even links to an extremely weird dialogue where Netlify's response boils down to, "if you're hosting a small site that gets DDoS'd, don't."
https://answers.netlify.com/t/limit-bandwidth-to-avoid-high-...
https://www.netlify.com/security/ sez “Active DDoS mitigation — Netlify monitors for traffic pattern anomalies and spikes, and effectively controls for them as needed” and now I'm curious about what that actually means.
It means they protect themselves from layer 3 and 4 DDoS. For layer 7 you're mostly on your own. That's what most companies mean when they talk about DDoS anyway.
3 replies →
It means that they will charge you 20k (a year's rent for me, no biggie) instead of 100k for your free website, or 5k if you got lucky.
6 replies →
They reroute the network traffic to ensure none of it gets dropped so they can accurately overcharge you for the the correct amount.
I'm hesitant to use "fancy" cloud service/hosting providers for reasons like this.
I don't understand why they won't just raise a 503 if the traffic exceeds the spend limit, or at the very least provide that as an option.
Playing ”devil’s” advocate: tracking spend in real-time is not trivial. It adds complexity to stack. Bugs in the feature can cause sites to go down (for long time) without a reason. Larger online businesses likely rather sort out the problems later than risk shutting down in the middle of unexpected success.
(But I also would like to see this feature)
8 replies →
What's spend limit?
Autoscaling is a feature!
19 replies →
Yep, for a static site you can throw nginx on some VPS for $10 a year and it'll handle a decent amount of traffic.
in other words, "if you're thinking of using netlify, don't".
true. I have a 9€/mo vps at Contabo for my blog and once boasted on HN that my small VPS is able to handle reddit/hn hugs which one user seemed to take personally and they started a DDOS against my VPS.
I only realized this after Contabo contacted me and said the traffic is so high that other clients service is also degraded and they will have to take my VPS down if its much longer (which was understandable). Gladly the ddos stopped soon.
But never was there any talk about any cost, they were very supportive
1 reply →
To some extent, that answer is fair enough, assuming they make this clear up front. If their service is "we'll keep your site up no matter what, for a price" that's a fine service to offer. It's not what the vast majority of people want, of course.
If their advertising is targeted to small businesses and individuals who could never afford this type of service, they could be guilty of false advertising, at least morally guilty. I haven't seen their marketing so I wouldn't want to say.
Their marketing is very much like this. It’s completely misleading. They are definitively not selling “keep it up at all costs, money no object”
This rings so true!
I've dealt with Netlify's support [1], and one of their CS heads was incredibly rude to me and blamed me for the problem they created.
[1] https://news.ycombinator.com/item?id=35610956
“Stop dressing so sexy if you don’t like the attention” is the vibe I got.
I don't fully understand Netlify, but it seems though it tries to be a one-stop solution for everything it doesn't have to be - you could put free Cloudflare in front of it and probably mitigate this kind of thing?
https://docs.netlify.com/domains-https/custom-domains/config...
> And since my amount is too large, they offer to discount to 5%, which means I still need to pay 5 thousand dollars.
If they just reduce to 5% like that, it shows how disconnected this is from their real bandwidth cost. Really does feel like a scam.
Especially since they admit it was a DDoS attack. What I find outrageous is first that they charge for incomming traffic (which is often free with other providers), but also 55$ per 100GB. For comparisson, Hetzner charges you 1€ per 1TB of outgoing traffic while incoming is free.
If the attacker downloaded the 3,44mb audio file that OP mentioned, aren't we talking about outgoing traffic?
1 reply →
They overcharge egress by about 500x https://getdeploying.com/reference/data-egress.
So even a reduction to 0.2% would habe been possible. Honestly don't understand why anyone feels comfortable overpaying so much. Especially when there is no configurable spending limit.
It wouldn’t be possible for them. Netlify doesn’t own transit, so AWS needs to get their fat cut even if Netlify waives their fatter cut.
no, they charge 6x according to that table. Because they are using aws
1 reply →
Eh, I wouldn’t say that’s necessarily the case. AWS support, for example, tends to be really good about waiving charges for things that are clearly your mistake, like an unused instance that you forgot to turn off for a couple months. That’s not because hosting instances doesn’t actually cost Amazon anything! It’s because they want to keep you as a customer even if it loses them a bit of money right now.
In the Netlify case, though, insisting that this person still pay 5% is downright insulting. I’m sure they’re taking a hit already - just waive the whole thing.
AWS support, for example, tends to be really good about waiving charges for things that are clearly your mistake, like an unused instance that you forgot to turn off for a couple months.
This is an admission that their UX sucks and makes it hard to know what state your account is in and what you're paying for. They waive the fees because a few high profile cases of people paying thousands due to the AWS console being awful would drive a lot of customers away.
I would say it is a scam, because you can't set a budget limit.
1 reply →
Our AWS TAM says they don’t do this anymore, and we spend tens of millions of dollars with them annually. n=1, ymmv
7 replies →
That’s not because hosting instances doesn’t actually cost Amazon anything
Except it doesn't cost them anything. The marginal cost of keeping your single instance running is $0 (unless they were 100% out of capacity and they could have sold that instance to someone else either at full price or spot price)
16 replies →
Some quick back of napkin math says 190TB would cost about $12k in AWS cloudfront costs.
FYI: From my experience, if you do more the 20-25TB, you can get 75% off no questions asked.
2 replies →
>it shows how disconnected this is from their real bandwidth cost
It's a value added service, they don't trade bandwidth as a commodity. Therefore unfair characterisation.
Plus, if you dive deeper: Bandwidth doesn't cost anything because bandwidth is just about pulsing some light in some glass fiber and applying some minuscule voltage on some metal fiber.Okay, maybe it costs some amount of electricity but all this is just a business model for paying on capital expenditure through time share arrangements. People can have all kind of models for this, for example you can come together with others or pay it all by yourself to install the equipment and have free bandwidth for the lifetime of the equipment.
It's all just arrangements to cover the capital investment and earn something on top of it. That's not a scam. A scam would be if they didn't account correctly for the timeshare usage or induce usage to boost payments.
> they don't trade bandwidth as a commodity
I really don't get your point. If you're a hosting provider, the very thing you're selling is bandwidth (and disk space). Everything else is a value added service.
1 reply →
This is true for all businesses but maybe more so for tech:
Don't have a business model that charges customers for your mistakes.
This customers bandwidth usage jumped from free tier to $100k in very short time. To be honest, this shouldn't even be possible. Any "free" tier that allows for a surprise $100k bill is not a free tier.
This bandwidth usage is the result of a mistake on Netlify's part. That much seems clear.
To go and suggest that the customer is responsible for any portion of the bill is where things really went sour imo. Don't do this. Ever. Unless you want your company to go viral for all the wrong reasons.
If you want another good example of how badly this can backfire, look at what happened when Unity announced their new pricing scheme. Unity's new pricing scheme also allowed for unbounded bills. At first they didn't even deny this. Later they said it was a customer misunderstanding. I.e., they blamed the customer for their mistake.
Thankfully, the CEO of Unity was fired.
The lessons are very straightforward:
1) Don't implement predatory pricing schemes (this can even be done unintentionally, but the intent doesn't matter).
2) If you do implement predatory pricing, the worst thing you can do is put on your surprised pikachu face when the customer asks why their bill is bigger than their annual income.
Since the author has gone viral I expect some netlify exec is going to take over and write this bill off to $0. In the words of Kramer “these big companies, they just write it off!”
A moment of silence for the people who got DDoS-ed, didn’t go viral and still had to pay $5k.
The very fact that you expect that making front page of hn will make them cancel that bill means that it will soon be over.
These kind of stories (alongside cancelled accounts) repeat over and over again and will soon become so not newsworthy that they will either not end up in front page, nor people will check on the eventual outcome which means these companies will get away with not moving a finger.
Nah. I think the dev community has long memory. Events like this is damaging for years. Whenever Netlify is mentioned, someone will inevitably point to this thread for a few years.
1 reply →
Thanks.
When I received a message from the bank saying my account was in the red I discovered that AWS had been billing me 1100 / month for 5 months before I even noticed. It was for something I'd set up one night while bored and then forgot about it. They drained my account :( Even had the nerve to say I had to pay for premium support only to get a "lol, pay" response.
You don't even know what a write-off is.
For those who don't get the reference: https://twitter.com/JayBauman1/status/1665024103282012161/vi...
1 reply →
But they do. And they’re the ones writing it off.
Neither did Kramer :D
Do you?
If you bend over and pay in such circumstances, you are part of the problem. Twilio tried to pull this crap on me and I simply created another account with an email forwarder and left them holding the bag on the previous account
didnt even take 2 hours till your predication came true, you are the messiah
Here is my shiny new super business plan for a startup that will profit thousands from a supposedly non-paying customer:
1. Offer “free static website” with lots of templates and guides to help you build one
2. The first 100GB is free and beyond that it’s $0.01/MB. But no worries! Very few customers actually use up that free bandwidth and in case you need more you can purchase packages for $100/TB. Also we offer a free service that will help you get your site more visible by advertising it, it’s included by default.
3. After a month or so, randomly help a customer bump the website and make it popular by putting it in some list that is frequently crawled. Secretly hire someone else to crawl these websites and make lots of download requests
4. Once the customer suddenly gets 10TB of traffic, bill them for 9900GB which is $99000
5. As long as 1 out of 100 customers pay, you are profiting $990 per customer! For the rest of customers, offer a 5% discount so they only have to pay $1980. Threat taking them to collections if they refuse.
Become the next millionare by just selling free static websites to 1000 customers! Anyone join us?
Are you taking angel investors? Throw in that it comes with an AI chat assistant or a special vectordb so you can raise money and I’m in
There's no way in hell anyone should ever under any circumstance use a free service that might, for reasons entirely outside your control, suddenly bill you 5k, or 104k... or any non trivial amount really.
Just suspend service on excessive overages...
Yeah. Elsewhere in these comments there's a link to a support thread[0] where Netlify support essentially says you shouldn't ever need an option to suspend your site at a certain point because:
#1. If you think there's any chance of getting DDoSd, you should already be on a business plan instead of a starter tier.
#2. If you think there's any chance of your site going viral, you're going to want to pay the cost anyway to let all those people visit.
I agree that's ridiculous and that the lack of any option of capping costs would mean I'd never sign up for the service. But that's the official response, for what its worth.
[0] https://answers.netlify.com/t/limit-bandwidth-to-avoid-high-...
So anyone whos site might be posted to HN should be on a business plan.
1 reply →
Like in the past, when you went over your limit your page went offline. The good old Slashdot effect
Yeah Like wtf is wrong with that? Are people just to lazy to check what the conditions are when exceeding traffic? I'd never ever sign up for anything that just keeps charging...!?
13 replies →
I understand that some businesses might want to take the hit from a cost surge because they get an even higher revenue surge. But a large fraction of sites aren't like that and would prefer a loss of service to a cost overrun. Service providers should always offer a "maximum out-of-pocket cost" service option. Those that don't aren't suitable vendors for most customers and customers should be warned about them.
I believe that's what Firebase Hosting does.
As I recall, you have to actively sign up for the paid plan (Blaze) to get pay-as-you-go billing. Otherwise, you get free quota, and if it's up, it's up.
I think it also integrates into all of Google Cloud's billing management stuff, but I've never had to bother with that.
> use a free service that might, for reasons entirely outside your control, suddenly bill you 5k, or 104k... or any non trivial amount really.
Like all of the big clouds with free tiers and nuke it from orbit level footguns lying about everywhere?
Use a virtual card with just a small amount of money on it to limit your liability. Won't work if you've entered a contract, but for a lot of these providers, including AWS it works.
This "one crazy trick" does nothing to limit your true liability.
If you go to a restaurant someone at your table orders 5,000 plates of mozzarella sticks, the fact that your credit card only covers $5 doesn't mean you are magically absolved from the rest of the bill.
For $100k, a debt collection firm would be more than happy to get a judgement against you. Credit card or no.
Totally agree, it's an unacceptable risk
I once got a $65,000 water bill from the city for one month. I laughed and called them and asked them to re-read the meter and correct it, and expected a quick resolution. But no, they insisted it was correct for some time and that I needed to pay it. They said I probably had a leaky faucet or running toilet.
There was no awareness on the part of the customer service people how ridiculous that was. It would be physically impossible for my service pipe to deliver that volume of water even if it had been running full open for the entire month. I kept escalating until I reached someone who agreed, and they sent someone out to re-read the meter. And my bill was reduced to about $35.00, the normal amount.
Front line customer support isn't always very in tune with what is sensible for a given customer's account.
To be fair to front line support -- a lot of times it's just a warm body reading a script and it's paid accordingly.
Not always - but a lot of the times, especially for lower quality companies.
Water is a regulated utility. Anyone in a similar situation can contact the government authority who will gleefully tell the company to go to hell and possibly implement fines for inappropriate billing.
I wish that was so straight forward. You can google this incident where an empty lot got a 35K water bill and the water company said it was an error, then backtracked on that and still saying 35K is due.
> Towards the end of 2023, the DWM seemingly corrected the issue. Revive received an email stating: “The prior balance on the account reflected water leakage that was the result of Department of Watershed actions. Once the leak was addressed and the account properly adjusted, the corrected balance for the property is $219.24.” However, DWM soon backtracked and claimed that the $219.24 quote was made in error and that the nearly $30,000 balance still applied.
https://lawblog.legalmatch.com/2024/02/26/empty-atlanta-lot-...
1 reply →
I deleted all my Vercel project, moved to Cloudflare, send this support message to Vercel:
Vercel seems like the perfect solution, and I love how it supports the development community.
I am moving all my current and future hobby projects away from Vercel due to concerns raised in this discussion. https://news.ycombinator.com/item?id=39520776
Although I am a very small customer of Vercel, I have been advising larger organizations on IT and data infrastructure for the last decade or so.
I can say with very high certainty that spending limits are a critical discussion point in every large organization when making IT decisions. I've observed multiple instances where a potentially better solution was not selected due to the risk of overspending.
Vercel has had spend management for a while now: https://vercel.com/blog/introducing-spend-management-realtim...
Thanks @doque !!
Even though I've been actively looking for this, I couldn't find it.
However, this solution ONLY provides notifications; it doesn't address the underlying problem. I could be sick for five days and not check my phone.
What I need—and, in my experience, what all larger companies require—is a method to halt consumption entirely, similar to Snowflake."
5 replies →
This is my worst nightmare as a bootstrapped founder. And that there's no way to put a limit on spend is ridiculous. Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere.
Just went through Vercel's docs:
---
"Vercel helps to mitigate against L3 and L4 DDoS attacks at the platform level. Usage will be incurred for requests that are successfully served prior to us automatically mitigating the event. Mitigation usually takes place within one minute.
Usage will be incurred for requests that are not recognized as a DDoS event, such as bot and crawler traffic.
You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP, User-Agent header value, or other identifiers."
---
That doesn't help me sleep well.
I feel that by now, these hosting providers should simply adopt best ddos protection practices and take responsibility for failure to protect.
"You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP" - there should be some really good defaults for this right?
Clearly it's possible - Cloudflare's ddos protection is worded more strongly.
I'm willing to pay more for the service for peace of mind. Like, even $10/mo more to insure against getting smacked out of nowhere.
> Cloudflare's ddos protection
Yeah, we got hammered once with over 10TB/mo and noped out of Netlify as fast as we could: https://twitter.com/rethinkdns/status/1370342245841342466 Had to pay the bill in full.
Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.
CloudFlare pricing is indeed positively ridiculous.
At OpenTofu[0] we’re using CloudFlare R2 to host the providers and modules registry[1]. Bandwidth is free, you only pay for requests.
This already would be great, but there’s more - you only pay for requests that actually hit R2. So with an almost 100% cache hit ratio, we barely register any billable requests.
Recently someone decided to load test us and generated ~1TB of traffic over 1-3 days. All but a few of these requests were cached, so the whole situation probably cost us less than a cent.
[0]: https://opentofu.org
[1]: https://github.com/opentofu/registry
3 replies →
> Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.
Yeah that's how Cloudflare can reach total control over the Internet. With thunderous applause by people that should know better.
I know that my position is outright blasphemous in this day and age, where even self-hosting a static site has become black magic and we need a third party to do it for us.
6 replies →
That’s a free tier that doesn’t sound sustainable then, so that raises alarm bells to me.
30 replies →
> Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0
It's not really ridiculous if you think about what you're giving them.
You are massively benefiting their platform by providing them data which they use to train their services and then sell those services to other customers.
I'd make a case that the data they collect is the most important part of their business and the free tier is a major component of this.
1 reply →
I don't think it's fair to call it their free tier - it's their discretionary tier, there are numerous cases of the rug being pulled as and when it suits their business requirements to do so. Being left homeless vs. urgently coughing up is exactly the wrong problem to be dealing with mid-attack, I can't see any way to consider it free by any practical definition
I know that putting all eggs on one basket and giving it all to Cloudflare is not a good idea, if they have an outtage then I would also have it to. But when they are down, one third of the internet is down with them too. With 240$ a year for CDN, 60$ a year for serverless and $0.015 / GB-month for S3-compatible storage with free egress, I don't think anyone could find a better alternative than CF. I'm mixing with AWS, CF and self-hosted machines and the infra cost is less than 5k$ a year. Now I can spend the remain hard earned money for some fresh marlboro cigarettes.
Use a token bucket on your web server to catch abusive IPs and then blackhole them using `iptables -t raw -I PREROUTING -s ip -j DROP`. I know. I run https://ipv4.games/ which invites hackers to unleash their botnets, and the service runs on a small VM with only a few cores. It's been attacked by botnets with 49,131,669 IP addresses. There's no Cloudflare frontend or anything like that, because back when I used Cloudflare, the people who attacked the service would actually bring down the Cloudflare nodes before they brought down my web server. I doubt I've ever paid more than $100/month to operate the service. Please note that your service provider needs to have free ingress in order for this strategy to be effective.
This strategy may work for a (D)DoS that is targeted to an application layer, but won't work if the attack is designed to exhaust your bandwidth.
Once you're receiving more traffic than you network cards can handle, it does not matter if you'll drop the packets with iptables or not.
I was the target of attacks that caused Hetzner to terminate my contract. I was leasing physical servers there, so I assume the attacks were overwhelming their infrastructure.
3 replies →
Eventually you're probably going to want an ipset, at least. Otherwise processing your chain will continuously cost more, and more, and more.
2 replies →
If you want to sleep tight just get a dedicated server or VPS from something like Hetzner and/or combine with CDN providers like BunnyCDN - set up alerts just in case though. It takes more time and resources to manage it but you could save a lot on it in this case.
This so much. My hetzner (best choice for a media server within Europe) has 0 downtime in 1.5 years. And exactly as you said I am using bunny as well, which costs me a few $ per year.
> It takes more time and resources to manage it
For most of the new web projects, setting up your brand new server is pretty well documented process and should not take more than couple of hours.
It get complicated when you grow and add more servers or components. But at that point, you should be able to afford a part-time consultant to handle complicated tasks or just use Cloud then.
I'd even say build your system so as it can run on shared hosting. This way you even save the management.
That is my setup after leaving AWS for some of my services (low user amount b2b).
I put in far less resources and maintenance after I had the system running. Especially if you need to manage the software running anyway.
This might be a good time to point out Cloudflare Pages: https://pages.cloudflare.com/
Under the free tier:
> Unlimited bandwidth
I'm moving everything to Cloudflare.
6 replies →
I didn't even know Cloudflare offered a JAMstack platform. I'm going to switch as I already use Cloudflare for domains.
Yeah, I'm already using Cloudflare because of Google Domains got de facto killed by Google via transferring it to Squarespace. Why not Cloudflare Pages, CDN, and R2 (S3-compatible storage) too? I'm even considering paying for the paid tier in the future if I ever go above the limits of 20 000 files per static site and the 25 MiB single file size limit [^1] (more than enough right now or in the near future).
[^1]: https://developers.cloudflare.com/pages/platform/limits/
I was looking for a static site hosting option recently and tried out cloudflare pages. Fit my need perfectly. The generous free tier and the reasonable pricing model were the big factors.
Oh, and the ability to put some authentication in front of it was a big feature for me.
Host on a provider which bills per hour. This caps your cost. It also makes your users pissed because you will go down, but if you’re small, you can afford that. If you’re big, you already have scaling options and should have a team to handle ddos.
My experience is that customers don't really care that much about small amounts of downtime no matter what size you are, people mostly get that unexpected stuff happens as long as you don't get hacked or misplace their data. Customers might complain a bit but seldom leave because of a few hours downtime.
This seems to mostly hold true to developers also, GitHub manages to survive just fine after all.
2 replies →
Yeah. Any host that won't infinitely scale out will solve this concern for you.
1 reply →
Vervel charges $400/TB for excess bandwidth, it's not even DDoS you should worry about, just moderate success.
That’s a crazy high bandwidth. Bandwidth isn’t free, but $400 will get you a month of 10gig in my local peering point, that’s 1TB in 15 minutes.
> Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere.
An interesting story that expands on the above concept but a different vector entitled, "Illegal Life Pro Tip: Want to ruin your competitors business?" : https://news.ycombinator.com/item?id=36566634
Imagine you lost your job. So you are here enjoying creating and hosting your hobby projects in theses services. Now, suddenly one fine morning you get slapped with $104K bill because someone decided to randomly ddos your one page dog lover website.
Now, who in the would would be thinking of having ddos protection for their hobby project? This is just absurd thinking.
No. This is absolutely common. I remember well how shared hosters 10 years ago already put caps on cheap packages and took the websites offline in case of traffic. And today it's Amazon who bills small players into dept.
There are many provider who don't tho.
1 reply →
Can't hosts just make a site unavailable once it reaches its plan's bandwidth limit, DDoS or not?
I think being offline is a lesser headache than a large bill, especially for those who are inclined to a free tier to begin with.
1 reply →
This may seem weird, but I believe ToS ae the real problem here. I call it the "car rental" problem.
When I rent a car in person, I am often given a contract. And this contract is filled with tiny print, and pages of it.
There are often people behind you, waiting, and bored/annoyed people behind the counter, waiting. This is beyond unreasonable.
A point of sale contract should be short, in readable text, and understandable. For example, renting a car? Under a page, easily parseable, and if the person behind the counter cannot explain it, it is null and void.
From a legal side, you can do this. And you can explain legal terms. Of course this means you are describing intent, which limits one in court, oh boo hoo Mr Lawyer. Cry me a river.
Well the same should be true of any retail contract. Sign up for a service? One page with costs listed.
At least then, there is hope of an end-user sort of understanding. And as one could claim that a DoS was actually targetting the provider, and not the website, that should be described too.
So back to the topic at hand. I would write a demand letter, insistong Netify explain the charges, and ask them if they and their IP ranges were DoS, and if so that the charges be reversed.
Because you shpuld not be paying, if someone attacks Netify.
This letter should also be sent by mail, sig required, to the corporate address too.
4 replies →
It shouldn't be like this, but it is.
Unfortunately, in today's world, DDoS protection is the equivalent of basic hygiene, foid and road safety. It's just a travesty that the hosting providers don't feel like it's their responsibility to address it.
I always run mine through Cloudflare, at least in part for this reason.
How do these hosting providers sleep?
In enormous mansions atop large piles of money.
I guess on eiderdown pillows.
Vercel seems to exist only to promote lock-in.
“We leave your safe deposit box unlocked. You might want to forge your own lock and key. If we happen to notice someone stealing out of your box, we will let them grab as much as they can for one minute, then maybe install our own lock if our revenue is close to target.”
Can anyone share an example of edge middleware that might protect you on Vercel?
You can also turn on soft and hard spend limits on Vercel (including SMS alerts) https://vercel.com/blog/introducing-spend-management-realtim...
1 reply →
Those services exist, and you have the option to use them. Netlify is not one. Apparently, you chose that the un-insured solution was best for you.
Wait until you learn that Vercel only supports blocking IP CIDR ranges on the Enterprise plan.
More reasons why I avoid clouds with outrageous bandwidth fees, and prefer Hetzner's low cost fixed pricing with Cloudflare R2's 0 egress fees.
Even if the DDoS wasn't caught by Cloudflare, the total cost for 192TB bandwidth on Hetzner would be €172. Although even after 10 years on Hetzner I've never paid for any bandwidth, always well within their generous 20TB free bandwidth.
First of all Hetzner would't let your server to be DDOSed for 192TB if it's not your normal usage. They'll likely just null route your IP if serious attack hit.
They also likely drop any charges if you escalate via support in case it was actually DDOS. E.g if you normally have 100GB / month and now you magically have 50TB / day.
What Netlify does is a scam.
I feel a class action lawsuit is incoming. Potentially with FTC support...
I really hate Clouflare and at the same time love them. Love them for their free, generous 500k req./day pages and workers, hate them for not having spending limits (or at least I can't find them). I get that they probably don't care about individuals and small businesses paying them peanuts, and corps can afford to pay extra if something blows up, but for me it just means I will only use a free account, and they get none of my money.
I also take advantage of their free services and only pay for R2 on Cloudflare since it's the best value managed provider I could find.
I prefer to keep my App's stateless and running in Docker containers which means storing all uploaded files and generated assets in R2 managed storage - which is also used for Litestream backups of our SQLite databases.
Blowouts are minimized when using low cost services, e.g. we had a rogue process that ended up causing 1.5M writes to R2, which only ended up costing us $4.50 in that month.
Hetzner will just null route your server if you’re DDoS’d.
Which is vastly preferable to a thousand dollar bill (not to mention a hundred thousand dollar bill) for most websites.
1 reply →
I recently rewrote a website/small backend API for a non-profit organization. I could've gone with a serverless architecture for our forms handling API and reduced spending to nearly the free tier, but I had no good way to protect against a scenario like this. There was just not good enough documentation about how to completely cut off spending in the scenario of an attack, and I wasn't comfortable leaving the organization open to a cost attack like this.
So we're using Github Pages for static hosting and a $5 box from OVH now. Unmetered bandwidth, plenty resources for our purposes. Cheap enough, and we will never, EVER, have to worry about an attack like this. Well worth it imo.
Imo, serverless is great for internal jobs where you can control spending. For public facing things, you have to be a lot more careful.
Yeah I'm doing the same, a combination of OVH, Scaleway and Hetzner.
Sure there is no such thing as "free unlimited" bandwidth but I much prefer unlimited with a fixed cost until they decide it's not worth it and shut me down vs unlimited risk with no ability to cap it.
The lack of cap is the worst part and it's 100% a business decision. Every provider who tracks bandwidth could add a cap but they just choose not to because it's too profitable and the risk is mostly* on the customer anyways.
*there is of course a tiny chance they the customer goes bankrupt and they get almost notning, but usually they just need to pretend to be nice and forgive all or most of it
I am confused. Why couldn’t GitHub Pages suffer a DDOS attack? Also they don’t want you using them for business purposes:
https://docs.github.com/en/pages/getting-started-with-github...
GP probably refer to getting a huge surprise bill after a ddos attack, not github never got any ddos attack.
Worth reading - thanks for sharing.
Arguably a non-profit (unless it was selling stuff from the site, which is unlikely) would be exempt from their list of prohibitions.
It can but it won't result in surprise bill because the bandwidth is capped.
I just checked pricing on hetzner (not affiliated with them). First 20 TB are included with VM price, after which it's 1.19 euros per TB. If this happened to someone hosting there it would cost them additional 50 euros. Can't believe the difference in traffic pricing. Netlify is over 1000x more expensive per TB.
People will throw up all sorts of excuses for not just renting a VM for a static site - scalability, ease of use, security.
But in the vast majority of cases you could just take a $5 VM, apt-get install nginx and be absolutely fine. A tiny bit more effort and you can make sure it's always up to date and very secure. Plus you get a VM you can use for other things when needed.
Just regular managed web hosting is enough for simple websites for people who don't actually want to manage a whole server. You don't have to manage anything with such a plan, just put your files where you're told and that's all. It's how simple websites have worked for decades.
Hetzner's plans start at 1,76€/month with a domain name included and unlimited traffic. OVH is slightly cheaper.
I'm probably too old (?) to understand the appeal of Netlify or other similar services, but I really don't understand why they get used.
By the way, Netlify says "100 GB bandwidth" is included with the free plan which I thought mirrored Hetzner's number of "30 TBit total bandwidth", but you have to click the details to see that it's 100 GB per month. So not bandwidth at all, but traffic.
Hetzner doesn't bill incoming or internal traffic [0], only outgoing, so in OP's case he would not have had this problem.
[0] https://docs.hetzner.com/cloud/billing/faq/#how-do-you-bill-...
He would, the attacker was requesting a 3MB .mp3 file, so that was all outbound traffic.
Problem is if you get 10x that traffic, which would already be 500€ in Hetzner, right? Of course its much cheaper and it's very unlikely that you'd get a huge bill but with them, but after all, it would be great if you could just say "Shut my machine down when my monthly spending gets to 100€"
Woof. I was considering kicking the tires on Netlify, but they are officially out of consideration now.
Doesn't make me feel great about Vercel either (which I was looking at)
Id recommend avoiding Vercel like the plague. Their entire business model is poorly aligned with your interests as a customer.
Vercel has spend management controls, setting limits and SMS alerting: https://vercel.com/blog/introducing-spend-management-realtim...
Why? If you use Vercel in the Free Tier you cannot automatically go to the Pro tier.
You need to manually upgrade.
Unless you need more than the Free-Tier Vercel should be fine.
5 replies →
There are lots of reasons to avoid Netlify.
https://github.com/netlify/cli/issues/739
That sounds like a company that is happy to ignore customer pain. Agree, default-on telemetry that cannot fully be disabled is a red flag that on their side, the people with development experience have no say.
This seems like a lot of brouhaha about nothing. They just recorded the fact that a user opted out of telemetry, they didn't stealthily send telemetry against user wishes.
They removed this call in the meantime [0].
[0] https://github.com/netlify/cli/pull/740/files
A few years ago I taught an introduction to website developmment module at a university where students built jekyll sites. I got them to to host on netlify. Now that I no longer teach at that uni and their email accounts will have expired I'm wondering if there's any way I can contact them to tell them to take their sites off netlify... Disaster if they get hit by charges like this.
At the rate it's going this story will probably reach them without your help.
Not necessarily, the majority of them probably wouldn't have ended up in anything tech related. But they probably did end up back in China, so hopefully it would be hard to demand payment at least?
Netlify charge $55 per 100GB for overages on their starter plan. Keep in mind that they’re being billed by their provider at 95th percentile billing. That means they have ports running at say 100 gigabits per second and have a commit of say 50 gb/s and they’re billed a flat rate for that as long as they don’t go over. Because they’re billed 95th percentile they can spike their connection traffic massively for 5% of the time and not get billed more. So Netlify themselves have a safety net and don’t need real-time monitoring to immediately cut their usage. And of course any spikes from a single user are massively diluted among their entire user base. So, yeah, they’re gouging big time.
What they should have is monitoring per user and a default that 503s the site with no overages that has to be proactively disabled by the user. Instead they’re just letting it ride and trying their luck by negotiating down the overage charge to what they think the user can stomach.
It's also worth pointing out that Netlify's pricing of $550 per 1TB is 5x more than AWS, Google Cloud, and other massive cloud providers charge, and those are already known for extortionate data transfer fees (50-100x more than more reasonable providers).
Defaulting to unbounded liability as the standard operating procedure for cloud infrastructure should be illegal.
And it is, I remember cases where telcos were prevented from charging thousands for roaming expenses because they were not licensed to make large loans.
Oh God roaming is one my biggest nightmares and it’s the worst UX to not get charged, too. ‘You want to go on vacation? Make sure to hunt down an option hidden 3 levels deep and enabled by default because we really like money and you didn’t read the fine print? Too bad, we really like money’
1 reply →
looks like it is not just defaulting to unbounded liability. it is forcing unbounded liability as there is no way to turn it off and limit your exposure.
Yep, this is a company that screams for more regulation.
If people are paying attention then Netlify should be done as a company.
I have been using Netlify for years, for my own projects, but also recommended it to all my freelance clients to host the projects that I was building for them. Going forward I will move all my static pages to other hosting providers.
The Netlify team must think: we waive the fees, because in this instance we noticed the negative press and want to avoid this from blowing up. When this happens to other users, we don’t care, as long as it does not go viral.
Such a pity, Netlify has great UX and I was so happy hosting static pages on their service. But without spending limits, this is not an option for me any more. I could not sleep well when there is a possibility of a $10.000 invoice reaching my inbox.
There is a lot to be said for just hosting your own VPS and learning how it works especially for smaller sites. The effort to host is small and you're not going to get smashed with a bill like this.
Pulling my netlify hosted stuff today. F that. Sorry to hnbadges users!
Me too, it took like 30 mins to create cloudflare account and move my small static sites.
Lol I just deleted the account (surprisingly easy). Since I didn’t have a custom domain, just let it die for now.
This and that similar story recently about Vercel makes me feel iffy about these services at this point. I guess I’ll either use Cloudflare pages or simple dedicated machines from Hetzner going forward.
I was searching for 'cloudflare spending limits' based on comments here, at first glance they (like Vercel) seem to have a notify don't terminate policy.
What I didn't appreciate is static assets are completely free: https://developers.cloudflare.com/pages/functions/pricing/#s...
So therefore I assume static cloudflare pages are free.
Incidentally Vercel does seem to have an (annoyingly indirect) way of halting usage based on spending: https://vercel.com/docs/accounts/spend-management
Has anyone implemented this, are there any problems with it?
Here's my workaround for Vercel:
- Enable spend management (https://vercel.com/team_name/~/settings/billing).
- Set webhook to pause the project: https://vercel.com/docs/rest-api/endpoints/projects#pause-a-...
When the amount hits the targeted value, Vercel will call the webhook that pauses the project.
Have you tested it end to end? When was the last time? I'm slightly worried about solutions like that. I like when it's all host's responsibility. (Having your own system on top is nice though)
You could probably set up rate limiting with iptables/nftables as well if it’s a vps.
This happened to me on a much smaller scale about a year back. I was never happier to have stuck to my guns, building my whole site with a single `hugo` command - it made it very easy to migrate off that platform for good.
If anyone has a solid bash one liner to stress test a website, so that I can test whether my cloud billing cap will work correctly if I accidentally try to egress 100 MB of data or something, I would seriously appreciate it. There was one on a blog post here like a year back, using apache iirc, but I forgot to bookmark it.
apachebench (cli command ab), is an Apache licensed stress testing utility, not much relation to the web server, which can be easily used to stress test a webserver.
the binary should be easy to install with your package manager, you may have it installed already
example:
ab http://something.com
Vegeta worth a look if you want something a bit more sophisticated: https://github.com/tsenart/vegeta
Yes! That's the one, thank you.
I am a business user with netlify. I have unlimited functions calls on that plan (fair use of course!) and use their JWT protection that redirects to a login site at the cdn level, so you can rate limit. Not a solution for public static sites though! You are metered on the starter and pro plans after the starting limits.
They seem to have dropped that plan now.
I was starting to move back to traditional hosting as these platforms are convenient, but you do lose control and get hammered for their addon services and simple things like static ips are beyond them, even if you offer to pay.
Also, if their cdn is naughty listed, corporate networks may block your site as you are sharing pro and business plans with free sites that maybe serving malware etc.
Hearing this story has pushed me to move.
I hope they sort that for you, they really should have the ability to protect a site and let you choose what to do if you are exceeding your limits.
Hosting personal sites on cheap hardware at home makes even more sense to me after reading this horror story.
Why do not more technically inclined people self-host? Is it force of habit from how things are done at work?
I would much rather run the "risk" of some occasional dowtime, than keeping the lights on at all costs under a DDoS-attack.
It's impractical because residential internet speed is very asymmetrical and behind NAT. You will still need external service to get around the NAT thing and the fact you don't have a static IP.
You do not need a lot of speed to host a personal or hobbyist blog, and NAT is just something you configure once and it's done. I use my domain providers API to update the DNS records as soon as my public IP changes (with a tiny shell script on my server). I even host some semi-commercial sites from home and it is fine :)
I would much rather run the chance of some occasional downtime in exchange of being in control of the infrastructure and owning my own data. I really like the idea of a inter-connected net that is kinda spread all over, not super concentrated to a handful of data hubs.
Only some ISPs are like that
Why not go for GitHub hosting then?
That could be a nice backup for when I experience downtime, but I would rather run a simple webserver at home for stuff that does not require extreme scale or a lot of resources. Hosting some websites at home is not that complicated and it keeps the dream of a distributed Internet alive!
This is the reason I've never used all of these user-facing serverless services. The price depends on the usage, but if anything goes wrong they are the ones to decide what you pay. It's not comfortable thinking that you could screw up, or get DDoS'd, and the remedy is hoping they wave the bill.
We need a separate ‘shame’ tab at the top for crap like this. There must be so many more of these asshole bills being sent out that aren’t lucky enough to get voted up.
What the hell. Netlify has implemented all these customer-hostile billing changes. Literally the worst. I migrated my company's site off of Netlify when they tried charging for per user on one of my repos (a documentation site). Glad I left them and never looked back.
The pay as you go model is completely wrong.
Companies should be legally required to allow their customers to set a ceiling to their monthly spending.
Being able to set a soft limit (email me when I go beyond it) and a hard limit (shut down my service) seems like a very simple solution. How is it not a thing in 2024?
Also, being at the mercy of a CEO and their understanding is a no go, whether you're a company or an individual. We want to be 100% sure we're not going to get bankrupt when using a service. "Don't worry, our policy is to usually erase the debt" is not an appropriate answer.
That discount to 5% is incredibly odd and hand wavy, I would escalate to their manager.
They should definitely be able to accomodate and account for what should be a very common issue (does).
They can discount to 5% since their fees are 10,000% higher than what it's costing them...
Really uncomfortable with how many services like this are “we scale to your needs” and end up here. I guess capacity planning from oversubscribing bandwidth is its own can of worms but surely that is a bit… less surprise generating.
A successful distributed denial of money is likely much more devastating to anyone except really large companies than a successful DDoS. For a personal accounts it is entirely devastating with no upside, but even for a startup it would probably be better for your site to just go down instead of having huge bills generated.
We really need to have list of these shady SaaS that will try to charge you money when you using their "Free" tier and never given them credit card.
This should not be tolerated. Full stop.
I guess I never want my personal blog/site to be successful. Can't trust that I will wake up to a huge bill from EC2 or my CDN provider. Feels like health insurance in that even when you have insurance, you are never 100% confident you will not get some huge hospital bill. The pessimistic part of me says this is all deliberate to prevent the "little guy" from competing these days.
If you run on a single EC2 instance and you aren’t running an auto scaling cluster or anything of the sort, it would be pretty hard to get a huge bill. I much prefer that and the chance that it goes down then autoscaling or severless. Most serverless solutions have also gotten so config heavy or complex to make changes that most projects feel much better to me on an instance I can ssh into and poke around without having to call up support.
Thanks, I use a "small" EC2 instance for my personal stuff (about $35 a month), with the Cloudflare free version, but I honestly don't know and what would happen if something went viral or a DOS. Would the bill be double? Triple? Would the site simply crash. What does Cloudflare do? I honestly have no idea.
1 reply →
This is such a shame, I found the user experience pretty good on Netlify and now I can’t risk staying with them.
What was the website that got DDOSed? https://jyutping.org/en ? (Since it's behind Cloudflare now, I guess linking it here is fine.)
At first I thought it might've been https://hanhngiox.net/ , but that one appears to have simply expired.
So if I want to migrate off of netlify to something better, where?
Cloudflare. You will get a call if on a free or pro ($25/month) plan [1] if your bandwidth usage is so high it would warrant increasing your plan. Worst case, they turn you off. Preferred over denial of money attack based on your use case. Set and forget after pointing at your origin (time is money).
You'll get a call on any of their plans if your bandwidth usage exceeds certain thresholds, I am assuming your median usage is relatively tame.
Disclosure: Cloudflare enterprise customer, no other affiliation. I don't get anything for saying nice things.
[1] https://www.cloudflare.com/plans/
You typically can't replace Netlify with Cloudflare. You need something like Github actions with some storage, S3 or something and then one can put Cloudflare in front of it all for caching, DDOS protection and so on.
3 replies →
For less than 1% of OP's monthly bill, you can build or obtain a more-than-enough server, drop your static files on it, and serve through nginx. And you get to keep it forever; there's no monthly subscription fee!
Seriously, maybe I'm just old, but I look at the pricing of these hip and modern SaaS products for dead simple software and I cannot believe my eyes. The "old fashioned way" works just fine (and has always worked just fine) and is orders of magnitude cheaper.
Hetzner or DigitalOcean with Coolify [0] works great, it's like an open source Heroku that runs on any host, you get git push to deploy, and a bunch of other features built in. It only works on one machine at a time though so it's not like a CDN but for small sites, it's great.
[0] https://coolify.io
2 replies →
But you need a static IP, stable internet, UPS, and permission from your wife to have a 24/7 powered noisy box in a room
5 replies →
Having built serverless apps and "old-fashioned" apps, I seriously believe the old fashioned way is better.
The best of both worlds is to host on AWS EC2 or a similar product from your web service provider of choice.
1 reply →
A lot less than 1%.
I mean you gotta put your server somewhere (I guess hosting it on your connection?)
I use BunnyCDN to host several sites, they have a minimum cost of $1 per month and I usually pay $1 per month.
I run my sites [0] on Hugo and copy the generated sites (Makefile) to BunnyCDN with their command line tool.
It's a plain CDN, but does include DNS hosting for easy SSL certificates and has scriptable DNS [1] where you can run Javascript for dynamic DNS.
I went with them b/c they are in the EU, but I've stayed because I love them.
[0] e.g. https://www.amazingcto.com/
[1] https://bunny.net/dns/
Love Bunny too, wonderful service and great team. I wish there'd be an easy way to set up auto-deploy to Bunny Edge Storage on GitHub commit (to avoid doing so manually), but I guess it's not to hard to do through GitHub Actions.
1 reply →
Looks nice but then so does Netlify at face level! Seems like it is pay as you go, and there is a settable max spend per month.
Does their ddos protection work differently than Netlify and could Bunny ever pull the same stunt with billing?
3 replies →
I switched from Bunny to BlazingCDN, with same performance... 1000TB - $0.0025 per Gb And the support helps a lot, they even made the config for Asia for free
https://blazingcdn.com/pricing/
1 reply →
Hetzner VPS + nginx works fine and goes a looong way. Alternatively, SourceHut pages or GitHub pages is okay too.
If it’s been a while since you did any server work: https://www.dannyvankooten.com/blog/2024/static-site-hosting...
I'm hosting a few large (multiple TB) projects on Cloudflare R2 with no issues for more than a year now. Super happy.
CloudFlare Pages/ Workers is pretty nice
https://render.com/ is free for static sites with custom domain support and SSL included. been happy so far!
used to use s3 for the longest time, but aside from costing a nominal fee, it's so unnecessarily complicated in this day and age.
Keep in mind that according to their forum you'll be charged $30/100GB for bandwidth over free allowance of 100GB:
https://community.render.com/t/confused-about-the-free-tier/...
> Exceeding allotted Bandwidth does result in automatic overage charges. $30 for additional 100 GB blocks.
So the same shady pracrice as on Netlify.
1 reply →
Just something to keep in mind: this also has about half of additional bandwidth costs (above 100GB) of the reddit post, so in your case you'd be billed for ~$57k or so with similar DDOS. At least they seem to provide monitoring/alerts based on their Security page.
1 reply →
> used to use s3 for the longest time, but aside from costing a nominal fee, it's so unnecessarily complicated in this day and age
its like 10 minutes of setup tops to host on s3
3 replies →
For static sites my choice is usually CloudFlare in front of a S3 bucket. It costs pennies. With cloudflare R2 it might be even easier.
Why not just use Cloudflare Pages?
1 reply →
For personal servers you can use the dynamic DNS feature on your modem in combination with Cloudflare if you like doing it at home, if it's for your business you can see if you can afford the €2 per month for Hetzner managed hosting or your local equivalent.
Depending on your use case, Hetzner is as cheap as it gets while retaining high quality.
I've always been a fan of cloudflare or firebase for static sites.
Github works well for me.
If it’s really pure static, github is great because it’s impossible for you to be billed. If you want some functions, Cloudflare free plan is nice because you can configure it to stop operating when the usage limit is reached, or pay $5 a month for more than you’ll likely ever need for a hobby project. Also bandwidth is free.
2 replies →
I use AWS, S3, Cloudfront, ACM with a budget alert.
I just migrated some projects to Cloudflare Pages for the time being.
Firebase Hosting is rad.
Github static page.
Infomaniak
There's a reason price plans are 'pay as you go' - so that they don't take on any risk for things like this. Moral of the story: never go for pay as you go by usage plans if this scenario is not mitigated through a third party. By no means am I trying to suggest a get out for Netlify here - more a rule of thumb for anyone worried the same could happen to them.
When you sign up for their "Free tier" there are no big red warning that they can charge you $100,000 out of the blue.
But they do tell you the pricing structure, right?
they are taking a risk because they haven't credit checked the user
most people wouldn't be able to pay a $100k bill
I don't see why people are surprised by this or why people are calling it a scam. Netlify and others are extremely transparent about the fact that there are no limits. I completely understand not liking it and can see why the lack of limits would make it a bad option for plenty of people, but I don't see how it can possibly be called a scam.
Because it's unbounded liability.
Not to mention the strong conflict of interest for netlify, who stands to gain from their customers being attacked. Netlify is getting paid for something criminal in nature having occurred.
It's like who is responsible for credit card fraud? If customers are responsible for credit card fraud, and it's their responsibility not to get scammed, then who implements fraud prevention measures and what effect would that have on the volume of fraud?
Companies like these give out ridiculously huge free tiers in the hopes that very few users end up using the high free bandwidth limits. In most cases, they do. However, they do need to make their money back somehow.
I don't really get why people put their tiny static sites on hosts designed to never fall over no matter the traffic generated, no matter the situation. You're running a blog, not a government service. You don't need AWS or Netlify.
The ability to withstand almost any DDoS attack for a high price is a valuable service. It's not a scam. The people who get these huge bills just picked a hosting service that doesn't fit their requirements. I can promise you that the $3 shared hosting providers won't charge you $5k, five minutes after the DDoS starts your site just goes down.
7 replies →
> Not to mention the strong conflict of interest for netlify, who stands to gain from their customers being attacked. Netlify is getting paid for something criminal in nature having occurred.
I think you could argue that Netlify is guilty of racketeering in OP's case.
1. They admit illegal activity happened (a DDoS attack).
2. They demand money to be reimbursed for the illegal activity. However, the reimbursement they ask is several hundred times higher than the actual damages incurred.
My previous understanding was that service would be stopped once you hit past the free tier.
Upon review, it does not look like this is the case. I have several very low traffic projects on which would have never been anywhere close to the free limit. However, if I get involved in a random spam attack, it seems I could be on the hook for several thousand dollars.
This is incredibly dangerous. Netlify is often used as a beginner friendly free tier for static hosting. Not as something that is cheap, but as something that is free. This is just an overall dangerous position to put people in.
It does say it's pay-as-you-go on their pricing page. However they probably should have a giant warning page for new users who don't know that this is how this kind of service works if they want to target the beginner web-dev market. As far as I know, no other similar service has this though.
I looked up the definition of scam and the goal is to make money out of the naivety of the victim. It involves a crisis, the illusion of shared exposure to a risk.
The price of cdn bandwidth is about 0.01/gb on low volume (cloudflare, aws, azure…) so op should be billed around $500 with 40TB. Netlify probably buys this for way less. He was presented a bill at $104k, « generously » reduced to $5k, still a x10 margin. Vercel and Netlify are outrageously expensive for what they do.
>Netlify and others are extremely transparent about the fact that there are no limits
Are they also transparent about the fact that they
1. Won't do anything about a DDoS, and
2. In case there's a DDoS (or some other unusual traffic spike), you'll only get notified waaaaay after the fact when you get the $100K bill, instead of getting a timely alert that would allow you to shut your site down to prevent getting extreme charges?
No and no.
It's a scam.
The primary purpose of these services is to be able to scale up and continue working under heavy load, shutting the site down when this occurs would defeat the entire purpose of the service. I would say that they are transparent about both of the things you have listed by virtue of being one of those scaling serverless hosting services.
3 replies →
I understand the lack of limits but I haven't accounted for DDoS attacks on Netlify infrastructure to impact me. I was assuming this only included real, "organic" traffic.
What I think Netlify needs on their Plans page is to include "DDoS attacks is included in your traffic" as well as their 20%/5% charge system.
> and can see why the lack of limits would make it a bad option for plenty of people
Just out of curiosity, can you see any scenario where it WOULD be a decent option to use a free tier where you may be hit by a $20,000 or $5,000 bill out of the blue and outside of your control? You say "plenty" so I assume you consider this a reasonable system to some?
It smells like a scam, because they can suddenly bill any user they want for a scary number like $100,000, then when the user complains they "generously" reduce that to only 5%, or $5,000, hoping the user will just pay the massively reduced cost. This kind of thing - showing a huge number upfront then reducing it to a "small" number - is a classic scam.
Who controls the DDOS bots? Are they truly a separate entity? There is no direct evidence to link them together, but you would think that an honest company would be more proactive in preventing problems like this for their customers.
According to the linked reddit story, this is a known issue with Netlify and their response to past incidents is basically to pound sand. It all adds up to them purposefully trying to find ways to generate a high bill for their customers and hoping a small amount will pay for it.
> extremely transparent
> they probably should have a giant warning page for new users who don't know that this is how this kind of service works
Pick one
If it’s a static website; use GitHub to host and put behind cloudflare.
If you need a nice front end, spin up a Wordpress instance with a provider like digital ocean or vultr or any other number of places. It’s like $5-10/ month and has terabyte bandwidth without issue typically.
Then put the site behind cloudflare or at least configure a plugin for ddos protection.
Netlify doesn't offer alerts or budget-triggers? This is my nightmare every time I put up my card details with a cloud service provider.
The funny thing is - such platforms "scale" easily with the underlying assumption that scale equals profits, enough to justify increased cost. Needless to say, inaccurate assumption.
Any lawyer here can suggest if a class action suit is appropriate?
The test for federal class action lawsuit includes 4 prongs, all of which must be satisfied: numerosity, commonality, typicality, and adequacy. [1]
I would think (as a former lawyer with only passing familiarity with class actions) that 'typicality' would be the key question.
> to determine typicality the courts consider to what extent plaintiffs’ claims are markedly different or are generally the same (for instance arising from the same event or pattern) as those of other class members with respect to the relevant legal theory and factual circumstances of the case. [1]
The defendant would probably claim that each plaintiff's issues are quite unique. However, this prong is apparently not based on the typicality of the specific facts giving rise to the lawsuit, but rather the typicality of the nature of the claim or defense. And it's apparently hard to 'win' (defeat a class action) via this prong. [2]
1: https://www.bonalaw.com/insights/legal-resources/what-are-th...
2: https://california-business-lawyer-corporate-lawyer.com/clas...
Been using Netlify for 7 years now and I'll be moving all of my projects to cloudflare this week. That's a hell of a lot of risk to host a few hobby sites that virtually nobody visits. Cant imagine running a bootstrapped business and having this happen. Seems extremely predatory.
Free plan can be used with some iamnobody@gmail.com email and without any credit card info. It looks like good idea because I thought too that Starter plan is 100% free and if you would use all yours resources they should shut down your page and wait for improving plan/adding credit card details.
I would not like to pay for something which was advertised as free (don't have a nervous for paying without hard limit for home project, I can host it even on rasberry pi + pay just for address). That's why I find it is better to not use your real name as long you don't want to make real business :(
And that’s how you lose loyal customers who have evangelised your products for years. I’ve helped local mom-and-pop stores, bootstrapped startups etc setup on Netlify and now I fear that Netlify might send them a hefty bill.
I’ll remove all my websites from Netlify and moving to Cloudflare. Auf Wiedersehen.
There should be a "max $$ per day" set by the customer.
For example if my personal blog exceeds $1/day, I am ok if it goes down. Having no limit is insane.
Netlify's 2017 blog post says you don't need Cloudflare.
from https://www.netlify.com/blog/2017/03/28/why-you-dont-need-cl...
"...Top to bottom, our infrastructure redundancies make sure we keep traffic flowing, so there's no need to add more redundancy with Cloudflare. ...
You don't need Cloudflare when you use Netlify
As you can see, we already offer what Cloudflare does, and more. If your site is not on Netlify, perhaps consider us for your one stop solution for hosting, SSL, DDoS protection, DNS load balancing, and continuous deployments. ..."
It's like all cloud providers are in some cartel where they refuse to implement the most basic counter-bankruptcy features.
The request is very simple: max budget per month. Shut it down if it exceeds it. Implement it already and stop with the excuses.
Just switched my site to Cloudflare Pages. It took all of 5 minutes including futzing around with DNS stuff through the (ever-annoying) Google Domains settings page.
I'm usually pretty chill, and can't help but feel like I'm overreacting - but the risk of being sent an unexpected bill of even a few hundred dollars for something outside my control is unacceptable. Trusting that Netlify will do right by me and forgive the debt in a similar situation is not an acceptable strategy for managing that risk, so epiccoleman.com now lives at a provider without this risk.
This is the beauty of static sites, btw - switching to a new host is the work of mere minutes.
Your comment with it taking only 5 minutes made me move my sites. And indeed it was super fast. Thanks for the needed motivation!
I don’t think you are overreacting. I had Netlify in pro mode at work and their pricing tactics are predatory. It was really hard to figure out why I got charged a seat, even though nobody had rights etc. And I couldn’t then remove these seats, I had to wait a month and hope I figured out where those contributors came from.
I have a bunch of pet projects on netlify free tier and I could never afford to pay this amount of money. What are some good alternatives that don't have this issue? I've already noticed cloudflare pages mentioned in these comments.
My pipeline and hosting solutions are:
Static: Github Actions to build and deploy to BunnyCDN
Non-static: selfhosted Dokku on Hetzner
Neither is free, if you're looking for free, Github Pages or Cloudflare for static sites. Free non-static, I'm not sure there are solutions that don't have the same problem Netlify has.
Dokku is free (the Pro version is paid but that is really more an enterprise kind of thing).
I’m stunned there is no way to limit spend and have warning and a cap on a service like that. You should be able to specify billing thresholds and have sensible defaults when the downside is basically unlimited. Basic customer UX.
Does this issue only occur if you have billing info on file?
I'm using the free tier and have no billing info set. According to this https://github.com/netlify/ask-netlify/issues/6#issuecomment... > if you have an event that puts you over the free-tier limits, Netlify will ask you to update your billing information and add a CC
Although worryingly > We just had this happen and our site didn't stop working.
Is there any way to ensure if you hit the limit sites just stop working and you don't get billed?
I'm also interested to know this. I have a couple of static sites running on the free tier for friends/family and now I'm planning on moving them all to a VPS as soon as I can.
It is beyond ridiculous that serverless providers don't offer a way to cap spending. The idea that it might cause your site to go offline is a complete non-argument. That what I _want_ to happen. I want to be able to say sure, I'm happy to sustain 10x traffic for a few hours, and maybe 3x sustained over days, but after that take it offline. I don't want infinitely scaling infra precisely because of the infinitely scaling costs.
No, and this is by design. If you go over the limits (can also happen if a build machine times out, ask me how I know), you will be billed without any recourse. If you have no billing information and refuse to set it, at the very least they'll permanently ban you from their platform.
Which, if it remains the only consequence, seems like a blessing now.
Wow, insane. I would not have guessed that they simply charge you for additional bandwidth instead of just shutting your site down for the rest of the month.
This is not something you would assume from the pricing page. On the pricing page, they show you:
In my understanding, an Add-on is something I need to enable, not something THEY enable for me if they see fit. When I am logged into Netlify, they tell me:
Which apparently means nothing, as they will just automatically enable the add-on for me!!!
I have seven static sites on there, deleted 3 of them right now, will migrate the other 4 soon. Unbelievable bullshit.
I had a couple of orphaned static sites on Netlify. Just deleted them, and won't put anything else on there now.
Now I get that this is their 'product'—selling hosting to high traffic customers, and I don't particularly begrudge them charging whatever margins they think are suitable for their product and let people make their decision.
But no mechanism to cap maximum spend is completely ridiculous. Even if you require a minimum cap of say $5/$10 dollars, as long as it's clear and transparent, I think that would be reasonable.
Anyway, this has scared me into never trusting Netlify with anything any more.
I must add that at Netlify are consistent across disparate systems.
They consistently missed someone that doesn't get more than 10GB traffic a month now maxing out a 5Gbps line (60TB/day stated in TFA) for several days in a row.
And consistently missed someone with a tiny bill's now racking up 10s of thousands of dollars per day. Even if they do run a mainframe and batch process at the end of each day, that still went a few days. If extending lines of credit of tens of thousands of dollars is legal, that's very generous.
What further consistency is available at Netlify?
It feels even worse than that if you consider with this being widely known, competitors could make a point to attack Netlify customers knowing it may bankrupt them.
Cloudflare got a bunch of new "customers" judging by this thread.
Competitors could rinse and repeat this strategy to put netlify in the dirt for good.
Reading this article made me realise not to use these services for good. $5000 is a lot man.. It's not that we make money out of thin air.
I am a Netlify Pro customer. I was not aware of this unlimited spending concept. This is simply unacceptable from a risk-perspective.
Just to toss this out there: There are too many services that don't really let you cap your spend. AWS, for example: You can set alarms, but as far as I know, you cannot sent a spend limit.
I had a couple of small businesses hosted there, but this always worried me, so I moved them to a local provider, who provides what they need for a flat fee.
Is the entire idea of paying for bandwith not absurd ? You can't really control it
I have some websites with milions + autogenerated webpages and it's flooded by bot activity that I don't particularly care about.
I've blocked some through cloudflare but some look exactly like he describes: old machines with old versions of some OS and browser scattered around the globe that seem to be scanning my entire website, maybe for AI training purpose ?
The point is, I can't block them at all.
see this thread here : https://www.reddit.com/r/webdev/comments/1azv0fs/is_this_tra...
There are providers that change you for connection speed and have no limits besides that. They're more expensive.
I guess the questions starts for more important websites.
I pay 10 euro / month for 3 python apps (inluding the one with the bot traffic around 22 Gb per month).
For static websites anything more would be insane
A lot of today's managed service solutions turn scaling problems into billing problems. While that's a useful tradeoff for many business use-cases, there should still be a way to set limits so you don't wind up with insane cost overruns.
Render may have the same problem: https://community.render.com/t/usage-100gb-for-a-static-site...
(Render CEO) Cloudflare DDoS protection is built into Render, and we don't charge for bandwidth classified as DDoS.
only if you have a card on file, otherwise it'll suspend your account? https://community.render.com/t/will-it-make-me-pay/11161/2
You just posted it on Reddit and I already see few comments of people say they will never use it. Stuff like this can cause PR damage worth millions to a company. I'm pretty sure that it will go viral and they won't charge you.
Exactly. I think this also might be a reason why they didn't implement ddos protection on this level. So, they can grab as much as they can.
I can't get past why Netlify's infrastructure continued serving a 3MB file to a botnet 55,000,000 times? I would've assumed their system was smarter than the ubuntu/apache install in my basement.
That it why I always refuse to use hosting services with on-demand pricing, especially for personal projects: in the worst case scenario I'd rather see it down for a few days than be indebted for whatever amount.
"190TB in 4 days"
That's approximately one month of transit through a gigabit connection, which indeed could be pretty expensive. Even at IP transit prices it would be something like $300/mo for a 1GB connection via HE but you would need something bigger to handle this traffic. I pay about $2k at bulk for that amount of bandwidth. Any way you cut it, you're getting some sort of bill (or kicked off) for that without some sort of ddos forgiveness.
That said, $104k feels excessive to me for static hosting ($0.57/gb, did I do that right?).
I'm a huge Netlify fan, but stories like these are scary. Just spent the last hour building a custom serverless function that sends email / SMS at various bandwidth usages - potentially could be improved with an auto shutdown of the offending site beyond 75 GB:
https://gist.github.com/princefishthrower/4517ff44a9f4c5b2d2...
For now it's still a workaround... looks like I'll be migrating to Hetzner soon...
It’s crazy to me that platforms like these don’t have a reasonable spend limit set by default or easily configured.
I understand it can be difficult to stop spending exactly at a certain amount to the cent or event to the dollar or, hey, tens of dollars if traffic is really crazy. But hundreds? Thousands? Tens of thousands? Your system should be able to measure that and stop it in time.
The lesson to be learned is to pay attention to pricing before signing up, and don't buy really expensive services. Assuming a gigabit connection, you have the potential to pay $13000 per day.
Posted 4 hours ago with 958 points and it's already sitting at 68th place? Was the story too inconvenient for Y Combinator given that Netlify has bought 2? of YC backed companies?
very strange indeed, never seen anything like this on HN before...
And back to the front page at #23.
I hate that a lot of times the only solution is to post on HN/reddit/Twitter etc publicly. Just imagine all the other people who don’t use social media and ended up paying…
There's good discussion on this here, and on Reddit.
In addition to all of that, the real moral of the story is to not use anything with dynamic billing for personal projects, even if there is a free tier. Always ensure that the free tier does not automatically turn into a paid tier.
That said, I've never used Netlify, so I don't know how they present this service at the time of signing up.
GitHub Pages is great for static sites, and included Jekyll build support is a bonus.
smelled this company long ago: https://news.ycombinator.com/item?id=24939931
I am running my static sites on a VPS for only 1€ per month. It includes unlimited traffic, DDOS protection, and IDS/IPS at the service provider level.
There are no surprises! :)
Provider name and specifications please?
IONOS 1 vCore, 1GB RAM, 10 GB NVMe
It’s probably worth wrapping the entity that contracts with these type of services in a separate anonymous LLC structure so a DDoS attack doesn’t bankrupt you.
Oh wow, I'm already hosting a small website with them and was thinking of hosting more. Definitely out of the question now, will move to Cloudflare.
Cloudflare has free DDOS protection.
Additionally, they own ( or co-own) their DC, while Netlify and Vercel doesn't. So they can fix any billing issues at their end.
Just deleted my netlify site and accounts after reading this. I had no idea what I agreed to and the potential consequences. Dodged a bullet!
This is going to be a dumb question. I am new to coding ( 3 months in ). I am building a simple static personal website with GitHub pages. I am worried since I had to input billing data to GitHub for my global campus application since I am a student.
Is this something I need to worry about? Does cloudflare provide a service that is cheap that can prevent something like this for my GitHub pages site?
Getting DDoSed is rare but highly destructive when it does happen if you happen to use something like Netlify. GitHub Pages is under fair use and only for personal projects (no commercial uses). For commercial use, something like Cloudflare Pages would be better. Cloudflare can also help with their whole suite of tools with mitigating attacks and they will call you if you do exceed the free tier according to this comment: https://news.ycombinator.com/item?id=39520894.
If they offer a platform as a service and they fail to protect that platform from ddos attacks, they should incur the cost and not the customer
People deploying low-traffic side projects - to VPSes or similar that allows price capping - and who do use a database: what deployment stack do you use these days?
I assume Docker + something is most popular, but what something? Does terraform work sanely for cheap virtual hosting? Ansible? I don't want to manually install any more stuff than the minimum I can get away with!
For my own projects I do Terraform/Pulumi + Ansible. I use Hetzner & DigitalOcean and this setup works great with both.
I don't use docker for my projects, as I deploy on RHEL like systems which I'm intimately familiar how to configure (and have snippets I mix and match).
You use both terraform and ansible in the same project? I always thought of them as competitors filling more or less the same role, do you find it useful to use them together? Is it that hetzner and digitalocean TF providers do a good job but provide limited functionality, and ansible fills in the gaps for you?
1 reply →
Any cloud service that offers prepaid billing on a credits model would get a lot of customers.
(Prepaid is also superior from a cash flow perspective too.)
https://upcloud.com does pre-paid billing.
Why is there no self-hostable Netlify yet?
Is it just that no one would pay for it? I'm well aware of how terrible a customer developers make, but has to be nearly a non-issue with Hetzner in the USA now, with how much free traffic they give you (or any other provider, DO, etc). There's even Cloudflare R2 nowadays.
Your blog probably doesn't actually need sub 100ms serve times.
https://coolify.io/ might be worth a look
Yeah there are a bunch of selfhostable things:
Caprover (https://caprover.com/)
Dokku (https://github.com/dokku/dokku)
But people still choose Netlify and Vercel for ease of use I think.
Maybe we need something that's just Netlify. The closest I've seen to the "right" UX is Ness:
https://ness.sh
Though of course it's a tui so some people will get turned off of that (especially people who are willing to spend).
1 reply →
> This feels more like a scam to me. Why do serverless platforms like Netlify and Vercel not have ddos protection, or at least a spend limit?
Normally it should just stop serving the bandwidth once the limit hits, just like when you max out your credit card, it just gets declined, they don’t charge you forever just because someone else is abusing it..
Typically, it is not even a Distributed Denial of Service (DDOS) attack; instead, it is traffic from (SEO) bots that crawl through every possible combination of the page, such as query parameters for web shop filters. I hate those down to the guts, because you have to design a rate limiter that will limit them and let other traffic through.
> Don’t pay the bills, post it on HN! They have good reach.
This line scares me so much, the fact that we have to rely on an external forum to sort things out. What if my HN post doesn’t gain any attention? Is that it for me?
I understand that someone would pull all their strings in hope for solution but still, companies should be available directly.
When this occurs does the company count that as revenue and record the discount or do you adjust the accounting to reconcile?
I've seen in the thread that bobfunk (netlify CEO) and raiyu (cofounder at digital ocean) say the bill forgiveness is the right strategy. It got me curious on the above question though.
I like companies like this, that give you such a straight-forward reason never to do business with them.
I will ask the same question as for BigQuery:
>I thought you could cap(limit) your spending e.g. "if I reach $1000 in costs, abort/pause operations"[0].
[0] https://news.ycombinator.com/item?id=39472553
If we take 60TB / 24 hours we get roughly 5.5Gbits/second of traffic.
You can get a 10gbps connection, per month, for under $4K a month to a datacenter. For full usage for one month, not one day. If buying at the scale of a much larger bandwidth user, the price is much less.
If architected properly having many people request the same file should make things work better and make it cheaper.
The internet and www are obviously marvelous technologies with many people to praise, webtorrent is nice but in a way a hack replicating things we already had.
We can do better
Doesn't Netlify have budget alerts?
I know during heavy DDoS attacks they might be too late, but also cache?
I'm wondering whether this story has been manually downranked.
Posted only 3 hours ago and 800+ votes but it's suddenly dropped from top 3 (on page 1) to 38th (on page 2).
Is it worth noting that Netlify bought two Y Combinator startups or is that a crazy conspiracy theory?
> On May 19, 2021, Netlify announced the acquisition of FeaturePeek, a Y Combinator and Matrix Partners backed startup that enables developer teams to preview frontend content.
> On November 17, 2021, Netlify acquired Y Combinator and SignalFire-backed OneGraph to allow for the composition of apps with APIs and services using GraphQL.
If this happened to me, joke would be on them, I'm grandfathered in with no credit card details attached to my file. Still, I had a mild panic attack reading this and am now thinking I shouldn't use them anymore.
I'm curious what would've happened had the bill been for 10k or 1k? Would it still have been reduced, would OP have paid instead of posting on reddit/hn, would it have gotten as much attention?
Same spiel as all the big clouds...not allowed to protect yourself. Unlimited risk all on you and afterwards you may beg for forgiveness from support and when that fails go to the other support page aka hn.
Bloody scam.
How does one even check Vercel and Netlify for costs your incurring? I have Vercel and just went to my account and couldn't even find my spend lol.
Btw -- I don't really use these. Usually spin up my own VPS
I was regularly at 80-90% of my free-tier Netlify bandwidth of 100GB.
The site was not live.
How come a post with so many votes and comments, submitted just 4 hours ago, is already on the 3rd page? Would have missed an interesting discussion haven't I seen a link to it on Reddit
I love it that the reddit post asks him straight away to make a post on HN. Its like everyone knows HN is the default customer support for these 10x web 2.0 hyperscale companies...
Just use Hetzner and never worry about getting into debt and having to beg some company's customer service to be so kind and reduce your imaginary "debt" with them.
Hmm, I think I'm going to look into Cloudflare static hosting after this. I don't think they charge you on their free level CDN if you get DDOSed.
This is why I put my Netlify site behind cloudflare. I know CF is not well liked by the HN crowd but they don't pull this sort of moves.
How come no one is mentioning that the source of all this traffic was the downloading of an mp3 that the OP probably doesn't own rights to distribute?
I don't see how that's in any way relevant to the real story. It just as easily could've been a 3 MB photograph.
Put Cloudflare proxy in front of Netlify/Vercel deploys
Every Netlify project is assigned a Netlify subdomain (i.e. `example.netlify.app`) that cannot be removed or proxied.
If anyone figures out what your Netlify subdomain is, it's my understanding that they can DDoS you and there's nothing you can do about it.
That makes sense, but is the Netlify subdomain visible from your custom domain? How would they be able to figure it out, other than humans leaking it somehow?
1 reply →
Wow.
I can't read it because I block reddit, but I assume it's a DDOS? With a bill that large, it would actually make business sense for them to team up with DDOSers.
That's why for my static site, I always prefer to use GitHub. Simple, deploy changes in push and can connect to a custom domain if needed.
If Netlify and Vercel don't offer ddos protection, or at least a spend limit.. any other recommendations for next/ react hosting?
(I work at Vercel) We do have DDoS protection, and spend limits. We're imminently launching improvements to both of these as well Importantly, OP was on a free tier on Netlify which does allow you to pay for additional usage on demand. Vercel's free tier does not – if you are on a hobby plan and you go over the included usage, your site will be paused automatically.
1: https://vercel.com/blog/introducing-spend-management-realtim...
Thank you for clarifying, and sorry for making that incorrect statement.
Woah! I had two hobby sites from 2018. I thought I shut them down after the policy change, but I may have forgotten. Just deleted my account.
Well you should guard your site with Cloudflare, at least Cloudflare will not charge me for this situation.
A No Surprice Act is needed for these companies.
Terrifying. Just deleted all of my Netlify sites.
Is this website a court or ombudsman or what? If a business suddenly charges you 10000x more than usual, just take them to court.
They might not get Ddos protection because Shirky principal. Institutes further problems they are meant to solve
After doing some math it doesn't feel like a ddos:
- $104k at ($55 / 100 GB) = 189 TB of traffic
- It means the popular ~3.5 MB media file was downloaded ~54M times
- Which sound like a lot, but if you get popular in a country with 1.4B people, it's not (~3% accessed).
What if it happens at AWS Cloudfront? At $.1/GB it sums up to ~$18k. In the light of these, Netlify's offer of ~$5k seems generous.
The author claimed Netlify's own support agent said it was an L7 ddos and offered a 95% concession because it was a ddos.
There is no reason to question whether it was a ddos or not because, allegedly, both parties in this dispute already agree it was a ddos.
Also, DDoS or not DDoS it's very reasonable to believe that an individual (or even a company) isn't ok for a 100K USD bill.
Billing amounts should go through quotas requests, so you can explicitly ask to be migrated to the upper level, but by default have an active safety net.
This is perhaps a good lesson for considering cloudflare.
I’m surprised Netlify has no ddos protection.
Yeah, not gonna use their services, don't want to deal with an issue like this.
That is very concerning, I guess I am moving my site from Netlify urgently.
Is it possible to put my Netlify site behind Cloudflare DDOS protection?
Is it the same for Vercel? It's even more expensive there. Scary
Thats criminal, hope you get things sorted. Amazon would wave it.
Moved from Netlify and deleted my blog just in case
I ended up self hosting my own stuff behind cloudflare
good time to point out tiiny.host -> https://tiiny.host/
whoah, thats expensive. reminds me free services from early 2000's (lycos, 50web etc) with these file limits and banners
I feel like unless a customer very explicitly opts in saying "I NEVER want my site to go down" then the host should just shut the site down if traffic spikes to high levels.
>Since Netlify charges 55$/100GB for the exceeding bandwidth
Absolutely absurd fees, there is no basis in reality for that. Sounds like a very scummy company.
I hope you don't live in the US buddy... If you don't, you're probably fine. If you do and Netlify decides to go after you, it's probably gonna be tough and costly.
Which is crucial Netlify is called out for this. Hiding behind the "fine print" is pathetic, not a way to do business.
I mean I get it but you get notifications and emails every time you trigger extra traffic. Don't get me wrong it should be automatic or there should be a safety net but this is not only on netlify.
Good reminder, just moved my blog to cloudflare pages!
I'm assuming they have DDoS protection by default, being a Cloudflare product? Or is the reasoning simply that they allow you to set a bandwidth/usage cap?
Yes, that was my understanding. Also unlimited bandwidth even on the free tier. So you get protection from DDoS and asshole business practices ;) Although there might be other caveats
Scary. I deleted my netlify account. Just in case.
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]
It's called web scale.