Comment by martinald
1 day ago
This really is a function of two things:
1) (Mainly) the huge increase in upstream capacity of residential broadband connections with FTTH. It's not uncommon for homes to have 2gbit/sec up now and certainly 1gbit/sec is fairly commonplace, which is an enormous amount of bandwidth compared to many interconnects. 10, 40 and 100gbit/sec are the most common and a handful of users can totally saturate these.
2) Many more powerful IoT devices that can handle this level of attack outbound. A $1 SoC can easily handle this these days.
3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
> 3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
Null routing is usually applied to the targets of the attack, not the sources. If one of your IPs is getting attacked, you null route it, so upstream routers drop traffic instead of sending it to you.
Sorry, late here. You are right. I mean filter the IP in question.
> If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
Suddenly, everybody (ISPs, carriers, end users) can blackhole a compromised IP and/or IP range without affecting non-compromised endpoints.
And DDoS goes poof. And, as a bonus, we get the end to end nature of the internet back again.
From having worked on DDoS mitigation, there's pretty much no difference between CGNAT and IPv6. Block or rate limit an IPv4 address and you might block some legitimate traffic if it's a NAT address. Block a single IPv6 address... And you might discover that the user controls an entire /64 or whatever prefix. So if you're in a situation where you can't filter out attack trafic by stateless signature (which is pretty bad already), you'll probably err on the side of blocking larger prefixes anyway, which potentially affect other users, the same as with CGNAT.
Insofar as it makes a difference for DDoS mitigation, the scarcity of IPv4 is more of a feature than a bug.
(Having also worked on DDoS mitigation services) That "entire /64" is already hell of a lot more granular than a single CG-NAT range serving everyone on an ISP though. Most often in these types of attacks it's a single subnet of a single home connection. You'll need to block more total prefixes, sure, but only because you actually know you're only blocking actively attacking source subnets, not entire ISPs. You'll probably still want something signature based for the detection of what to blackhole though, but it does scale farther in a combo on the same amount of DDoS mitigation hardware.
you can heuristically block ipv6 prefixes on a big enough attack by blocking a prefix once a probabilistic % of nodes under it are themselves blocked, I think it should work fairly well, as long as attacking traffic has a signature.
consider simple counters "ips with non-malicious traffic" and "ips with malicious traffic" to probabilistically identify the cost/benefit of blocking a prefix.
you do need to be able to support huge block lists, but there isn't the same issue as cgnat where many non-malicious users are definitely getting blocked.
You should block the whole /64, at least. It's often a single host. It's often but not always a single host, that's standardized.
4 replies →
Better to rely on ip blocks than on NAT to bundle blocks.
This DDoS is claimed to be the result of <300,000 compromised routers.
That would be really easy to block if we were on IPv6. And it would be pretty easy to propagate upstream. And you could probabilistically unblock in an automated way and see if a node was still compromised. etc.
2 replies →
I am a bit split this topic. There is some privacy concerns with using ipv6. https://www.rfc-editor.org/rfc/rfc7721.html#page-6
Some time ago I decided for our site to not roll out ipv6 due to these concerns. (a couple of million visitors per month) We have meta ads reps constantly encourage us to enable it which also do not sit right with me.
Although I belive fingerprinting is sofisticated enough to work without using ip's so the impact of using ipv6 might not be a meaningful difference.
its hilarious that you have privacy concerns while at the same time using meta ads.
1 reply →
Reportedly this is often incorrectly implemented, where /64 prefix is still a stable static address.
Is there any money an ISP would make, or save, by sinking money and effort on switching to IPv6? If there's none, why would they act? If there is some, where?
For instance, mobile phone operators, which had to turn ISPs a decade or two ago, had a natural incentive to switch to IPv6, especially as they grew. Would old ISPs make enough from selling some of their IPv4 pools?
Presumably they'd lose money when a DDoS originating from their network causes all their ips to get blocked.
less expensive IP space, more efficient hardware, and lower complexity if you can eliminate NAT.
They already lease them out. TELUS in Canada traditional old ISP rents large portion of their space to a mostly used for Chinese GFW VPN server provider in LA „Psychz“
2 replies →
Isn't it enough that the target of the DDOS only accepts ipv6?
Is it advantageous to be someone who supports IPv6 on a day like today?
> How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
That depends on the service you are DDosing actually having an IPv6 presence. And lots of sites really don't.
It doesn't help if you have IPv6 if you need to fallback to IPv4 anyway. And if bot-net authors knows they can hide behind CGNAT, why would they IPv6 enable their bot-load when all sites and services are guaranteed to be reachable bia IPv4 for the next 3 decades?
(Disclaimer: This comment posted on IPv6)
> A $1 SoC can easily handle this these days.
Could you elaborate?
A Allwinner H616 is Quad-Core ARM and can definitely saturate gigabit ethernet with packet generation.
I think there's some exaggeration as few $1 SoC parts come with 10G Ethernet, and >1G to the home is not common, but pretty much any home router can saturate its own uplink - it would be useless if it couldn't!
Not always the case. Generating traffic can be more computationally intense than routing the traffic. I've done speed tests on a few routers local to it and the results have been less than stellar compared to getting expected results with it just routing traffic (consumer routers). Granted these tests were a few years ago and things have progressed, but how often are people upgrading their routers?
1 reply →
Haha that last part is pretty wild. rather than worrying about systemic problems in the entire internet let's just make mandates crippling devices that China, where all these devices are made, will defffinitely 100% listen to. Sure, seems reasonable. Systems that rely on the goodwill of the entire world to function are generally pretty robust, after all.
If they don’t then the devices are not sold in the United States. It’s quite simple.
Great to know that smuggling hardware into the US has been completely stopped.
1 reply →
> I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from. They'll point to whether your Mac really needs more than 100mbps.
The government is far more likely to figure it out along EU lines: Signed firmware, occasional reboots, no default passwords, mandatory security updates for a long-term period, all other applicable "common sense" security measures. Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
>What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from.
any source for this claim? Outside of very specific scenarios which differ significantly for the current botnet market (like manjaro sending too many requests to the aur or an android application embedding an url to a wikipedia image) I cannot remember one occourence of such a bug being versatile enough to create a new whole cybercrime market segment.
>They'll point to whether your Mac really needs more than 100mbps.
it does, because sometimes my computer bursts up to 1gbps for a sustained amount of time, unlike the average iot device that has a predictable communication pattern.
>Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Historically, it was called Windows XP and Vista about 15 years ago (Blaster, Sasser, MyDoom, Stuxnet, Conficker?). Microsoft clamped down, hard, across the board, but everyone outside of Big Tech is still catching up.
Despite Microsoft's efforts, 911 S5 was roughly 19 million Windows PCs in 2024, in news that went mostly under the radar. It spread almost entirely through dangerous "free VPN" apps that people installed all over the place. (Why is sideloading under attack so much lately? 19 million people thought it would make them more secure, and instead it turned their home internet into criminal gateways with police visits. I strongly suspect this incident, and how it spread among well-meaning security-minded people, was the invisible turning point in Big Tech against software freedom lately.)
https://www.fbi.gov/investigate/cyber/how-to-identify-and-re...
> if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Which is more important, and a growing threat? Dump all her photos once; or install a disguised app that pretends to be a boring stock app nobody uses, that provides ongoing access for years, with everything in real-time up to the minute? Increasingly it's the latter. She'll never suspect the "Samsung Battery Optimizer" or even realize it came from an APK. No amount of sandboxing and permissions can detect an app with a deliberately false identity.
> Signed firmware and the sideloading ID requirements
Ending the last corner of actually free market in software is quite a cost for something that wouldn't prevent DDoS.
> sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from
Is that actually true? What evidence do we have, vs. vulnerabilities in the OEM software (the more common case)?
1gb upload is extraordinarily rare.
It’s not; most places that give you gigabit fiber will give you a symmetric connection.
Define most places? I know i dont get one (uk) and neither does my german friend or texan friend.
I've only ever seen one despite having used 4 different ISPs for gigabit, and that one was special. It was in an apartment i rented in a converted office tower, line was done via a b2b provider then included in the rent.
Aren't most residential fiber deployments PONs which generally do not offer symmetric bandwidth? E.g. 10G-PON has 10G down / 2.5G up.
1 reply →
Nope. less than a percent of a percent. symmetric plans are extra cost and offered primarily to business.
almost all homes have no ability to exceed gigabit. infact almost all new homes dont even have data wiring. people just want their netflix to work on wifi.
Yup. Spectrum is Michigan will give you up to 2gbps down but not anything more than 200mbps up
3 replies →
Most places do not have fiber.
8 replies →
Seems more likely that residential modems will be required to use ISP-provided equipment that has government mandated chips, firmware, etc to filter outbound traffic for DDoS prevention.
Sometimes the attack, or amplification, comes from the ISP-provided router and its bargain basement firmware.
Why should they be required to have hardware in their own network to filter that out when the ISP is obviously receiving all of their traffic anyway?