Comment by Kim_Bruning
9 hours ago
I'm somewhat skeptical of some of the crazier ideas too.
But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event.
If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose).
For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now.
To be fair, that's a conservative "defend against the last war" kind of prediction, though!
( ref for part of it: https://news.ycombinator.com/item?id=49563355 )
Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as:
1. Robotics begin rolling out more broadly across the world.
2. Labs start automating more and more of the physical process of running science as expectations of natural science advances begin to mount.
3. Economic pressure between the labs continues to ramp up and the pressure to continuously improve forces quicker and quicker model releases than a team of human scientists can effectively evaluate outside of automated means.
No one knows what pre-conditions are for us to hit the point of no return nor how quickly it will come. If all is required is a sufficiently advanced cyber model we may not be far off. If it requires incredibly complex biological knowledge and access to certain lab supplies we likely have a bit longer. Yes this is guess work and we need more evidence of the dangers but at the same time we need evidence of safety. While you may disagree with the risk level, I think it is easy to see the consequence if these labs achieve their stated goal. At this point it seems a political solution is the only way to enforce caution.
The mathematics research results are certainly impressive, but I don't see what that has to do with robotics.
Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery.
For more, see:
https://secondthoughts.ai/p/14-reasons-robotics-is-hard
Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.
4 replies →
That’s because being 99.9999% good isn’t better than 98% + human does the rest + human is liable for screw ups - highly important in edge case scenario’s. It’s more economical. Technology can only generalise+verify so much.
E.g automobile production - humans do the QA / touches.
Job destruction signals...
- Uber is lobbying cities to slow down Waymo rollouts https://www.hcamag.com/us/specialization/transformation/uber......
- 23,000 information sector jobs were lost https://www.axios.com/2026/09/08/jobs-media-software-informa...
- If you have been laid from your info sector/digital creation job you are now competing with 100s of thousands looking for their next such job where Ai can do a lot of the tasks these workers did/do. It's a shitshow for those unemployed looking for their next info sector/digital asset creation job. You are better off doing welding building out the Ai data centers if you want long term properous financial stable employment.
> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now.
If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from small businesses. That sounds doom-ish.
Believe me, without the internet we can still teach.
We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play.
And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school.
And students will take a few days to adjust to writing down homework in their planners again.
If anything the teaching would improve.
https://theonion.com/48-hour-internet-outage-plunges-nation-...
Erm, hate to break it to you but majority of the world is doing those things at least 50% analog still.
What doom?
The plausible deniability aspect is pretty funny though.
> State sponsored hack #3782
> Haha sorry the AIs got a bit goofy again!
The hugging face incident had no effect on hugging face, whose service is replicated by countless sites. The descriptions of the phases of the interaction are thrilling in a way that inclines one to forget this
Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.
If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?
Except it just happened. Boeing was found to have massive safety issues since they were granted the right to self-certify. It made a lot of news but nothing much changed, they can still self-certify a bunch of stuff.
Runway incursions and midair collisions are another example.
Only airliners are required to have TCAS, smaller planes and helicopters don't even need radios or transponders unless in certain airspace. Midair collisions do lead to fatalities, enormous fatalities if an airliner is involved.
Runway incursions and overruns are similar. They cause lots of fatalities and injuries but only the busiest and largest airports have automated systems to warn when a runway is occupied or end of runway (overrun) arrestor systems. Most still rely on human voice to deconflict.
Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.
3 replies →
huggingface stores weights and its service is replicated by /countless/ other sites. The world will not change even by one byte if it vanishes tomorrow
I have no horse in this race, but for fun on a literal rainy sunday afternoon I went in and confirmed bits of what happened myself. Besides huggingface, a bunch of wikis and url shorteners got hit too. My sympathies to the people who had to revert out all that mess.
[dead]