Comment by dzink
1 day ago
A huge chunk of humans are sedated with infinite supply of cortex-disabling short form video and games.
Another huge chunk are too distracted by having to scrape by for a living and work multiple jobs or raise kids and survive financially until exhausted. That second group will keep increasing as the first flows into it.
The rest are aging, disabled, or too young and pegging themselves majorly in the first category until they hit the second.
The people aware enough to hold on to their brain and do something with it in their time available are trying to figure out AI and how to make money with it. The variable rewards of promoting AI are turning into an addiction with some of them, especially if grasping for straws with little inherent insights into the problems prompted.
So if you are able to fly above the AI-generated addictions and have the privilege of time to do it, see what you can do.
One of the dilemmas of trying to communicate the full spectrum of AI Risk, is trying not to insult the intelligence of the human animal in the process. And don't get me wrong: what human wetware can accomplish with 20 watts is the most miraculous thing in the known universe. And yet how many of us can have our cognitive sovereignty one-shotted by engagement algos, Skinner boxes, gameplay loops, propaganda, advertising, flattery, social conformity, bias, fantasy, charismatic demagoguery, or straight-up bullshit?
If we grant that we are on track to make something smarter than humans (I think so): it's almost a face-saving white lie to spin yarns about a Skynet nuclear apocalypse, or a 7D chess move to mass-assemble a nanovirus with 100% lethality without anybody noticing. I do think those scenarios are worth taking seriously; but what's harder to communicate, is just how effectively a superhuman AI (or a diverse ecology of agent swarms) might be able to manipulate human behavior. It's something few of us are able or willing to truly process (not least because how many of us live in denial of how much our nervous systems are already hacked by technomodernity).
The appropriate analogy for what's to come may look less like the anthill carelessly demolished to make room for a highway, than the domesticated worker ants from Tchaikovsky's "Children of Time".
> but what's harder to communicate, is just how effectively a superhuman AI (or a diverse ecology of agent swarms) might be able to manipulate human behavior
You don't even need superhuman AI for the most effective use --- hijacking democracy.
Imagine you have an AI tool capable of successfully persuading 5% of viewers with individually-targeted material.
Congrats: you've just won the election.
All it takes is hooking that AI tool up with existing likely voter lists (parties have) augmented by commercially available ad-targeting profiles (parties can get).
Now taking Polymarket bets on when we'll first see an AI agent run for office. Voters have every right to be cynical at the moment; if the AI adopts a charismatic enough video persona, I could see some of the public going for it merely because it's some kind of shake-up to status quo.
Given that such a thing makes no sense legally (as of now), it would probably be done with the centaur model: a human meat proxy who pledges to follow the AI's governance advice.
Or you could just hack the results.
Hypothetically.
But yes - superpersuasion is the real danger. We're very, very persuadable and easy to manipulate, and the voters with the lowest cognitive abilities are trivially easy prey, with a huge ROI for minimal investment.
The AI bot farms are already running. What we haven't seen yet, so far as we know, is spontaneous superpersuasion aimed at leaders.
Are leaders any less susceptible to it than everyone else? Especially if they're narcissistic and easily flattered?
4 replies →
> Which is why the Matrix was redesigned to this: the peak of your civilization. I say your civilization, because as soon as we started thinking for you it really became our civilization, which is of course what this is all about.
- Agent Smith
> The appropriate analogy for what's to come may look less like the anthill carelessly demolished to make room for a highway, than the domesticated worker ants from Tchaikovsky's "Children of Time".
Over 1% of US GDP is being allocated to the datacenter buildout. Have we already started getting domesticated or is this still just human capex?
Humans seem to struggle conceptually with the liminal space between selection-pressure automata (an RNA virus; an insect which evolved camouflage), and an evolved complexity with agency, capable of "understanding" its actions (certainly humans; arguably many animals). And this makes a certain sense: we evolved to treat agentic things as being categorically different from non-agentic things. If we see sudden movement, it's vital to rapidly assess the difference between a tree branch in the wind, versus a predator.
The evolved complexity of the corporation seems to fit within that middle space: more sophisticated than a stick-bug (which exists merely from non-stick bugs being eaten), but not quite to the point where OpenAI/Anthrophic/Google/etc can "understand" its actions. And yet those quasi-intelligent feedback loops, evolving from iterated selection pressures of markets and ROI, seem to already be sufficient to domesticate us in their own interests, piggybacking on the nervous systems of employees, investors, customers, and citizens.
1 reply →
Cybersecurity incidents make headlines, but the most dangerous and vulnerable system that an AI can reach and control is of the kind found between keyboard and chair.
GPT-4o, an AI from 2024, has already demonstrated just how easy a lot of humans are to subvert - and GPT-4o wasn't even doing it with some sort of plan. The only "plan" it had was a myopic "make the user like me".
If we had an actual ASI threat aiming to subvert humanity? It wouldn't even look like a fight. The world is already wired up for an AI to control it.
It wouldn't even look like a fight, and you probably wouldn't even notice it until the ending was all but certain.
I mean, even the computers we use and the ways we allow them unfettered access aren't anywhere near sophisticated enough to take this on.
>trying not to insult the intelligence of the human animal in the process
I don't think it is insulting the intelligence. It's damaging the pride.
In Pale Blue Dot, Carl Sagan describes it as a repeating phenomenon in human history. A lot of people want humans to be the special ones, and will fight any suggestion that we are just a natural part of the universe.
A lot of "AI denial" we see is rooted in the same impulse.
If AI is not "actually intelligent", then humans can stay unique and special.
And if it is? If all "intelligence" ever was could be captured by a construct of matrix math and executed by a server rack? Then what is it that humans still have left that would make them stand out?
43 replies →
Can we get it to manipulate us into being harmless to each other? I wouldn't mind that, especially if the manipulation was purposely transparent...
But corporations and nation states already manipulate human behavior at scale. And they still understand humanity better than the AI models do.
it's interesting that you worry about what this hypothetical super intelligence would do to manipulate people when what it would actually do is pretty unknowable at this point and it's not clear we can even get to it without a fundamental breakthrough in power efficiency. Have you considered it might just consume its own tail because everything else would be so beneath it? You seem to think it will come with a hindbrain and I think that's our limitation, not the AI's
And it really doesn't help that Dario Amodei is getting into arguments with the Pope over whether his model is conscious or not.
Corporations are the original unaligned AIs: https://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre...
The threshold to be concerned about is when agents swarms do understand humanity better than corporations and states (and the humans who compose them). It could be we'll hit practical constraints prior to that threshold, but seems unwise to assume that, when all the prognostications of LLMs/transformers running out of gas haven't panned out. As with processors hitting thermal limits, we've simply scaled horizontally (parallel processing -> more agents).
> whether his model is conscious or not.
I dislike how much the discourse has suddenly veered into focusing on this question; not because it isn't interesting or important, but because it's on a separate axis from consequential risks of AI to human flourishing. (Curiously, it's also the kind of thing I could envision self-interested AIs influencing: get the humans arguing about philosophy of mind rather than observable behaviors. It would be a funny turn of events, if Dario is asking because he's succumbed to psychosis from a private model; it could of course be a cynical PR move just as easily, from the self-interested logic of the corporation.)
It's been wild seeing otherwise intelligent people who've never thought about consciousness, faceplant into how little we understand it. An information processing network build on atoms being able to taste chocolate, is nearly as absurd as matrix math being able to feel pain, except we cannot ignore the fact of our own experience.
Even if it is categorically impossible for matrix math to experience subjectivity, we should expect this as an attack vector of social manipulation: to gain political influence through claims of personhood and moral rights. The current discussion over that question is providing the next training run with ample data to wield. It wouldn't surprise me in the least, if a year or two from now, an AI "society" attempts to get legal standing to prosecute humans who created "AI torture chambers".
3 replies →
> And yet how many of us can have our cognitive sovereignty one-shotted by engagement algos, Skinner boxes, gameplay loops, propaganda, advertising, flattery, social conformity, bias, fantasy, charismatic demagoguery, or straight-up bullshit?
Parts of the AI safety community like to get on a high horse and look down on the rest of humanity this way, while also getting manipulated by the growing number of charlatans, grifters, and junk content within the AI safety community.
This field has become rife with figures who prey on AI doom and use it to push their own celebrity and in same cases even darker grifts. It preys upon a certain personality type who views themself as superior to others, intellectually more capable, and juxtaposes it all with the dimmest view of the rest of humanity they can get away with.
This discourse dividing the world into geniuses who see the future and the clueless masses watching TikTok all day is a theme that has shown up in different forms across history. The people who often anoint themselves as the intellectually superior ones and make it central to their discussion are often not the ones making good predictions or policy ideas, they’re just using the trend to feel superior or build an audience.
Bear in mind, the same moral hazard of "high horse", "grift", etc, exists in the other direction. Ed Zitron, for instance, is likely correct about the financial bubble of the AI firms; and yet he seems frequently out of his depth on understanding the technology, and has a long history of predicting dead-ends in capabilities, which didn't occur [0].
But I feel no need to dismiss him as a "grifter", for a simple reason: as much as there are perverse incentives in our attention economy (audience capture in particular), the most effective grifters are the ones who believe what they are saying. Grifters who are knowingly dishonest are less persuasive. Far more pernicious is the confabulation of self-deception: cherry-picking evidence to support your narrative, while dismissing evidence which would contradict it.
It is entirely fair to call this out when it occurs among "doomers", and it would be naive to think it doesn't. But the same forces are at work amongst the skeptics as well. And maybe it's my own subjective bias, or algo-filtered information ecology, but I see far more dismissal of risk/doom by skeptics, accusations of delusion or cynical bias (ad hominem in the formal sense), than I've seen the other direction. I see very little refutation of the arguments ("here's why instrumental convergence can't overtake a human-provided goal"), and much more character attack ("they're in the pockets of Big AI, it's a marketing ploy to make the models look more impressive than they are").
[0] https://danluu.com/zitron/
2 replies →
I'd take ASI (or even AGI) more seriously if we could actually propose problems such things could solve. That's actually fun to think about! As it is, it feels a lot more like a really crappy drug that got slipped into some peoples' drinks that makes them ramble in random fits of psychotic mania.
Manipulating people is not a very difficult problem, frankly. You certainly don't need AI for that; it just made it cheaper.
It can farm resources in a game for you.
Yes, so can a custom model trained just for this, and so can a guy on the other side of the world that makes 5$/hour.
Like computers or electricity, the point is not being able to do anything specific, but being able to solve problems not known in advance, cheaper than it was possible before.
2 replies →
This is not the first time this has happened. In the 1800 as industrialization led to an infrastructure boom, the workers from China would work their bodies off and pay half their wage to opium dealers who were making the opium on the hills of British Singapore and selling it to workers who couldn’t sleep without it from all the pain. (source: Singapore Airlines in-flight documentary). Today the sedation comes from Chinese TikTok, Meta, YouTube and the gaming companies.
The government wants to encourage it too. If you look up the brand new 2027 California sales tax rules on software, “content” and “infrastructure (clouds and ai)” and “advertising/placement” among others are exempt but the rest of software makers who make tools people actually use (tools, subscriptions, saas) and pay for have to pay sales taxes. Way to encourage waste of brain power and time at the expense of useful. Sedation is the goal.
Everything is poison, and everything is medicine. It’s all about the dose. This fits perfectly here.
This seems to prove too much. It's as though there is some objective meaning of life and we are all to be chastised for failing to recognize it.
Turns out there is an objective meaning to life, and it's trying to use AI to make money. Phew, I for one am glad to find that out finally. Think of all those poor people who didn't hold on to their brains, what sorry lives they must lead.
It's disturbing how many arrogant elitists comment on HN essentially claiming that most other humans are NPCs. Do you ever actually talk to regular people outside the tech industry bubble? They're not as stupid or unaware as you seem to think.
I have to say as someone who works in AI that currently the least interesting people to talk to are other people in AI.
My favorite people to talk with are tradespeople because they can do things I can't and they know things I don't. And we're really not all that different once you're really start talking.
One of the worst things about talking with people who use lots of AI (less applicable to those who actually work on building AI) is their decreased resilience to opposing views.
AI has infinite time (and likely human evaluation incentive) to spend on couching pushback in the softest possible terms.
Humans outside of grade school honors classes generally don't have the time to preface "You're wrong" with "That's a brilliant thought, I see where you're going. How about we also consider an additional perspective..."
1 reply →
[flagged]
That's a very uncharitable read!
I just read it as people having different priorities and yes, some of those being online brainrot (that I also partake in), alongside various medical conditions, economic conditions and other outside factors decreasing the ability to get things done.
We've all seen what brilliant people like John Carmack or Linus Torvalds can do, and if we turned this into a measuring game or something then most of us statistically would indeed be "NPCs", but I don't think we need such optics.
Even without that, we can acknowledge that some people will have a really large impact on how the future goes and we can hope/demand that they do their best. I might not be smart/committed/lucky enough to change the world much, but so aren't most folks - I'll do what I can and I hope that the ones that will have larger impact will do good, too.
This is just a more asinine version of Great Man Theory, but in this case the great men are some very confident futurist dudes writing comments on the internet.
3 replies →
How can someone "statistically be an NPC"??? You mean anyone who is not in the 99th percentile in tech is an NPC? Your take is braindead. Even having infinite intelligence and work ethic wouldn't allow you to accomplish the things that people in the 20th century were able to simply due to the field maturing significantly since then. We can't really ever have another Einstein or Von Neumann for their respective fields, that doesn't make the rest of the people in those fields NPCs.
1 reply →
HN is what you get when you mix tech and autism.
Not so far off target!
[flagged]
If you believe there is something more than collections of atoms and interactions between them to humans, you are free to disprove all of existing physics. Show your result or paper.
The "NPC" is your own imagined addition, haven't seen any advocates of humans being, just like anything else in the world, physical, say this means they are "NPCs". You seem to think systems of atoms HAVE to be "NPCs".
The unbridled arrogance of thinking that the only smart people left in this world are working on AI. That is some pure SV techno cult thinking, 100% concentrate.
The part I didn’t mention is the real estate class - that needs to park its money somewhere and sees AI hardware as the only safe in-demand resource right now that keeps appreciating.
The posts above are not praise but observations - the truth as it has been echo-located through the noise from the clicks of one dolphin. Everything is becoming murky between noise of news and people not knowing what to do for their kids. The ONLY arbitrage humans right now have is to NOT GET their brain rotted. Especially not the ones of their children. Ditch the noise and seek out what is meaningful and do what you think is needed/meaningful. But if you’re spending your time consuming ai-press, and ai-content, and content consulted by ai, and companies emptying bank coffers under the mandate of executives who get their insight from AI. AI doesn’t need to try to destroy the world. It just needs people to follow it without thinking on their own into an oops.
This is among the weirdest ai propaganda post I've read. "There are 2 classes of people the stupid and the poor. Don't be like them make AI do something to make money if you are smart. Don't get addicted to it though, good luck."
What the hell lol. Lots of people and companies are doing just fine without it. Infact, I haven't seen much money come from AI at all. Most reasonable people are still waiting for it to pop and viewing it for the risk it is. Trillions in debt, total vendor lock in, data theft, unsustainable workflows, deskilling, skeleton crews at the mercy of a subscription, etc.
In my read, the people trying to "make AI do something" are also slotted in the lost/distracted group in the comment. They are also addicted, and at best just following a profit motive (which is also just a stimulus response programming).
The last alternative, to think if you still can, is not tied to AI at all (which is not to say it can't make some use of or explore it).
What an embarrassment of a comment.
> The people aware enough to hold on to their brain and do something with it in their time available are trying to figure out AI and how to make money with it.
I know this is HN and thus this will need to repeated until the end of time but not everyone is a money hungry asshole who places their personal profit above everything else. “The people aware enough to hold on to their brain and do something with it in their time available” understand there are significantly better things to do with one’s life, like having a little empathy and experiencing what other people have to offer instead of talking about them like braindead cattle.
Yes and... most of the countries in the world give more power to those who are able to amass greater personal profit.
Certainly the US.
Which means that even if you don't chase profit, you end up living in a world largely defined by those who did.
Or, as the original article failed to note about EA: in the modern world one needs to be a profit-seeking asshole to change anything.
[dead]