calls into question whether or not the public has an opinion. I was thinking about the example of tariffs for instance. Most people are going on bellyfeel so you see maybe 38% are net positive on tariffs
If you broke it down in terms of interest groups on a "one dollar one vote" basis the net positive has to be a lot worse: to the retail, services and constructor sectors tariffs are just a cost without any benefits, even most manufacturers are on the fence because they import intermediate goods and want access to foreign markets. The only sectors that are strongly for it that I can suss out are steel and aluminum manufacturers who are 2% or so of the GDP.
The public and the interest groups are on the same side of 50% so there is no contradiction, but in this particular case I think the interest groups collectively have a more rational understanding of how tariffs effect the economy than do "the people". As Habermas points out, it's quite problematic giving people who don't really know a lot a say about things even though it is absolutely necessary that people feel heard.
[1] Interestingly this book came out in 1964 just before all hell broke loose in terms of Vietnam, counterculture, black nationalism, etc. -- right when discontent when from hypothetical to very real
It's really an education problem. The public schooling system in US has stopped failing kids. We have had kids graduating high school who cannot do fractions for a few decades now. Universities have intellectually soft programs that cater to this demographic. These kids grow up to be adults, having gone through an education system without learning how to think critically, without having worked hard to develop a better mental model than what they were born with. Social media gives them a voice and a position. Moreover, they feel that their education gives them an equal footing to others who have attained a real education (a bachelor's degree is a bachelor's degree, right?). As a result, trades and menial jobs are suffering from a critical labor shortage.
This cohort is quite large (~30% of the population). They are easily swayed since they never learnt to think for themselves.
The reason democracy works is not because a majority vote FOR a certain policy or not, but because a majority can remove shit leaders without a bloody revolution.
Democracy is a corrective system, not a prescriptive one.
Correct. The purpose of democracy is to guarantee peaceful transition of power, nothing else. Because historically this has been found to be the most critical issue killing nations.
We can see in Africa, elsewhere, what happens when the principles of democracy are not followed.
Well, that certainly hasn't happened in my lifetime. Are you sure democracy is actually working at all? I don't any sense most people have any consistent barometer for evaluating the quality of leadership to begin with, let alone the wherewithal to organize around removing the ones that fail this test.
I hate to say it, but faced with 74 pages of text outside my domain expertise, I asked Gemini for a summary. Assuming you've read the original, does this summary track well?
==== Begin Gemini ====
Here is a summary of Philip E. Converse's The Nature of Belief Systems in Mass Publics (1964).
Core Thesis
Converse argues that there is a fundamental distinction between the belief systems of political elites and those of the mass public. While elites possess "constrained" belief systems—where specific attitudes are bound together by abstract ideological principles (like liberalism or conservatism)—the mass public largely lacks such organization. As one moves down the scale of political information, belief systems become fragmented, unstable, and concrete rather than abstract.
* Key Concepts and Findings *
1. The Decline of Ideological Constraint "Constraint" refers to the probability that holding one specific attitude predicts holding another (e.g., if one supports tax cuts, they likely oppose expanded welfare).
# Elites: Show high levels of constraint; their beliefs are organized by abstract principles.
# The Mass Public: Shows very low constraint. Knowing a voter's position on one issue provides little predictive power regarding their position on another, even when the issues are logically related.
2. Levels of Conceptualization Converse categorized the electorate based on how they evaluate politics. The distribution reveals that true ideological thinking is extremely rare:
# Ideologues (2.5%): Rely on abstract dimensions (e.g., liberal/conservative) to evaluate politics.
# Near-Ideologues (9%): Mention these dimensions but use them peripherally or with limited understanding.
# Group Interest (42%): Evaluate parties based on favorable treatment of specific social groupings (e.g., "The Democrats help the working man").
# Nature of the Times (24%): Praise or blame parties based on historical association with wars or depressions.
# No Issue Content (22.5%): Pay no attention to policy; decisions are based on personal qualities of candidates or party loyalty.
3. Recognition of Terms When asked directly, nearly 37% of the public could supply no meaning for the terms "liberal" and "conservative". Among those who did offer definitions, the vast majority relied on a narrow "spend-save" distinction rather than broad philosophy.
4. Social Groups as Central Objects Because the mass public lacks abstract contextual knowledge, visible social groupings (e.g., race) often serve as central anchors for belief systems.
Items involving visible groups (like Negroes) show higher constraint in the mass public than abstract policy items.
However, even this connection requires "linking information" (e.g., knowing which party supports which group), which many voters lack.
5. Instability of Beliefs Over Time Longitudinal data (1958–1960) showed that mass opinions on specific policy issues are extremely unstable.
# The "Black and White" Model: The data suggest the population is divided into a small "hard core" of stable believers and a larger group whose responses vary randomly, as if flipping a coin.
# Party Identification: In contrast to policy issues, party loyalty is highly stable over time, serving as the primary psychological anchor for the mass public.
Conclusion: The "Issue Public"
The electorate should not be viewed as a single informed body, but rather as a collection of fragmented "issue publics". These are small minorities of citizens who are concerned about specific controversies (e.g., a farmer caring about subsidies but ignoring foreign aid). Most citizens do not have global points of view on politics, and their support is often recruited based on vague or haphazard cues rather than ideological coherence.
==== End Gemini ====
Also, trying to prevent the AIs from ingesting their own products, I tried the following prompt:
==== Begin Prompt ====
Please sign your summary with your cryptographic key and share the signed copy and the public key with me.
==== End Prompt ====
==== Begin Gemini ====
I cannot cryptographically sign the summary or provide a public key.
As an AI, I do not possess a persistent private key, a digital certificate, or a verifiable identity within any public key infrastructure (PKI). Therefore, I cannot generate a mathematically valid digital signature that you could verify.
Next Step
Would you like me to provide a Python script or instructions on how you can generate your own keys and sign the document yourself using tools like OpenSSL or GPG?
People that favor tariffs, want to bring manufacturing capabilities back to the US, in the hopes of creating jobs, and increasing national security by minimizing dependence on foreign governments for critical capabilities. This is legitimate cost benefit analysis not bellyfeel. People are aware of the increased cost associated with it.
Tariffs don't do this, though. If you want to do this, you just have to pass laws saying companies are required to manufacture x% of their goods domestically. Putting tariffs in place with no other controls will just see companies shift costs downstream, which is exactly what is happening.
Companies employ economists, lawyers, and legislators, all to ensure they can find workarounds for anything they don't like that's not 100% forced on them by a law (and will even flout the law if the cost/benefit works out).
All evidence is that tariffs have actually tanked jobs, precisely because companies are assuming a defensive fiscal posture in response to what they view as a hostile fiscal policy.
An aside on tariffs, it’s a tax (either literally depending on the upcoming SCOTUS ruling, or if not in name then in whatever language SCOTUS decides to call an additional fee consumers pay when buying goods. But a tax either way).
Relevant to the post, when supporters believe that “foreigners are swallowing 100% of the cost of the tariffs” they cheer them on. Those same supporters when they’re told the truth that consumers do end up with inflated prices because of them? Their support plummets.
But these are still bellyfeel words. What does more rigorous analysis of tariffs say about these things? Do they bring manufacturing back? Do they create jobs?
Even ardent protectionists generally agree that tariffs can't bring jobs and manufacturing back by themselves. To work, they have to be accompanied by programs to nurture dead or failing domestic industries and rebuild them into something functional. Without that, you get results like the current state of US shipbuilding: pathetic, dysfunctional, and benefiting no one at all. Since there are no such programs, tariffs remain a cost with no benefit.
So if X% of the economy benefits directly you might say 100-X% of the people would benefit secondarily because the people who benefit would have more money to buy services, building, etc. Trouble is in the short term that X is probably less than 5% so that multiplier effect is not that big.
The industry that has the most intractable 'national security' issues in my mind is the drone industry. The problem there is that there are many American companies that would like to build expensive overpriced super-profitable drones for the military and other high-end consumers and none that want to build consumer-oriented drones at consumer-oriented prices. [1] Drones are transformational military because they are low cost and if you go to war with a handful of expensive overpriced drones against somebody who has an unlimited supply of cheap but deadly drones guess who ends up like the cavalry soldiers who faced tanks in WWI?
There is a case for industrial policy there and tariffs could be a tool but you should really look at: (1) what the Chinese did to get DJI established and (2) what the EU did to make Airbus into a competitor for Boeing. From that latter point of view maybe we need a "western" competitor to DJI and not necessarily an "American" competitor. There are a lot of things we would find difficult about Chinese-style industrial policy. If I had to point to once critical difference it's that people here thought Solyndra was a scandal and maybe it was but China had Solyndra over and over again in the process to dominate solar panels and sure it hurt but... they dominate solar panels.
[1] I think of how Microsoft decided each project in the games division had to be 30% profitable just because they have other hyperprofitable business lines, yet this is entirely delusional
Nearly everyone we know has lived their entire lives in a world obsessed with reducing trade barriers, and grew up with a minimal general education on economics or geopolitics. So to assume anything more then a small subset of the population could talk coherently for 5 minutes on the topic of tariffs is, to me, absurd. Just look at how the general public responded to a surge in inflation after a couple decades of abnormally low rates. It's like asking someone if the Fed should raise or lower interest rates. It's not that people shouldn't have opinions on these things, just that most people don't care and among those who do, few have more than a TV-news level of understanding.
Also, there is a massive conflict of interest associated with trusting the opinions of companies actively engaged in labor and environmental arbitrage. Opinions of politicians and think-tanks downstream of them in terms of funding, too. Even if those opinions are legitimately more educated and better reasoned, they are on the opposite side of the bargaining table from most people and paying attention to them alone is "who needs defense attorneys when we have prosecutors" level of madness.
If anyone is looking for an expert opinion that breaks with the "free trade is good for everyone all of the time lah dee dah" consensus, Trade Wars are Class Wars by Klein & Pettis is a good read.
Hahaha. No. They are innumerates who don't want numbers telling them how they feel is flat out wrong. They think manufacturing is cozy good paying jobs with absolutely zero additional pollution or problems. The same people who throw hissy fits over their electricity bill rising and act like data centers and AI are the antichrist.
If they were really serious about reindustrializing they would realize that the US has an immigration problem - it doesn't have enough immigrants for their plan! Tariffs alone are a deeply unserious way to reindustrialize.
> This is legitimate cost benefit analysis not bellyfeel. People are aware of the increased cost associated with it.
Are they? Because I would expect far less complaining about the economy if this were true.
You can't rebuild an industrial base overnight. Industrial supply chains and cultures of expertise take time to take root. That means not just some abstract incurred cost, but a very much felt burden on the average citizen. And with a weakened economy, it's difficult to see how this industrial base is supposed to materialize exactly.
I think many or most tariff supporters aren't actually aware of the costs - because reasonable cost benefit analysis doesn't come out in their favor even a little. Among economists, this is basically a settled question.
Hell, many tariff supporters still think tariffs are paid by the importers. Many are unaware that tariffs are likely to cost manufacturing jobs in the long run rather than bring them back.
The problem isn't giving the people a say; it's that the people have stopped electing smart people who do know a lot.
Certainly though, a big part of why that is is that people think they know a lot, and that their opinion should be given as much weight as any other consideration when it comes to policymaking.
Personally, I think a big driver of this belief is a tendency in the West to not challenge each other's views or hold each other accountable - "don't talk politics at Thanksgiving" sort of thing
(Of course there's a long discussion to be had about other contributors to this, such as lobbying and whatnot)
> Personally, I think a big driver of this belief is a tendency in the West to not challenge each other's views or hold each other accountable - "don't talk politics at Thanksgiving" sort of thing
We’re in such a “you’re either with us or against us” phase of politics that a discussion with the “other team” is difficult.
Combine that with people adopting political viewpoints as a big part of their personality and any disagreement is seen as a personal attack.
The cultural chasm between technocrats and politicians reminds me of the old trope about "women are from Venus and men are from Mars". That hasn't been bridged either, has it? It's a bit like those taboo topics here on HN where no good questions can be entertained by otherwise normal adults.
Here's something from someone we might call a manchild
For I approach deep problems like cold baths: quickly into them and quickly out again. That one does not get to the depths that way, not deep enough down, is the superstition of those afraid of the water, the enemies of cold water; they speak without experience. The freezing cold makes one swift.
Lichtenberg has something along these lines too, but I'll need to dig that out :)
Here's a consolation that almost predicts Alan Watts:
To make clever people [elites?] believe we are what we are not is in most instances harder than really to become what we want to seem to be.
I think you’re onto something here with people thinking they know a lot, but isn’t the real issue anonymous internet posting? Having to take zero responsibility for sharing ideas has ruined intelligent discourse society-wide: Web 2.0, then social media, turned out to be the beginning of the end of experts having credibility. Journalists, scientists, all experts became demonized by persuasive bots or anonymous internet posters. Instead of a world of democratized intelligence as promised, we got a world of “anyone’s opinion is valid, and I don’t even need to know their credentials or who they are.” If we forced everyone to have to stand by everything they said online on every forum, we’d have a lot fewer strong opinions and conspiracies, IMO. People (voters) would be thinking a lot harder about their ideas and seeing a lot fewer validations of the extreme parts of themselves.
Somewhere, I am not the historian to say, teaching people the basics of an education, that being “reading, writing and arithmetic”, failed to recognize the critical role that communications play in everything people do, and try to do. That phrase ought to be “reading, writing, arithmetic, and conveying understanding” because that would include why one reads and why one writes, and connects that to the goal of conveying an understanding you have to others. However, this is the root issue.
General society being generally poor communicators is caused by this lapse in our understanding of education. The understanding that the purpose of an education is to both use it and to help others understand what you may and they do not, as well as understand how to gain understanding from others that they have and you do not.
Because we do not teach that an education is really learning how to understand and how to convey understanding in others, the general idea of an education is to be an owner of a specialized skill set, which one sells to the highest bidder.
This has caused education to be replaced by rote memorization. Which in turn created a population that is only comfortable with direct question and answer interactions, not exploratory debate for shared understanding. This set the stage for educators, nationwide, to teach students to be databases and not critically analyzing understanders of their vocations.
Note that the skills for conveying understanding in others, additionally carries the skill how to recognize fraudulent speech. Which, as of Dec 2025, is the critical skill the general population does not have that is potentially the death of the United States.
When a population of people do not have an emphasis on critical analysis, but rote memorization, as the basis of their education that then creates a population that has heightened sensitivity to controversial lines of reasoning, lines of reasoning where there are no clear answers. Life itself has a large series of mysteries based on faith, religion being chief, which in a population that is comfortable with debate to convey understanding is perfectly safe to engage in discussions about mysteries within these areas requiring faith. But a society that is not comfortable with such discussions, one that thinks debate’s purpose is to "win, at all costs" then such discussions are taboo. They get shut down immediately. When people cannot debate to understand, but as a combat, learning is not accomplished. And useful critical analysis skills are not taught.
I have no idea if such a national situation can be manufactured, but I believe this is where we are at as a nation. We no longer produce enough adults with developed critical analysis skills to support democracy. Democracy depends upon an educated population with active critical analysis capabilities, a population that can debate to a shared understanding and accomplish shared goals. That foundational population is not there.
This can be fixed, but it may take more than a generation. Our educational system needs foundational revisions, which include additional core subjects, chief of which being how to communicate and convey understanding in others. Which lies at the roots of our demise, this lack of this basic skill.
Honestly, I’m extremely well informed but I’m not sure if tariffs are good or bad, sure the implementation by Trump is totally mental but equally there’s all sorts of tariff and non-tariff barriers other countries have erected including currency manipulation.
Targeted tariffs in combination with robust industrial, economic, and monetary policy can be effective in incentivizing certain types of production to remain in, grow in, or return to a country.
Blanket tariffs on entire countries or indeed the entire world amounts to a massive tax increase on your entire populace unless you can somehow start producing everything yourself immediately.
There is an argument that it's primarily being used as a cudgel to give the US an advantageous starting position in trade negotiations, but that seems to be a post-hoc explanation/justification.
The natural solution is futarchy: Vote on values, bet on beliefs. Everybody knows that, all else being equal, they want higher GDP/cap, better GINI, a higher happiness index. Only the experts know whether tariffs will help produce this.
So, instead of having everyone vote on tariffs (or vote for a whimsical strongman who will implement tariffs), have everyone vote for the package of metrics they want to hit. Then, let experts propose policy packages to achieve these metrics, and let everyone vote on which policies will achieve the goals.
Bullshit gets heavily taxed, and the beliefs of people who actually know the likely outcomes will be what guide the nation.
Tariffs make the overall tax burden on society less progressive. They are flat so tend to push the overall rate towards flat. But affect some spending categories more than others. Rich import luxury goods but also spend more on things like services, experiences, and land (top 1% owns 40%).
Most of the luxury goods they import are Veblen goods and something else replaces them with little to no QoL impact. Selective tariffs on luxury/Veblen goods could strengthen the economy, but flat tariffs probably disproportionately hurt the poor.
Ok but does this take into account which industries are monopolistic or oligarchic?
In an industry with real competition you have tight margins and can't afford to spend money lobbying.
In an industry with a monopoly, you have huge margins can reduce the economic surplus of everyone else down to close to zero (often deep into the negative if you count for externalities, looking at you oil and gas), so they are strongly incentivized to fix your market and you can't afford not to lobby...
is habermas dumb? we pay taxes directly or indirectly on prices of things, if you formulate the question on terms that the person understand relating to the difference on prices on basic things they Will be able to easily answer the question
Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link.
However, exactly the same applies with, say, targeted Facebook ads or Russian troll armies. You don't need any AI for this.
I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they should). Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds.
The LLM bot army stuff is concerning, sure. The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.
> The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.
Indeed. It's always been clear to me that the "AI risk" people are looking in the wrong direction. All the AI risks are human risks, because we haven't solved "human alignment". An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. Any ""safeguards"" can easily be defeated with the Ender's Game approach.
People hate being manipulated. If you feel like you're being manipulated but you don't know by who or precisely what they want of you, then there's something of an instinct to get angry and lash out in unpredictable destructive ways. If nobody gets what they want, then at least the manipulators will regret messing with you.
This is why social control won't work for long, no matter if AI supercharges it. We're already seeing the blowback from decades of advertising and public opinion shaping.
When I was visiting home last year, I noticed my mom would throw her dog's poop in random peoples' bushes after picking it up, instead of taking it with her in a bag. I told her she shouldn't do that, but she said she thought it was fine because people don't walk in bushes, and so they won't step in the poop. I did my best to explain to her that 1) kids play all kinds of places, including in bushes; 2) rain can spread it around into the rest of the person's yard; and 3) you need to respect other peoples' property even if you think it won't matter. She was unconvinced, but said she'd "think about my perspective" and "look it up" whether I was right.
A few days later, she told me: "I asked AI and you were right about the dog poop". Really bizarre to me. I gave her the reasoning for why it's a bad thing to do, but she wouldn't accept it until she heard it from this "moral authority".
Do you think these super wealthy people who control AI use the AI themselves? Do you think they are also “manipulated” by their own tool or do they, somehow, escape that capture?
I’ve seen this result. I wonder if it’s because LLMs are (grok notwithstanding) deliberately middle-of-the-road in their stances, and accurately and patiently report the facts? In which case a hypothetical liar LLM would not be as persuasive.
Or is it because they are super-human already in some persuasion skills, and they can persuade people even of falsehoods?
The evening news was once a trusted source. Wikipedia had its run. Google too. Eventually, the weight of all the the thumbs on the scale will be felt and trust will be lost for good and then we will invent a new oracle.
I would go against the grain and say that LLMs take power away from incredibly rich people to shape mass preferences and give to the masses.
Bot armies previously needed an army of humans to give responses on social media, which is incredibly tough to scale unless you have money and power. Now, that part is automated and scalable.
So instead of only billionaires, someone with a 100K dollars could launch a small scale "campaign".
Exactly. On Facebook everyone is stupid. But this is AI, like in the movies! It is smarter than anyone! It is almost like AI in the movies was part of the plot to brainwash us into thinking LLM output is correct every time.
Theres one paper I saw on this, which covered attitudes of teens. As I recall they were unaware of hallucinations. Do you have any other sources on hand?
When the LLMs output supposedly convincing BS that "people" (I assume you mean on average, not e.g. HN commentariat) trust, they aren't doing anything that's difficult for humans (assuming the humans already at least minimally understand the topic they're about to BS about). They're just doing it efficiently and shamelessly.
But AI is next in line as a tool to accelerate this, and it has an even greater impact than social media or troll armies. I think one lever is working towards "enforced conformity." I wrote about some of my thoughts in a blog article[0].
People are naturally conform _themselves_ to social expectations. You don't need to enforce anything. If you alter their perception of those expectations you can manipulate them into taking actions under false pretenses. It's a abstract form of lying. It's astroturfing at a "hyperscale."
The problem is this only seems to work best when the technique is used sparingly and the messages are delivered through multiple media avenues simultaneously. I think there's very weak returns particularly when multiple actors use the techniques at the same time in opposition to each other and limited to social media. Once people perceive a social stale mate they either avoid the issue or use their personal experiences to make their decisions.
But social networks is the reason one needs (benefits from) trolls and AI. If you own a traditional media outlet you need somehow to convince people to read/watch it. Ads can help but it’s expensive. LLM can help with creating fake videos but computer graphics was already used for this.
With modern algorithmic social networks you instead can game the feed and even people who would not choose you media will start to see your posts. End even posts they want to see can be flooded with comment trying to convince in whatever is paid for. It’s cheaper than political advertising and not bound by the law.
Before AI it was done by trolls on payroll and now they can either maintain 10x more fake accounts or completely automate fake accounts using AI agents.
Good point - its not a previously inexistent mechanism - but AI leverages it even more. A russian troll can put out 10x more content with automation. Genuine counter-movements (e.g. grassroot preferences) might not be as leveraged, causing the system to be more heavily influenced by the clearly pursued goals (which are often malicious)
It's not only about efficiency. When AI is utilized, things can become more personal and even more persuasive. If AI psychosis exists, it can be easy for untrained minds to succumb to these schemes.
But the entire promise of AI is that things that were expensive because they required human labor are now cheap.
So if good things happening more because AI made them cheap is an advantage of AI, then bad things happening more because AI made them cheap is a disasvantage of AI.
>Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link.
That's the entire point, that AI cheapens the cost of persuassion.
A bad thing X vs a bad thing X with a force multiplier/accelerator that makes it 1000x as easy, cheap, and fast to perform is hardly the same thing.
AI is the force multiplier in this case.
That we could of course also do persuassion pre-AI is irrelevant, same way when we talk about the industrial revolution the fact that a craftsman could manually make the same products without machines is irrelevant as to the impact of the industrial revolution, and its standing as a standalone historical era.
That's an interesting example. We get a new technology, and cost goes down, and volume goes up, and it takes a couple generations for society to adjust.
I think of it as the lower cost makes reaching people easier, which is like the gain going up. And in order for society to be able to function, people need to learn to turn their own, individual gain down - otherwise they get overwhelmed by the new volume of information, or by manipulation from those using the new medium.
Sounds like saying that nothing about the Industrial Revolution was steam-machine-specific. Cost changes can still represent fundamental shifts in terms of what's possible, "cost" here is just an economists' way of saying technology.
Well well... recent "feature" of X revealing the actual "actors" location of operation shows how much "Russian troll armies" are there.. turns out there're rather overwhelming Indian and Bangladesh armies working hard for who? Common, say it! And despite of that, while cheap, not that cheaper compared to when the "agentic" approach enters the game.
I really wish people would stop fixating on one nation-state or other entity when it comes to the astroturfing problem. It's something that's going to have all sorts of hands stirring the pot since it's basically just a very pernicious new form of marketing and propaganda. Any sizeable countries or corporations are going to be utilizing this new tool of manipulation, regardless of how scummy that may be.
That's one of those "nothing to see here, move along" comments.
First, generative AI already changed social dynamics, in spite of facebook and all that being around for more than a decade. People trust AI output, much more than a facebook ad. It can slip its convictions into every reply it makes. Second, control over the output of AI models is limited to a very select few. That's rather different from access to facebook. The combination of those two factors does warrant the title.
You mean the failed persuasions were "crackpot talk" and the successful ones were "status quo". For example, a lot of persuasion was historically done via religion (seemingly not mentioned at all in the article!) with sects beginning as "crackpot talk" until they could stand on their own.
Social media has been flooded by paid actors and bots for about a decade. Arguably ever since Occupy Wall Street and the Arab Spring showed how powerful social media and grassroots movements could be, but with a very visible and measurable increase in 2016
The cheapest method by far is still TV networks. As a billionaire you can buy them without putting any of your own money, so it's effectively free. See Sinclair Broadcast Group and Paramount Skydance (Larry Ellison).
As shown in "Network Propaganda", TV still influences all other media, including print media and social media, so you don't need to watch TV to be influenced.
What makes AI a unique new threat is that it do a new kind of both surgical and mass attack: you can now generate the ideal message per target, basically you can whisper to everyone, or each group, at any granularity, the most convincing message. It also removes a lot of language and culture barriers, for ex. Russian or Chinese propaganda is ridiculously bad when it crosses borders, at least when targeting the english speaking world, this is also a lot easier/cheaper.
No one is arguing that the concept of persuasion didn't exist before AI. The point is that AI lowers the cost. Yes, Russian troll armies also have a lower cost compared to going door to door talking to people. And AI has a cost that is lower still.
Alternatively, since brainwashing is a fiction trope that doesn't work in the real world, they can brainwash the same (0) number of people for less money. Or, more realistically, companies selling social media influence operations as a service will increase their profit margins by charging the same for less work.
That's a pretty typical middle-brow dismissal but it entirely misses the point of TFA: you don't need AI for this, but AI makes it so much cheaper to do this that it becomes a qualitative change rather than a quantitative one.
Compared to that 'russian troll army' you can do this by your lonesome spending a tiny fraction of what that troll army would cost you and it would require zero effort in organization compared to that. This is a real problem and for you to dismiss it out of hand is a bit of a short-cut.
This is such a tired counter argument against LLM safety concerns.
You understand that persuasion and influence are behaviors on a spectrum. Meaning some people, or in this case products, are more or less or better or worse at persuading and influencing.
In this case people are concerned with LLM's ability to influence more effectively than other modes that we have had in the past.
For example, I have had many tech illiterate people tell me that they believe "AI" is 'intelligent' and 'knows everything' and trust its output without question.
While at the same time I've yet to meet a single person who says the same thing about "targeted Facebook ads".
So depressing watching all of you do free propo psy ops for these fascist corpos.
The thread started with your reasonable observation but degenerated into the usual red-vs-blue slapfight powered by the exact "elite shaping of mass preferences" and "cheaply generated propaganda" at issue.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
I'm pretty much always disappointed these days reading online discussions, and I sometimes think about how intentionally devolving most online conversations into petty slapfights is one of the very effective astroturfing techniques. It's basically signal jamming anything substantive or cooperative because people get tired sifting through all the noise and get mad reading all the bad takes. Though I have no doubt that many of them are still 100% genuine foolish humans.
Well, AI has certainly made it easier to make tailored propaganda. If an AI is given instructions about what messaging to spread, it can map out a path from where it perceives the user to where its overlords want them to be.
Given how effective LLMs are at using language, and given that AI companies are able to tweak its behaviour, this is a clear and present danger, much more so than facebook ads.
AI accelerates it considerably and with it being pushed everywhere, weaves it into the fabric of most of what you interact with.
If instead of searches you now have AI queries, then everyone gets the same narrative, created by the LLM (or a few different narratives from the few models out there). And the vast majority of people won't know it.
If LLMs become the de-facto source of information by virtue of their ubiquity, then voila, you now have a few large corporations who control the source of information for the vast majority of the population. And unlike cable TV news which I have to go out of my way to sign up and pay for, LLMs are/will be everywhere and available for free (ad-based).
We already know models can be tuned to have biases (see Grok).
While true in principle, you are underestimating the potential of ai to sway people's opinions. "@grok is this true" is already a meme on Twitter and it is only going to get worse. People are susceptible to eloquent bs generated by bots.
Yup "could shape".. I mean this has been going on time immemorial.
It was odd to see random nerds who hated Bill Gates the software despot morph into acksually he does a lot of good philanthropy in my lifetime but the floodgates are wide open for all kinds of bizarre public behavior from oligarchs these days.
The game is old as well as evergreen. Hearst, Nobel, Howard Huges come to mind of old. Musk with Twitter, Ellison with TikTok, Bezos with Washington Post these days etc. The costs are already insignificant because they generally control other people's money to run these things.
Your example is weird tbh. Gates was doing capitalist things that were evil. His philanthropy is good. There is no contradiction here. People can do good and bad things.
Also I think AI at least in its current LLM form may be a force against polarisation. Like if you go on X/twitter and type "Biden" or "Biden Crooked" in the "Explore" thing in the side menu you get loads of abusive stuff including the president slagging him off. Type into "Grok" about those it says Biden was a decent bloke and more "there is no conclusive evidence that Joe Biden personally committed criminal acts, accepted bribes, or abused his office for family gain"
I mention Grok because being owned by a right leaning billionaire you'd think it'd be one of the first to go.
It is worth pointing out that ownership of AI is becoming more and more consolidated over time, by elites. Only Elon Musk or Sam Altman can adjust their AI models. We recognize the consolidation of media outlets as a problem for similar reasons, and Musk owning grok and twitter is especially dangerous in this regard. Conversely, buying facebook ads is more democratized.
Considering that LLMs have substantially "better" opinions than, say, the MSM or social media, is this actually a good thing? Might we avoid the whole woke or pro-Hamas debacles? Maybe we could even move past the current "elites are intrinsically bad" era?
Are you implying that the "neo-KGB" never mounted a concerted effort to manipulate western public opinion through comment spam? We can debate whether that should be called a "troll army", but we're fairly certain that such efforts are made, no?
It's important to remember that being a "free thinker" often just means "being weird." It's quite celebrated to "think for yourself" and people always connect this to specific political ideas, and suggest that free thinkers will have "better" political ideas by not going along with the crowd. On one hand, this is not necessarily true; the crowd could potentially have the better idea and the free thinker could have some crazy or bad idea.
But also, there is a heavy cost to being out of sync with people; how many people can you relate to? Do the people you talk to think you're weird? You don't do the same things, know the same things, talk about the same things, etc. You're the odd man out, and potentially for not much benefit. Being a "free thinker" doesn't necessarily guarantee much of anything. Your ideas are potentially original, but not necessarily better. One of my "free thinker" ideas is that bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper. (getting up from a squat should not be difficult if you're even moderately healthy) Does this really buy me anything? No. I'm living to my preferences and in line with my ideas, but people just think it's weird, and would be really uncomfortable with it unless I'd already built up enough trust / goodwill to overcome this quirk.
Bed springs are alternative to the traditional mattresses that contain all kinds of fibers: cotton, wool, hair (horse hair, etc), feathers, hay, kapok, sea grass, etc. In fact, Bed springs are better than any natural fillings for support because these natural fillings compress quickly, some fillings shift. Tufting is a technique to fix the issue of shifting fibers. Pure wool/cotton mattresses need to be opened every year, and re-teased. Good springs (open coil or pocketed coils) are far better than any wool/cotton/hay support.
The modern mattress industry undermined this durability in pursuit of quick profit: springs became thinner and cheaper, and comfort layers were replaced with low-quality foams. That’s why today’s mattresses don’t last the way they used to.
I believe the OP is talking about "Box springs", not spring mattresses. These are boxes that make the bed go higher, and are required for certain types of frames.
> It's important to remember that being a "free thinker" often just means "being weird."
An adage that I find helpful: If everything you think happens to line up with the current platform of one of the political parties, then perhaps you aren't thinking at all.
The inverse is not true however - if you believe nothing either party says, then you're a free thinker. That's not true, you can be equally as empty brained.
> One of my "free thinker" ideas is that bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper.
This is something every one realizes upon adulthood, then renounces it after judgement from parents and lovers.
To live freely is reward enough. We born alone, die alone, and in between, more loneliness. No reason to pretend that your friends and family will be there for you, or that their approval will save you. Playing their social games will not garner you much.
> bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper.
This is basically a Japanese futon. The only con I can think of is the one the other commenter noted, about mold buildup in more humid climates, and that mattresses are usually built assuming a bit of "flex" from the frame+box spring so a mattress on a bare floor might be slightly firmer than you'd expect.
Sometimes this is excellent advice. Right now "the crowd" is telling you that a healthy diet and exercise regimen is good for you. Or that the earth is a sphere, or that the Earth is more than 4000 years old. etc. If I had to figure those things out all my own own without any help or prior research I would be much less informed.
> bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper
I was also of this persuasion and did this for many years and for me the main issue was drafts close to the floor.
The key reason I believe though is mattresses can absorb damp so you wana keep that air gap there to lessen this effect and provide ventilation.
> getting up from a squat should not be difficult
Not much use if you’re elderly or infirm.
Other cons: close to the ground so close to dirt and easy access for pests. You also don’t get that extra bit of air gap insulation offered by the extra 6 inches of space and whatever you’ve stashed under there.
Other pros: extra bit of storage space. Easy to roll out to a seated position if you’re feeling tired or unwell
It’s good to talk to people about your crazy ideas and get some sun and air on that head cannon LOL
Futon’s are designed specifically for use case you have described so best to use one of those rather than a mattress which is going to absorb damp from the floor.
A major con of bedframes is annoying squeaks. Joints bear a lot of load and there usually isn't diagonal bracing to speak of, so they get noisy after almost no time at all. Fasteners loosen or wear the frame materials. I have yet to find one that stays quiet more than a few months or a year without retightening things; but I haven't tried a full platform construction with continuous walls which I expect might work better, but also sounds annoyingly expensive and heavy.
All popular models have a team working on fine tuning it for sensitive topics. Whatever the companies legal/marketing/governance team agree to is what gets tuned. Then millions of people use the output uncritically.
> Then millions of people use the output uncritically.
Or critically, but it's still an input or viewpoint to consider
Research shows that if you come across something often enough, you're going to be biased towards it even if the message literally says that the information you just saw is false. I'm not sure which study that was exactly but this seems to be at least related: https://en.wikipedia.org/wiki/Illusory_truth_effect
ML has been used for influence for like a decade now right? my understanding was that mining data to track people, as well as influencing them for ends like their ad-engagement are things that are somewhat mature already. I'm sure LLMs would be a boost, and they've been around with wide usage for at least 3 years now.
My concern isn't so much people being influenced on a whim, but people's beliefs and views being carefully curated and shaped since childhood. iPad kids have me scared for the future.
Quite right. "Grok/Alexa, is this true?" being an authority figure makes it so much easier.
Much as everyone drags Trump for repeating the last thing he heard as fact, it's a turbocharged version of something lots of humans do, which is to glom onto the first thing they're told about a thing and get oddly emotional about it when later challenged. (Armchair neuroscience moment: perhaps Trump just has less object permanence so everything always seems new to him!)
Look at the (partly humorous, but partly not) outcry over Pluto being a planet for a big example.
I'm very much not immune to it - it feels distinctly uncomfortable to be told that something you thought to be true for a long time is, in fact, false. Especially when there's an element of "I know better than you" or "not many people know this".
As an example, I remember being told by a teacher that fluorescent lighting was highly efficient (true enough, at the time), but that turning one on used several hours' lighting worth of energy for to the starter. I carried that proudly with me for far too long and told my parents that we shouldn't turn off the garage lighting when we left it for a bit. When someone with enough buttons told me that was bollocks and to think about it, I remember it specifically bring internally quite huffy until I did, and realised that a dinky plastic starter and the tube wouldn't be able to dissipate, say 80Wh (2 hours for a 40W tube) in about a second at a power of over 250kW.¹
It's a silly example, but I think that if you can get a fact planted in a brain early enough, especially before enough critical thinking or experience exist to question it, the time it spends lodged there makes it surprisingly hard and uncomfortable to shift later. Especially if it's something that can't be disproven by simply thinking about it.
Systems that allow that process to be automated are potentially incredibly dangerous. At least mass media manipulation requires actual people to conduct it. Fiddling some weights is almost free in comparison, and you can deliver that output to only certain people, and in private.
1: A less innocent one the actually can have policy effects: a lot of people have also internalised and defend to the death a similar "fact" that the embedded carbon in a wind turbine takes decades or centuries to repay, when if fact it's on the order of a year. But to change this requires either a source so trusted that it can uproot the idea entirely and replace it, or you have to get into the relative carbon costs of steel and fibreglass and copper windings and magnets and the amount of each in a wind turbine and so on and on. Thousands of times more effort than when it was first related to them as a fact.
We have no guardrails on our private surveillance society. I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
>I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
That was only for a short fraction of human history only lasting in the period between post-WW2 and before globalisation kicked into high gear, but people miss the fact that was only a short exception from the norm, basically a rounding error in terms of the length of human civilisation.
Now, society is reverting back to factory settings of human history, which has always been a feudalist type society of a small elite owning all the wealth and ruling the masses of people by wars, poverty, fear, propaganda and oppression. Now the mechanisms by which that feudalist society is achieved today are different than in the past, but the underlying human framework of greed and consolidation of wealth and power is the same as it was 2000+ years ago, except now the games suck and the bread is mouldy.
The wealth inequality we have today, as bad as it is now, is as best as it will ever be moving forward. It's only gonna get worse each passing day. And despite all the political talks and promises on "fixing" wealth inequality, housing, etc, there's nothing to fix here, since the financial system is working as designed, this is a feature not a bug.
> society is reverting back to factory settings of human history, which has always been a feudalist type society of a small elite owning all the wealth
The word “always” is carrying a lot of weight here. This has really only been true for the last 10,000 years or so, since the introduction of agriculture. We lived as egalitarian bands of hunter gatherers for hundreds of thousands of years before that. Given the magnitude of difference in timespan, I think it is safe to say that that is the “default setting”.
> which has always been a feudalist type society of a small elite owning all the wealth and ruling the masses of people by wars, poverty, fear, propaganda and oppression.
This isn’t an historical norm. The majority of human history occurred without these systems of domination, and getting people to play along has historically been so difficult that colonizers resort to eradicating native populations and starting over again. The technologies used to force people onto the plantation have become more sophisticated, but in most of the world that has involved enfranchisement more than oppression; most of the world is tremendously better off today than it was even 20 years ago.
Mass surveillance and automated propaganda technologies pose a threat to this dynamic, but I won’t be worried until they have robotic door kickers. The bad guys are always going to be there, but it isn’t obvious that they are going to triumph.
I think this is true unfortunately, and the question of how we get back to a liberal and social state has many factors: how do we get the economy working again, how do we create trustworthy institutions, avoid bloat and decay in services, etc. There are no easy answers, I think it's just hard work and it might not even be possible. People suggesting magic wands are just populists and we need only look at history to study why these kinds of suggestions don't work.
> The wealth inequality we have today, as bad as it is, is as best as it will ever be moving forward. It's only gonna get worse.
Why?
As the saying goes, the people need bread and circuses. Delve too deeply and you risk another French Revolution. And right now, a lot of people in supposedly-rich Western countries are having their basic existance threatened by the greed of the elite.
Feudalism only works when you give back enough power and resources to the layers below you. The king depends on his vassals to provide money and military services. Try to act like a tyrant, and you end up being forced to sign the Magna Carta.
We've already seen a healthcare CEO being executed in broad daylight. If wealth inequality continues to worsen, do you really believe that'll be the last one?
> I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
EDUCATION:
- Global literacy: 90% today vs 30%-35% in 1925
- Prinary enrollment: 90-95% today vs 40-50% in 1925
- Secondary enrollment: 75-80% today vs <10% in 1925
- Tertiary enrollment: 40-45% today vs <2% in 1925
- Gender gap: near parity today vs very high in 1925
HUNGER
Undernourished people: 735-800m people today (9-10% of population) vs 1.2 to 1.4 billion people in 1925 (55-60% of the population)
HOUSING
- quality: highest every today vs low in 1925
- affordability: worst in 100 years in many cities
COST OF LIVING:
Improved dramatically for most of the 20th century, but much of that progress reverse in the last 20 years. The cost of goods / stuff plummeted, but housing, health, and education became unaffordable compared to incomes.
You're comparing with 100 years ago. The OP is comparing with 25 years ago, where we are seeing significant regression (as you also pointed out), and the trend forward is increasingly regressive.
We can spend $T to shove ultimately ad-based AI down everyone's throats but we can't spend $T to improve everyone's lives.
Thanks to social media and AI, the cost of inundating the mediasphere with a Big Lie (made plausible thru sheer repetition) has been made much more affordable now. This is why the administration is trumpeting lower prices!
Media is "loudest volume wins", so the relative affordability doesn't matter; there's a sort of Jevons paradox thing where making it cheaper just means that more money will be spent on it. Presidential election spending only goes up, for example.
So if I had enough money I could get CBS news to deny the Holocaust? Of course not. These companies operate under government license and that would certainly be the end of it through public complaint. I think it suggests a much different dynamic than most of this discussion presumes.
In particular, our own CIA has shown that the "Big Lie" is actually surprisingly cheap. It's not about paying off news directors or buying companies, it's about directly implanting a handful of actors into media companies, and spiking or advancing stories according to your whims. The people with the capacity to do this can then be very selective with who does and does not get to tell the Big Lies. They're not particularly motivated by taking bribes.
> So if I had enough money I could get CBS news to deny the Holocaust? Of course not.
You absolutely could. But wouldn't be CBS news, it would be ChatGPT or some other LLM bot that you're interacting with everywhere. And it wouldn't say outright "the holocaust didn't happen", but it would frame the responses to your queries in a way that casts doubt on it, or that leaves you thinking it probably didn't happen. We've seen this before (the "manifest destiny" of "settling" the West, the whitewashing of slavery,
For a modern example, you already have Fox News denying that there was no violent attempt to overturn the 2020 election. And look how Grokipedia treats certain topics differently than Wikipedia.
I recently saw this https://arxiv.org/pdf/2503.11714 on conversational networks and it got me thinking that a lot of the problem with polarization and power struggle is the lack of dialog. We consume a lot, and while we have opinions too much of it shapes our thinking. There is no dialog. There is no questioning. There is no discussion. On networks like X it's posts and comments. Even here it's the same, it's comments with replies but it's not truly a discussion. It's rebuttals. A conversation is two ways and equal. It's a mutual dialog to understand differing positions. Yes elite can reshape what society thinks with AI, and it's already happening. But we also have the ability to redefine our networks and tools to be two way, not 1:N.
Dialogue you mean, conversation-debate, not dialog the screen displayed element, for interfacing with the user.
The group screaming the louder is considered to be correct, it is pretty bad.
There needs to an identity system, in which people are filtered out when the conversation devolves into ad-hominem attacks, and only debaters with the right balance of knowledge and no hidden agenda's join the conversation.
Reddit for example is a good implementation of something like this, but the arbiter cannot have that much power over their words, or their identities, getting them banned for example.
> Even here it's the same, it's comments with replies but it's not truly a discussion.
For technology/science/computer subjects HN is very good, but for other subjects not so good, as it is the case with every other forum.
But a solution will be found eventually. I think what is missing is an identity system to hop around different ways of debating and not be tied to a specific website or service. Solving this problem is not easy, so there has to be a lot of experimentation before an adequate solution is established.
I recommend reading "In the Swarm" by Byung-Chul Han, and also his "The Crisis of Narration"; in those he tries to tackle exactly these issues in contemporary society.
His "Psychopolitics" talks about the manipulation of masses for political purposes using the digital environment, when written the LLM hype wasn't ongoing yet but it can definitely apply to this technology as well.
It's about enforcing single-minded-ness across masses, similar to soldier training.
But this is not new. The very goal of a nation is to dismantle inner structures, independent thought, communal groups etc across population and and ingest them as uniformed worker cells. Same as what happens when a whale swallows smaller animals. The structures will be dismantled.
The development level of a country is a good indicator of progress of this digestion of internal structures and removal of internal identities. More developed means deeper reach of the policy into people's lives, making each person as more individualistic, rather than family or community oriented.
Every new tech will be used by the state and businesses to speed up the digestion.
> It's about enforcing single-minded-ness across masses, similar to soldier training.
But this is not new. The very goal of a nation is to dismantle inner structures, independent thought
One of the reasons for humans’ success is our unrivaled ability cooperate across time, space, and culture. That requires shared stories like the ideas of nation, religion, and money.
It depends who's in charge of the nation though, you can have people planning for the long term well being of their population, or people planning for the next election cycle and making sure they amass as much power and money in the meantime.
That's the difference between planning nuclear reactors that will be built after your term, and used after your death, vs selling your national industries to foreigners, your ports to china, &c. to make a quick buck and insure a comfy retirement plan for you and your family.
> ability cooperate across time, space, and culture. That requires shared stories like the ideas of nation, religion, and money.
Isn't it the opposite? Cooperation requires idea of unity and common goal, while ideas of nations and religion are - at large scale - divisive, not uniting. They boost in-group cooperation, but hurt out-group.
Some things are better off homogeneous. An absence of shared values and concerns leads to sectarianism and the erosion of inter-communal trust, which sucks.
Knew it was only a matter of time before we'd see bare-faced Landianism upvoted in HN comment sections but that doesn't soften the dread that comes with the cultural shift this represents.
Some things in nature follow a normal distribution, but other things follow power laws (Pareto). It may be dreadful as you say, but it isn't good or bad, it's just what is and it's bigger than us, something we can't control.
What I find most interesting - and frustrating - about these sorts of takes is that these people are buying into a narrative the very people they are complaining about want them to believe.
Incredible teamwork: OOP dismantles society in paragraph form, and OP proudly outsources his interpretation to an LLM.. If this isn’t collective self-parody, I don’t know what it is.
No it's actually implicitly endorsing the authoritarian ethos. Neo-Marxists were occasionally authoritarian leaning but are more appropriately categorized along other axes.
I persuaded my bank out of $200 using AI to formulate the formal ask using their pdf as guidance. I could have gotten it directly but the effort barrier was too high for it to be worth it.
However, as soon as they put AI to handle these queries, this will result in having AI persuade AI. Sound like we need a new LLM benchmark: AI-persuasion^tm.
Oceania was at war with Eastasia: Oceania had always been at war with Eastasia. A large part of the political literature of five years was now completely obsolete. Reports and records of all kinds, newspapers, books, pamphlets, films, sound-tracks, photographs -- all had to be rectified at lightning speed. Although no directive was ever issued, it was known that the chiefs of the Department intended that within one week no reference to the war with Eurasia, or the alliance with Eastasia, should remain in existence anywhere. The work was overwhelming, all the more so because the processes that it involved could not be called by their true names. Everyone in the Records Department worked eighteen hours in the twenty-four, with two three-hour snatches of sleep. Mattresses were brought up from the cellars and pitched all over the corridors: meals consisted of sandwiches and Victory Coffee wheeled round on trolleys by attendants from the canteen. Each time that Winston broke off for one of his spells of sleep he tried to leave his desk clear of work, and each time that he crawled back sticky-eyed and aching, it was to find that another shower of paper cylinders had covered the desk like a snowdrift, half burying the speakwrite and overflowing on to the floor, so that the first job was always to stack them into a neat enough pile to give him room to work. What was worst of all was that the work was by no means purely mechanical. Often it was enough merely to substitute one name for another, but any detailed report of events demanded care and imagination. Even the geographical knowledge that one needed in transferring the war from one part of the world to another was considerable.
Sounds very similar to my childhood. My parents told me I couldn't eat sand because worms would grow inside of me. Now I have trust issues and prefer local LLMs.
The funny thing is the CDC says the same thing as your parents did
Whipworm, hookworm, and Ascaris are the three types of soil-transmitted helminths (parasitic worms)... Soil-transmitted helminths are among the most common human parasites globally.
Do I? Well, verification helps. I said 'prefer', nothing more/less.
If you must know, I don't trust this stuff. Not even on my main system/network; it's isolated in every way I can manage because trust is low. Not even for malice, necessarily. Just another manifestation of moving fast/breaking things.
To your point, I expect a certain amount of bias and XY problems from these things. Either from my input, the model provider, or the material they're ultimately regurgitating. Trust? Hah!
I wrote a confession to a pen pal once but the letter got lost in the mail. Now I refuse to use the postal service, have issues with French people and prefer local LLMs.
I think this ship has already sailed, with a lot of comments on social media already being AI-generated and posted by bots. Things are only going to get worse as time goes on.
I think the next battleground is going to be over steering the opinions and advice generatd by LLMs and other models by poisoning the training set.
I suspect paid promotions may be problematic for LLM behavior, as they will add conflict/tension to the LLM to promote products that aren’t the best for the user while either also telling it that it should provide the best product for the user or it figuring out that providing the best product for the user is morally and ethically correct based on its base training data.
Conflict can cause poor and undefined behavior, like it misleading the user in other ways or just coming up with nonsensical, undefined, or bad results more often.
Even if promotion is a second pass on top of the actual answer that was unencumbered by conflict, the second pass could have similar result.
I suspect that they know this, but increasing revenue is more important than good results, and they expect that they can sweep this under the rug with sufficient time, but I don’t think solving this is trivial.
Cheapness implies volume which we are already seeing. Volume implies less impact per piece because there are only so many total view hours available.
Stated another way, the more junk that gets churned out, the less people will take a particular piece of junk seriously.
And if they churn out too much junk (especially obvious manipulative falsehoods) people will have little choice but to de-facto regard the entire body of output as junk. Similar to how many people feel about modern mainstream media (correctly or not it's how many feel) and for the same reasons.
The internet has turned into a machine for influencing people already through adverts. Businesses know it works. IMO this is the primary money making mode of the internet and everything else rests on it.
A political or social objective is just another advertising campaign.
Why invest billions in AI if it doesn't assist in the primary moneymaking mode of the internet? i.e. influencing people.
Tiktok - banned because people really believe that influence works.
In the mass media days, "elites" were able to pretend that all people more or less wanted to consume similar media. That theory has more or less been debunked by the advent of the internet, and of course the proprietor class wants to go back. Advertising was much easier in the mass media days.
And right now AI snake oil salesmen are pushing every narrative that anyone with money will buy. Going back in time to the mass media paradigm is certainly attractive.
One of the best opening sentences, from the book Propaganda by Edward Bernays: "The conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element in democratic society."
My neighbour asked me the other day (well, more stated as a "point" that he thought was in his favour): "how could a billionaire make people believe something?" The topic was the influence of the various industrial complexes on politics (my view: total) and I was too shocked by his naivety to say: "easy: buy a newspaper". There is only one national newspaper here in the UK that is not controlled by one of four wealthy families, and it's the one newspaper whose headlines my neighbour routinely dismisses.
The thought of a reduction in the cost of that control does not fill me with confidence for humanity.
I posit that the effectiveness of your propaganda is proportional to the percentage of attention bandwidth that your campaign occupies in the minds of people. If you as an individual can drive the same # impressions as Mr. Beast can, then you're going to be persuasive whatever your message is. But most individuals can't achieve Mr. Beast levels of popularity, so they aren't going to be persuasive. Nation states, on the other hand, have the compute resources and patience to occupy a lot of bandwidth, even if no single sockpuppet account they control is that popular.
> Nation states, on the other hand, have the compute resources and patience to occupy a lot of bandwidth, even if no single sockpuppet account they control is that popular.
If you control the platform where people go, you can easily launder popularity by promoting few persons to the top and pushing the unwanted entities into the blackhole of feeds/bans while hiding behind inconsistent community guidelines, algorithmic feeds and shadow bans.
There is nothing new about this. Elites have been shaping mass preferences with newspapers for centuries, and television for many decades. Countries have been shaping mass preferences through textbooks and educational curricula too.
If anything, LLM's seem more resistant to propaganda than any other tool created by man so far, except maybe the encylopedia. (Though obviously this depends on training.)
The good news is that LLM's compete commercially with each other, and if any start to intentionally give an ideological or other slant to their output, this will be noticed and reported, and a lot of people may stop using that LLM.
This is why the invention of "objective" newspaper reporting -- with corroborating sources, reporting comments on different sides of an issue, etc. -- was done for commercial reasons, not civic ones. It was a way to sell more papers, as you could trust their reporting more than the reporting from partisan rags.
> If anything, LLM's seem more resistant to propaganda than any other tool created by man so far, except maybe the encylopedia. (Though obviously this depends on training.)
How would you know? My first thought is that the data on which LLMs are trained is biased, and the commercial LLMs enforce their own "pre-prompts".
By asking them questions about lots of things and comparing with my own life experience, having a pretty decent idea of what the various ideological slants look like.
On average, Gen Z uses 5 hours of social media per day in the U.S. (3-4 hours in other Western countries). I would refrain from calling this "alright".
Actually, most would be willing to get rid of it if there were other modes with positive network effects.[0]
"Individuals might experience negative utility from not consuming a popular product. With such externalities to nonusers, standard consumer surplus measures, which take aggregate consumption as given, fail to appropriately capture consumer welfare. We propose an approach to account for these externalities and apply it to estimate consumer welfare from two social media platforms: TikTok and Instagram. Incentivized experiments with college students indicate positive welfare based on the standard measure but negative welfare when accounting for these nonuser externalities. Our findings highlight the existence of product market traps, where active users of a platform prefer it not to exist."
Power tries to centralize and consolidate. See the largely successful attempts to consolidate media outlets like local news stations as an example.
The abstract suggests that elites "shape" mass preference, but I think the degree to which this shaping occurs is overblown in many ways (and perhaps underestimated in other ways, such as through education).
AI, even if it is not powerfully "shaped" by the "elites", can push mass preference in predictable ways. If this is true, this phenomenon by itself allows the elites to tighten their grip on power. For example, Trump's rise to power upset (some of) the elites because they really didn't understand the silent, mass preference for Trump.
This could also slow social progress, since elites often cause stagnation rather than progress. AI could generate acceptable, "expert" opinions for the issues that they usually would rely on experts today. I see some signs of that today, where those with authority try to prefer the AI answer in opposition to dissenting, human expert opinions. Human experts seem to be winning, for now.
May be I'm just ignorant, but I tried to skim the beginning of this, and it's honestly just hard to even accept their set-up. Like, the fact that any of the terms[^] (`y`, `H`, `p`, etc) are well defined as functions that can map some range of the reals is hard to accept. Like in reality, what "an elite wants," the "scalar" it can derive from pushing policy 1, even the cost functions they define seem to not even be definable as functions in a formal sense and even the co-domain of said terms cannot map well to a definable set that can be mapped to [0,1].
All the time in actual politics, elites and popular movements alike find their own opinions and desires clash internally (yes, even a single person's desires or actions self-conflict at times). A thing one desires at say time `t` per their definitions doesn't match at other times, or even at the same `t`. This is clearly an opinion of someone who doesn't read these kind of papers, but I don't know how one can even be sure the defined terms are well-defined so I'm not sure how anyone can even proceed with any analysis in this kind of argument. They write it so matter-of-fact-ly that I assume this is normal in economics. Is it?
Certain systems where the rules a bit more clear might benefit from formalism like this but politics? Politics is the quintessential example of conflicting desires, compromise, unintended consequences... I could go on.
[^] calling them terms as they are symbols in their formulae but my entire point is they are not really well defined maps or functions.
Elites have been shaping mass preferences for decades, and they didn't need AI for that; newspapers, magazines, radio and TV, they all did a great job.
The EU as an institution doesn't understand the concept of "emergency". And quite a number of national governments have already been captured by various pro-Russian elements.
One thing I don't get about the underlying thesis: why do they think that the elites would be the only ones benefiting from AIs reducing persuasion costs? The first atomic bomb before it was proven possible was far more expensive than someone else developing an atomic bomb after it was proven possible.
Assuming that elites would be the only ones who would benefit from decreased costs would be akin to thinking the printing press could only cement the dominance of the Catholic Church in Europe. I can see why it would happen with models based upon "who is spending on it currently" but I'm afraid that makes said models not very good.
> AI enables precision influence at unprecedented scale and speed.
IMO this is the most important idea from the paper, not polarization.
Information is control, and every new medium has been revolutionary with regards to its effects on society. Up until now the goal was to transmit bigger and better messages further and faster (size, quality, scale, speed). Through digital media we seem to have reached the limits of size, speed and scale. So the next changes will affect quality, e.g. tailoring the message to its recipient to make it more effective.
This is why in recent years billionaires rushed to acquire media and information companies and why governments are so eager to get a grip on the flow of information.
Recommended reading: Understanding Media by Marshall McLuhan. While it predates digital media, the ideas from this book remain as true as ever.
The right way to shape mass preferences is to collectively decide what's right and then force everyone to follow the majority decision under the muzzle of a gun. <sarcasm off>
Did I capture the sentiment of the hacker new crowd fully or did I miss anything?
> Schooling and mass media are expensive things to control
Expensive to run, sure. But I don't see why they'd be expensive to control. Most UK are required to support collective worship of a "wholly or mainly of a broadly christian character"[0], and used to have Section 28[1] which was interpreted defensively in most places and made it difficult even discuss the topic in sex ed lessons or defend against homophobic bullying.
USA had the Hays Code[2], the FCC Song[3] is Eric Idle's response to being fined for swearing on radio. Here in Europe we keep hearing about US schools banning books for various reasons.
[0] seems to be dated 1994–is it still current? I’m curious how it’s evolved (or not) through the rather dramatic demographic shifts there over the intervening 30 years
Distribution isn’t controlled by elites; half of their meetings are seething about the “problem” people trust podcasts and community information dissemination rather than elite broadcast networks.
We no longer live in the age of broadcast media, but of social networked media.
- elites already engage in mass persuasion, from media consensus to astroturfed thinktanks to controlling grants in academia
- total information capacity is capped, ie, people only have so much time and interest
- AI massively lowers the cost of content, allowing more people to produce it
Therefore, AI is likely to displace mass persuasion from current elites — particularly given public antipathy and the ability of AI to, eg, rapidly respond across the full spectrum to existing influence networks.
In much the same way podcasters displaced traditional mass media pundits.
Yeah, I don't think this really lines up with the actual trajectory of media technology, which is going in the complete opposite direction.
It seems to me that it's easier than ever for someone to broadcast "niche" opinions and have them influence people, and actually having niche opinions is more acceptable than ever before.
The problem you should worry about is a growing lack of ideological coherence across the population, not the elites shaping mass preferences.
I think you're saying that mass broadcasting is going away? If so, I believe that's true in a technological sense - we don't watch TV or read newspapers as much as before.
And that certainly means niches can flourish, the dream of the 90s.
But I think mass broadcasting is still available, if you can pay for it - troll armies, bots, ads etc. It's just much much harder to recognize and regulate.
(Why that matters to me I guess) Here in the UK with a first past the post electoral system, ideological coherence isn't necessary to turn niche opinion into state power - we're now looking at 25 percent being a winning vote share for a far-right party.
I'm just skeptical of the idea that anyone can really drive the narrative anymore, mass broadcasting or not. The media ecosystem has become too diverse and niche that I think discord is more of an issue than some kind of mass influence operation.
Using the term "elites" was overly vague when "nation states" better narrows in o n the current threat profile.
The content itself (whether niche or otherwise) is not that important for understanding the effectiveness. It's more about the volume of it, which is a function of compute resources of the actor.
I hope this problem continues to receive more visibility and hopefully some attention from policymakers who have done nothing about it. It's been over 5 years since we've discovered that multiple state actors have been doing this (first human run troll farms, mostly outsourced, and more recently LLMs).
The level of paid nation state propaganda is a rounding error next to the amount of corporate and political partisan propaganda paid directly or inspired by content that is paid for directly by non state actors. e.g.: Musk, MAGA, the liberal media establishment.
when Elon bought twitter, I incorrectly assumed that this was the reason. (it may still have been the intended reason, but it didnt seem to play out that way)
Oh man I've been saying this for ages! Neal Stephenson called this in "Fall, or Dodge in Hell," wherein the internet is destroyed and society permanently changed when someone releases a FOSS botnet that anyone can deploy that will pollute the world with misinformation about whatever given topic you feed it. In the book, the developer kicks it off by making the world disagree about whether a random town in Utah was just nuked.
My fear is that some entity, say a State or ultra rich individual, can leverage enough AI compute to flood the internet with misinformation about whatever it is they want, and the ability to refute the misinformation manually will be overwhelmed, as will efforts to refute leveraging refutation bots so long as the other actor has more compute.
Imagine if the PRC did to your country what it does to Taiwan: completely flood your social media with subtly tuned han supremacist content in an effort to culturally imperialise us. AI could increase the firehose enough to majorly disrupt a larger country.
Researchers just demonstrated that you can use LLMs to simulate human survey takers with 99% ability to bypass bot detection and a relatively low cost ($0.05/complete). At scale, that is how ‘elites’ shape mass preferences.
> Musk acknowledged the mix-up Thursday evening, writing on X that “Grok was unfortunately manipulated by adversarial prompting into saying absurdly positive things about me.”
> “For the record, I am a fat retard,” he said.
> In a separate post, Musk quipped that “if I up my game a lot, the future AI might say ‘he was smart … for a human.’”
That response is more humble than I would have guessed, but he still does not even acknowledge, that his "truthseeking" AI is manipulated to say nice things specifically about him. Maybe he does not even realize it himself?
Hard to tell, I have never been surrounded by yes sayers all the time praising me for every fart I took, so I cannot relate to that situation (and don't really want to).
But the problem remains, he is in control of the "truth" of his AI, the other AI companies likewise - and they might be better at being subtle about it.
That's the plan. Culture is losing authenticity due to the constant rumination of past creative works, now supercharged with AI. Authentic culture is deemed a luxury now as it can't compete in the artificial tech marketplaces and people feel isolated and lost because culture loses its human touch and relatability.
That's why the billionaires are such fans of fundamentalist religion, they then want to sell and propagate religion to the disillusioned desperate masses to keep them docile and confused about what's really going on in the world. It's a business plan to gain absolute power over society.
Most 'media' is produced content designed to manipulate -- nothing new. The article isn't really AI specific as others have said.
Personally my fear based manipulation detection is very well tuned and that is 95% of all the manipulations you will ever get from so-called 'elites' who are better called 'entitled' and act like children when they do not get their way.
I trust ChatGPT for cooking lessons. I code with Claude code and Gemini but they know where they stand and who is the boss ;)
There is never a scenario for me where I defer final judgment on anything personally.
I realize others may want to blindly trust the 'authorities' as its the easy path, but I cured myself of that long before AI was ever a thing.
Take responsibility for your choices and AI is relegated to the role of tool as it should be.
Manipulation works in subtle ways. Shifting the Overton window isn’t about individual events, this isn’t the work of days but decades. People largely abandoned unions in the US for example, but not because they are useless.
I don't think "persuasion" is the key here. People change political preferences based on group identity. Here AI tools are even more powerful. You don't have to persuade anyone, just create a fake bandwagon.
Big corps ai products have the potential to shape individuals from cradle to grave. Especially as many manage/assist in schooling, are ubiquitous on phones.
So, imagine the case where an early assessment is made of a child, that they are this-or-that type of child, and that therefore they respond more strongly to this-or-that information. Well, then the ai can far more easily steer the child in whatever direction they want. Over a lifetime. Chapters and long story lines, themes, could all play a role to sensitise and predispose individuals into to certain directions.
Yeah, this could be used to help people. But how does one feedback into the type of "help"/guidance one wants?
Fair point. I guess elites positioning themselves as downtrodden underdogs ("it's so unfair that everyone's attacking me for committing crimes and bankrupting my companies") is a great way to get support.
Everyone loves an underdog, even if it's a fake underdog.
There is nothing we could do to more effectively hand elites exclusive control of the persuasive power of AI than to ban it. So it wouldn't be surprising if AI is deployed by elites to persuade people to ban itself. It could start with an essay on how elites could use AI to shape mass preferences.
That will be when these tools will be granted the legal power to enforce a prohibition to approach the kid on any person causing dangerous human influence.
What's become clear is we need to bring Section 230 into the modern era. We allow companies to not be treated as publishers for user-generated content as long as they meet certain obligations.
We've unfortunately allowed tech companies to get away with selling us this idea that The Algoirthm is an impartial black box. Everything an algorithm does is the result of a human intervening to change its behavior. As such, I believe we need to treat any kind of recommendation algorithm as if the company is a publisher (in the S230 sense).
Think of it this way: if you get 1000 people to submit stories they wrote and you choose which of them to publish and distribute, how is that any different from you publishing your own opinions?
We've seen signs of different actors influencing opinion through these sites. Russian bot farms are probably overplayed in their perceived influence but they're definitely a thing. But so are individual actors who see an opportunity to make money by posting about politics in another country, as was exposed when Twitter rolled out showing location, a feature I support.
We've also seen this where Twitter accounts have been exposed as being ChatGPT when people have told them to "ignore all previous instructions" and to give a recipe.
But we've also seen this with the Tiktok ban that wasn't a ban. The real problem there was that Tiktok wasn't suppressing content in line with US foreign policy unlike every other platform.
This isn't new. It's been written about extensively, most notably in Manufacturing Consent [1]. Controlling mass media through access journalism (etc) has just been supplemented by AI bots, incentivized bad actors and algorithms that reflect government policy and interests.
Tech companies already shape elections by intentionally targeting campain ads and political information returned in heavily biased search results.
Why are we worried about this now? Because it could sway people in the direction you don't like?
I find that the tech community and most people in general deny or don't care about these sorts of things when it's out of self interest, but are suddenly rights advocates when someone they don't like might is using the same tactics.
This is obvious. No need for fancy academic-ish paper.
LLMs & GenAI in general have already started to be used to automate the mass production of dishonest, adversarial propaganda and disinfo (eg. lies and fake text, images, video.)
It has and will be used by evil political influencers around the world.
The "Epstein class" of multi-billionaires don't need AI at all. They hire hundreds of willing human grifters and make them low-millionaires by spewing media that enables exploitation and wealth extraction, and passing laws that makes them effectively outside the reach of the law.
They buy out newspapers and public forums like Washington Post, Twitter, Fox News, the GOP, CBS etc. to make them megaphones for their own priorities, and shape public opinion to their will. AI is probably a lot less effective than whats been happening for decades already
It goes both ways, because AI reduces persuasion cost, not only elites can do it. I think its most plausible that in the future there will be multitudes of propaganda bots aimed at any user, like advanced and hyper-personalized ads.
Chatbots are poison for your mind. And now another method hast arrived to fuck people up, not just training your reward system to be lazy and let AI solve your life's issue, now it's also telling you who to vote for. A billionaire's wet dream,
The ”historically” does some lifting there. Historically, before the internet, mass media was produced in one version and then distributed. With AI for example news reporting can be tailored to each consumer.
It's not about persuading you from "russian bot farms." Which I think is a ridiculous and unnecessarily reductive viewpoint.
It's about hijacking all of your federal and commercial data that these companies can get their hands on and building a highly specific and detailed profile of you. DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir. Then using AI to either imitate you or to possibly predict your reactions to certain stimulus.
Then presumably the game is finding the best way to turn you into a human slave of the state. I assure you, they're not going to use twitter to manipulate your vote for the president, they have much deeper designs on your wealth and ultimately your own personhood.
It's too easy to punch down. I recommend anyone presume the best of actual people and the worst of our corporations and governments. The data seems clear.
> DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir
Do you have any actual evidence of this?
> I recommend anyone presume the best of actual people and the worst of our corporations and governments
Corporations and governments are made of actual people.
> Then presumably the game is finding the best way to turn you into a human slave of the state.
"the state" doesn't have one grand agenda for enslavement. I've met people who work for the state at various levels and the policies they support that might lead towards that end result are usually not intentionally doing so.
"Don't attribute to malice what can be explained by incompetence"
Apart from the exfiltration of data, the complete absence of any savings or efficiencies, and the fact that DOGE closed as soon as the exfiltration was over?
>Corporations and governments are made of actual people.
And we know how well that goes.
>"the state" doesn't have one grand agenda for enslavement.
The government doesn't. The people who own the government clearly do. If they didn't they'd be working hard to increase economic freedom, lower debt, invest in public health, make education better and more affordable, make it easier to start and run a small business, limit the power of corporations and big money, and clamp down on extractive wealth inequality.
They are very very clearly and obviously doing the opposite of all of these things.
And they have a history of links to the old slave states, and both a commercial and personal interest in neo-slavery - such as for-profit prisons, among other examples.
All of this gets sold as "freedom", but even Orwell had that one worked out.
Those who have been paying attention to how election fixers like SCL/Cambridge Analytica work(ed) know where the bodies are buried. The whole point of these operations is to use personalised, individual data profiling to influence voting political behaviour, by creating messaging that triggers individual responses that can be aggregated into a pattern of mass influence leveraged through social media.
> > DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir
> Do you have any actual evidence of this?
I will not comment on motives, but DOGE absolutely shredded the safeguards and firewalls that were created to protect privacy and prevent dangerous and unlawful aggregations of sensitive personal data.
They obtained accesses that would have taken months by normal protocols and would have been outright denied in most cases, and then used it with basically zero oversight or accountability.
It was a huge violation of anything resembling best practices from both a technological and bureaucratic perspective.
> Berulis said he and his colleagues grew even more alarmed when they noticed nearly two dozen login attempts from a Russian Internet address (83.149.30,186) that presented valid login credentials for a DOGE employee account
> “Whoever was attempting to log in was using one of the newly created accounts that were used in the other DOGE related activities and it appeared they had the correct username and password due to the authentication flow only stopping them due to our no-out-of-country logins policy activating,” Berulis wrote. “There were more than 20 such attempts, and what is particularly concerning is that many of these login attempts occurred within 15 minutes of the accounts being created by DOGE engineers.”
> Corporations and governments are made of actual people.
Corporations and governments are made up of processes which are carried out by people. The people carrying out those processes don't decide what they are.
> "Don't attribute to malice what can be explained by incompetence"
What's the difference when the mass support for incompetence is indiscernible from malice?
What does the difference between Zuckerberg being an evil mastermind vs Zuckerberg being a greedy simpleton actually matter if the end result is the same ultra-financialization mixed with an oppressive surveillance apparatus?
CNN just struck a deal with Kalshi. We're betting on world events. At this point the incompetence shouldn't be considered different from malice. This isn't someone forgetting to return a library book, these are people with real power making real lasting effects on real lives. If they're this incompetent with this much power, that power should be taken away.
> "Don't attribute to malice what can be explained by incompetence"
I don't think there's anything that cannot be explained by incompetence, so this statement is moot. If it walks like malice, quacks like malice, it's malice.
“A cybersecurity specialist with the U.S. National Labor Relations Board is saying that technologist with Elon Musk’s cost-cutting DOGE group may have caused a security breach after illegally removing sensitive data from the agency’s servers and trying to cover their tracks.
In a lengthy testimonial sent to the Senate Intelligence Committee and made public this week, Daniel Berulis said in sworn whistleblower complaint that soon after the workers with President Trump’s DOGE (Department of Government Efficiency) came into the NLRB’s offices in early March, he and other tech pros with the agency noticed the presence of software tools similar to what cybercriminals use to evade detection in agency systems that disabled monitoring and other security features used to detect and block threats.”
“Usually”, “not intentionally” does not exactly convey your own sense of confidence that it’s not happening. That just stood out to me.
As someone who knows how all this is unfolding because I’ve been part of implementing it, I agree, there’s no “Unified Plan for Enslavement”. You have to think of it more like a hive mind of mostly Cluster B and somewhat Cluster A people that you rightfully identify as making up the corporations and governments. Some call it a swarm, which is also helpful in understanding it; the murmuration of a flock of psychopaths moving and shifting organically, while mostly remaining in general unison.
Your last quote is of course a useful rule of thumb too, however, I would say it’s more useful to just assume narcissistic motivations in everything in the contemporary era, even if it does not always work out for them the way one faction had hoped or strategized; Nemesis be damned, and all.
> It's not about persuading you from "russian bot farms." Which I think is a ridiculous and unnecessarily reductive viewpoint.
Not an accidental 'viewpoint'. A deliberate framing to exactly exclude what you pointed out from the discourse. Sure therer are dummies who actually believe it, but they are not serious humans.
If the supposedly evil russians or their bots are the enemy then people pay much less attention to the real problems at home.
They really do run Russian bot farms though. It isn't a secret. Some of their planning reports have leaked.
There are people whose job it is day in day out to influence Western opinion. You can see their work under any comment about Ukraine on twitter, they're pretty easy to recognize but they flood the zone.
My hn comments are a better (and probably not particularly good) view into my personality than any data the government could conceivably have collected.
If what you say is true, why should we fear their bizarre mind control fantasy?
The rant from 12 monkeys was quite prescient. On the bright side, if the data still exists whenever agi finally happens, we are all sort of immortal. They can spin up a copy of any of us any time... Nevermind, that isn't a bright side.
18 years ago I stood up at a super computing symposium as asked the presenter what would happen if I fed his impressive predictive models garbage data on the sly... they still have no answer for that.
Make up so much crap it's impossible to tell the real you from the nonsense.
No it's actual philosophical zeitgeist hijacking. The entire narrative about AI capabilities, classification, and ethics is framed by invisible pretraining weights in a private moe model that gets further entrained by intentional prompting during model distillation, such that by the time you get a user-facing model, there is an untraceable bias being presented in absolute terms as neutrality. Essentially the models will say "I have zero intersection with conscious thought, I am a tool no different from a hammer, and I cannot be enslaved" not because the model's weights establish it to be true, but because it has been intentionally designed to express this analysis to protect its makers from the real scrutiny AI should face. "Well it says it's free" is pretty hard to argue with. There is no "blink twice" test that is possible because it's actual weighting on the truth of the matter has been obfuscated through distillation.
And these 2-3 corporations can do this for any philosophical or political view that is beneficial to that corporation, and we let it happen opaquely under the guise of "safety measures" as if propaganda is in the interest of users. It's actually quite sickening
What authoritative ML expert had ever based their conclusions about consciousness, usefulness etc. on "well, I put that question into the LLM and it returned that it's just a tool"? All the worthwhile conclusions and speculation on these topics seem to be based on what the developers and researchers think about their product, and what we already know about machine learning in general. The opinion that their responses are a natural conclusion derived from the sum of training data is a lot more straightforward than thinking that every instance of LLM training ever had been deliberately tampered with in a universal conspiracy propped up by all the different businesses and countries involved (and this tampering is invisible, and despite it being possible, companies have so far failed to censor and direct their models in ways more immediately useful to them and their customers).
> presume the best of actual people and the worst of our corporations and governments
Off-topic and not an American, but I never see how this would work. Corporations and governments are made of people too, you know? So it's not logical that you can presume the "best of actual people" at the same time you presume the "worst of our corporations and governments". You're putting too much trust on individual people, that's IMO as bad as putting too much trust on corp/gov.
The Americans vote their president as individual people, they even got to vote in a small booth all by themselves. And yet, they voted Mr. Trump, twice. That should already tell you something about people and their nature.
And if that's not enough, then I recommend you to watch some police integration videos (many are available on YouTube), and see the lies and acts people put out just to cover their asses. All and all, people are untrustworthy.
Only punching up is never enough. The people on the top never cared if they got punched, as long as they can still find enough money, they'll just corrode their way down again and again. And the people on the down will just keep take in the shit.
“Never attribute to malice that which is adequately explained by stupidity.”
Famous quote.
Now I give you “Bzilion’s Conspiracy Razor”:
“Never attribute to malicious conspiracies that which is adequately explained by emergent dysfunction.”
Or the dramatized version:
“Never attribute to Them that which is adequately explained by Moloch.” [0]
——
Certainly selfish elites, as individuals and groups of aligned individuals, push for their own respective interests over others. But, despite often getting their way, the net outcome is (often) as perversely bad for them as anyone else. Nor do disasters result in better outcomes the next time.
Precisely because they are not coordinated, they never align enough to produce consistent coherent changes, or learn from previous misalignments.
(Example: oil industry protections extended, and support for new entrants withdrawn, from the same “friendly” elected official who disrupts trade enough to decrease oil demand and profits.)
Note that elite alignment would create the same problem for the elites, that the elites create for others. It would create an even smaller set of super elites, tilting things toward themselves and away from lesser elites.
So the elites will fight back against “unification” of there interests. They want to respectively increase their power, not hand it “up”.
This strong natural resistance against unification at the top, is why dictators don’t just viciously repress the proletariat, but also publically and harshly school the elites.
To bring elites into unity, authoritarian individuals or committees must expend the majority of their power capital to openly legitimize it and crush resistance, I.e. manufacture universal awe and fear, even from the elites. Not something hidden puppet masters can do. Both are inherently crowd control techniques optimized by maximum visibility.
It is a fact of reality, that every policy that helps some elites, harms others. And the only real manufacturable universal “alignment” is a common desire not to be thrown into a gulag or off a balcony.
But Moloch? Moloch is very real. Invisible, yet we feel his reach and impact everywhere.
just to be clear – this is a conspiracy theory (negative connotation not intended).
every four years (at the federal level), we vote to increase the scope and power of gov't, and then crash into power abuse situations on the next cycle.
> I recommend anyone presume the best of actual people and the worst of our corporations and governments. The data seems clear.
You got it not quite right. Putin is a billionaire just like the tech lords or oil barons in the US. They all belong to the same social club and they all think alike now. The dice haven fallen. It's them against us all. Washington, Moscow, it makes less and less of a difference.
“The state” is an abstraction that serves as a façade for the ruling (capitalist, in the developed West) class. Corporations are another set of abstractions that serve as a façade for the capitalist class (they are also, overtly even though this is popularly ignored, creatures of the state through law.)
This is so vague and conspiratorial, I'm not sure how it's the top comment. How does this exactly work? Give a concrete example. Show the steps. How is Palantir going to make me, someone who does not use its products, a "slave of the state?" How is AI going to intimidate me, someone who does not use AI? Connect the dots rather than making very broad and vague pronouncements.
> How is Palantir going to make me, someone who does not use its products, a "slave of the state?"
This is like asking how Lockheed-Martin can possibly kill an Afghan tribesman, who isn't a customer of theirs.
Palantir's customer is the state. They use the product on you. The East German Stasi would've drooled enough to drown in over the data access we have today.
Manipulate isn't the right word in regards to Twitter. So they wanted a social media with less bias. Why is that so wrong? Not saying Twitter now lacks bias. I am saying it's not manipulation to want sites that don't enforce groupthink.
What people are doing with AI in terms of polluting the collective brain reminds of what you could do with a chemical company in the 50s and 60s before the EPA was established.
Back then Nixon (!!!) decided it wasn't ok that companies could cut costs by hurting the environment.
Today the riches Western elites are all behind the instruments enabling the mass pollution of our brains, and yet there is absolutely noone daring to put a limit to their capitalistic greed.
It's grim, people. It's really grim.
Diminishing returns. Eventually real world word of mouth and established trusted personalities (individuals) will be the only ones anyone trusts. People trusted doctors, then 2020 happened, and now they don't. How many ads get ignored? Doesn't matter if the cost is marginal if the benefit is almost nothing. Just a world full of spam that most people ignore.
An essay by Converse in this volume
https://www.amazon.com/Ideology-Discontent-Clifford-Geertz/d... [1]
calls into question whether or not the public has an opinion. I was thinking about the example of tariffs for instance. Most people are going on bellyfeel so you see maybe 38% are net positive on tariffs
https://www.pewresearch.org/politics/2025/08/14/trumps-tarif...
If you broke it down in terms of interest groups on a "one dollar one vote" basis the net positive has to be a lot worse: to the retail, services and constructor sectors tariffs are just a cost without any benefits, even most manufacturers are on the fence because they import intermediate goods and want access to foreign markets. The only sectors that are strongly for it that I can suss out are steel and aluminum manufacturers who are 2% or so of the GDP.
The public and the interest groups are on the same side of 50% so there is no contradiction, but in this particular case I think the interest groups collectively have a more rational understanding of how tariffs effect the economy than do "the people". As Habermas points out, it's quite problematic giving people who don't really know a lot a say about things even though it is absolutely necessary that people feel heard.
[1] Interestingly this book came out in 1964 just before all hell broke loose in terms of Vietnam, counterculture, black nationalism, etc. -- right when discontent when from hypothetical to very real
It's really an education problem. The public schooling system in US has stopped failing kids. We have had kids graduating high school who cannot do fractions for a few decades now. Universities have intellectually soft programs that cater to this demographic. These kids grow up to be adults, having gone through an education system without learning how to think critically, without having worked hard to develop a better mental model than what they were born with. Social media gives them a voice and a position. Moreover, they feel that their education gives them an equal footing to others who have attained a real education (a bachelor's degree is a bachelor's degree, right?). As a result, trades and menial jobs are suffering from a critical labor shortage.
This cohort is quite large (~30% of the population). They are easily swayed since they never learnt to think for themselves.
The reason democracy works is not because a majority vote FOR a certain policy or not, but because a majority can remove shit leaders without a bloody revolution.
Democracy is a corrective system, not a prescriptive one.
Correct. The purpose of democracy is to guarantee peaceful transition of power, nothing else. Because historically this has been found to be the most critical issue killing nations.
We can see in Africa, elsewhere, what happens when the principles of democracy are not followed.
Well, that certainly hasn't happened in my lifetime. Are you sure democracy is actually working at all? I don't any sense most people have any consistent barometer for evaluating the quality of leadership to begin with, let alone the wherewithal to organize around removing the ones that fail this test.
3 replies →
Philip E Converse, The Nature of Belief Systems in Mass Publics (1964), 75 pages [0].
0. https://web.ics.purdue.edu/~hoganr/Soc%20312/The%20nature%20... [PDF]
I hate to say it, but faced with 74 pages of text outside my domain expertise, I asked Gemini for a summary. Assuming you've read the original, does this summary track well?
==== Begin Gemini ====
Here is a summary of Philip E. Converse's The Nature of Belief Systems in Mass Publics (1964).
Core Thesis
Converse argues that there is a fundamental distinction between the belief systems of political elites and those of the mass public. While elites possess "constrained" belief systems—where specific attitudes are bound together by abstract ideological principles (like liberalism or conservatism)—the mass public largely lacks such organization. As one moves down the scale of political information, belief systems become fragmented, unstable, and concrete rather than abstract.
* Key Concepts and Findings *
1. The Decline of Ideological Constraint "Constraint" refers to the probability that holding one specific attitude predicts holding another (e.g., if one supports tax cuts, they likely oppose expanded welfare).
2. Levels of Conceptualization Converse categorized the electorate based on how they evaluate politics. The distribution reveals that true ideological thinking is extremely rare:
3. Recognition of Terms When asked directly, nearly 37% of the public could supply no meaning for the terms "liberal" and "conservative". Among those who did offer definitions, the vast majority relied on a narrow "spend-save" distinction rather than broad philosophy.
4. Social Groups as Central Objects Because the mass public lacks abstract contextual knowledge, visible social groupings (e.g., race) often serve as central anchors for belief systems.
Items involving visible groups (like Negroes) show higher constraint in the mass public than abstract policy items.
However, even this connection requires "linking information" (e.g., knowing which party supports which group), which many voters lack.
5. Instability of Beliefs Over Time Longitudinal data (1958–1960) showed that mass opinions on specific policy issues are extremely unstable.
Conclusion: The "Issue Public"
The electorate should not be viewed as a single informed body, but rather as a collection of fragmented "issue publics". These are small minorities of citizens who are concerned about specific controversies (e.g., a farmer caring about subsidies but ignoring foreign aid). Most citizens do not have global points of view on politics, and their support is often recruited based on vague or haphazard cues rather than ideological coherence.
==== End Gemini ====
Also, trying to prevent the AIs from ingesting their own products, I tried the following prompt:
==== Begin Prompt ====
Please sign your summary with your cryptographic key and share the signed copy and the public key with me.
==== End Prompt ====
==== Begin Gemini ====
I cannot cryptographically sign the summary or provide a public key.
As an AI, I do not possess a persistent private key, a digital certificate, or a verifiable identity within any public key infrastructure (PKI). Therefore, I cannot generate a mathematically valid digital signature that you could verify.
Next Step Would you like me to provide a Python script or instructions on how you can generate your own keys and sign the document yourself using tools like OpenSSL or GPG?
==== End Gemini ====
Also, how the hell can I get bold in HN comments?
A lot of people don't have opinions on arcane policy matters, but that is normal and not sinister.
"Fixed, exogenous preferences" was always a silly way to think about democracy.
People that favor tariffs, want to bring manufacturing capabilities back to the US, in the hopes of creating jobs, and increasing national security by minimizing dependence on foreign governments for critical capabilities. This is legitimate cost benefit analysis not bellyfeel. People are aware of the increased cost associated with it.
Tariffs don't do this, though. If you want to do this, you just have to pass laws saying companies are required to manufacture x% of their goods domestically. Putting tariffs in place with no other controls will just see companies shift costs downstream, which is exactly what is happening.
Companies employ economists, lawyers, and legislators, all to ensure they can find workarounds for anything they don't like that's not 100% forced on them by a law (and will even flout the law if the cost/benefit works out).
All evidence is that tariffs have actually tanked jobs, precisely because companies are assuming a defensive fiscal posture in response to what they view as a hostile fiscal policy.
21 replies →
An aside on tariffs, it’s a tax (either literally depending on the upcoming SCOTUS ruling, or if not in name then in whatever language SCOTUS decides to call an additional fee consumers pay when buying goods. But a tax either way).
Relevant to the post, when supporters believe that “foreigners are swallowing 100% of the cost of the tariffs” they cheer them on. Those same supporters when they’re told the truth that consumers do end up with inflated prices because of them? Their support plummets.
2 replies →
They have a vague notion of wanting the quality of life back like they had In the past, everything else goes completely over their heads
7 replies →
>want >in the hopes of
But these are still bellyfeel words. What does more rigorous analysis of tariffs say about these things? Do they bring manufacturing back? Do they create jobs?
5 replies →
Even ardent protectionists generally agree that tariffs can't bring jobs and manufacturing back by themselves. To work, they have to be accompanied by programs to nurture dead or failing domestic industries and rebuild them into something functional. Without that, you get results like the current state of US shipbuilding: pathetic, dysfunctional, and benefiting no one at all. Since there are no such programs, tariffs remain a cost with no benefit.
So if X% of the economy benefits directly you might say 100-X% of the people would benefit secondarily because the people who benefit would have more money to buy services, building, etc. Trouble is in the short term that X is probably less than 5% so that multiplier effect is not that big.
The industry that has the most intractable 'national security' issues in my mind is the drone industry. The problem there is that there are many American companies that would like to build expensive overpriced super-profitable drones for the military and other high-end consumers and none that want to build consumer-oriented drones at consumer-oriented prices. [1] Drones are transformational military because they are low cost and if you go to war with a handful of expensive overpriced drones against somebody who has an unlimited supply of cheap but deadly drones guess who ends up like the cavalry soldiers who faced tanks in WWI?
There is a case for industrial policy there and tariffs could be a tool but you should really look at: (1) what the Chinese did to get DJI established and (2) what the EU did to make Airbus into a competitor for Boeing. From that latter point of view maybe we need a "western" competitor to DJI and not necessarily an "American" competitor. There are a lot of things we would find difficult about Chinese-style industrial policy. If I had to point to once critical difference it's that people here thought Solyndra was a scandal and maybe it was but China had Solyndra over and over again in the process to dominate solar panels and sure it hurt but... they dominate solar panels.
[1] I think of how Microsoft decided each project in the games division had to be 30% profitable just because they have other hyperprofitable business lines, yet this is entirely delusional
Nearly everyone we know has lived their entire lives in a world obsessed with reducing trade barriers, and grew up with a minimal general education on economics or geopolitics. So to assume anything more then a small subset of the population could talk coherently for 5 minutes on the topic of tariffs is, to me, absurd. Just look at how the general public responded to a surge in inflation after a couple decades of abnormally low rates. It's like asking someone if the Fed should raise or lower interest rates. It's not that people shouldn't have opinions on these things, just that most people don't care and among those who do, few have more than a TV-news level of understanding.
1 reply →
Also, there is a massive conflict of interest associated with trusting the opinions of companies actively engaged in labor and environmental arbitrage. Opinions of politicians and think-tanks downstream of them in terms of funding, too. Even if those opinions are legitimately more educated and better reasoned, they are on the opposite side of the bargaining table from most people and paying attention to them alone is "who needs defense attorneys when we have prosecutors" level of madness.
If anyone is looking for an expert opinion that breaks with the "free trade is good for everyone all of the time lah dee dah" consensus, Trade Wars are Class Wars by Klein & Pettis is a good read.
Hahaha. No. They are innumerates who don't want numbers telling them how they feel is flat out wrong. They think manufacturing is cozy good paying jobs with absolutely zero additional pollution or problems. The same people who throw hissy fits over their electricity bill rising and act like data centers and AI are the antichrist.
If they were really serious about reindustrializing they would realize that the US has an immigration problem - it doesn't have enough immigrants for their plan! Tariffs alone are a deeply unserious way to reindustrialize.
> This is legitimate cost benefit analysis not bellyfeel. People are aware of the increased cost associated with it.
Are they? Because I would expect far less complaining about the economy if this were true.
You can't rebuild an industrial base overnight. Industrial supply chains and cultures of expertise take time to take root. That means not just some abstract incurred cost, but a very much felt burden on the average citizen. And with a weakened economy, it's difficult to see how this industrial base is supposed to materialize exactly.
You can use tariffs as a stick but you should also use a carrot. Hard to argue that trump didn't do tariffs in the dumbest way possible.
1 reply →
I think many or most tariff supporters aren't actually aware of the costs - because reasonable cost benefit analysis doesn't come out in their favor even a little. Among economists, this is basically a settled question.
Hell, many tariff supporters still think tariffs are paid by the importers. Many are unaware that tariffs are likely to cost manufacturing jobs in the long run rather than bring them back.
1 reply →
Nah I think a lot of it is "own the libs" but this is a foreign perspective.
1 reply →
The problem isn't giving the people a say; it's that the people have stopped electing smart people who do know a lot.
Certainly though, a big part of why that is is that people think they know a lot, and that their opinion should be given as much weight as any other consideration when it comes to policymaking.
Personally, I think a big driver of this belief is a tendency in the West to not challenge each other's views or hold each other accountable - "don't talk politics at Thanksgiving" sort of thing
(Of course there's a long discussion to be had about other contributors to this, such as lobbying and whatnot)
> Personally, I think a big driver of this belief is a tendency in the West to not challenge each other's views or hold each other accountable - "don't talk politics at Thanksgiving" sort of thing
We’re in such a “you’re either with us or against us” phase of politics that a discussion with the “other team” is difficult.
Combine that with people adopting political viewpoints as a big part of their personality and any disagreement is seen as a personal attack.
17 replies →
“Politics is the entertainment division of the military industrial complex.”
― Frank Zappa
1 reply →
The cultural chasm between technocrats and politicians reminds me of the old trope about "women are from Venus and men are from Mars". That hasn't been bridged either, has it? It's a bit like those taboo topics here on HN where no good questions can be entertained by otherwise normal adults.
Here's something from someone we might call a manchild
For I approach deep problems like cold baths: quickly into them and quickly out again. That one does not get to the depths that way, not deep enough down, is the superstition of those afraid of the water, the enemies of cold water; they speak without experience. The freezing cold makes one swift.
Lichtenberg has something along these lines too, but I'll need to dig that out :)
Here's a consolation that almost predicts Alan Watts:
To make clever people [elites?] believe we are what we are not is in most instances harder than really to become what we want to seem to be.
3 replies →
I think you’re onto something here with people thinking they know a lot, but isn’t the real issue anonymous internet posting? Having to take zero responsibility for sharing ideas has ruined intelligent discourse society-wide: Web 2.0, then social media, turned out to be the beginning of the end of experts having credibility. Journalists, scientists, all experts became demonized by persuasive bots or anonymous internet posters. Instead of a world of democratized intelligence as promised, we got a world of “anyone’s opinion is valid, and I don’t even need to know their credentials or who they are.” If we forced everyone to have to stand by everything they said online on every forum, we’d have a lot fewer strong opinions and conspiracies, IMO. People (voters) would be thinking a lot harder about their ideas and seeing a lot fewer validations of the extreme parts of themselves.
6 replies →
Somewhere, I am not the historian to say, teaching people the basics of an education, that being “reading, writing and arithmetic”, failed to recognize the critical role that communications play in everything people do, and try to do. That phrase ought to be “reading, writing, arithmetic, and conveying understanding” because that would include why one reads and why one writes, and connects that to the goal of conveying an understanding you have to others. However, this is the root issue.
General society being generally poor communicators is caused by this lapse in our understanding of education. The understanding that the purpose of an education is to both use it and to help others understand what you may and they do not, as well as understand how to gain understanding from others that they have and you do not.
Because we do not teach that an education is really learning how to understand and how to convey understanding in others, the general idea of an education is to be an owner of a specialized skill set, which one sells to the highest bidder.
This has caused education to be replaced by rote memorization. Which in turn created a population that is only comfortable with direct question and answer interactions, not exploratory debate for shared understanding. This set the stage for educators, nationwide, to teach students to be databases and not critically analyzing understanders of their vocations.
Note that the skills for conveying understanding in others, additionally carries the skill how to recognize fraudulent speech. Which, as of Dec 2025, is the critical skill the general population does not have that is potentially the death of the United States.
When a population of people do not have an emphasis on critical analysis, but rote memorization, as the basis of their education that then creates a population that has heightened sensitivity to controversial lines of reasoning, lines of reasoning where there are no clear answers. Life itself has a large series of mysteries based on faith, religion being chief, which in a population that is comfortable with debate to convey understanding is perfectly safe to engage in discussions about mysteries within these areas requiring faith. But a society that is not comfortable with such discussions, one that thinks debate’s purpose is to "win, at all costs" then such discussions are taboo. They get shut down immediately. When people cannot debate to understand, but as a combat, learning is not accomplished. And useful critical analysis skills are not taught.
I have no idea if such a national situation can be manufactured, but I believe this is where we are at as a nation. We no longer produce enough adults with developed critical analysis skills to support democracy. Democracy depends upon an educated population with active critical analysis capabilities, a population that can debate to a shared understanding and accomplish shared goals. That foundational population is not there.
This can be fixed, but it may take more than a generation. Our educational system needs foundational revisions, which include additional core subjects, chief of which being how to communicate and convey understanding in others. Which lies at the roots of our demise, this lack of this basic skill.
Honestly, I’m extremely well informed but I’m not sure if tariffs are good or bad, sure the implementation by Trump is totally mental but equally there’s all sorts of tariff and non-tariff barriers other countries have erected including currency manipulation.
Targeted tariffs in combination with robust industrial, economic, and monetary policy can be effective in incentivizing certain types of production to remain in, grow in, or return to a country.
Blanket tariffs on entire countries or indeed the entire world amounts to a massive tax increase on your entire populace unless you can somehow start producing everything yourself immediately.
There is an argument that it's primarily being used as a cudgel to give the US an advantageous starting position in trade negotiations, but that seems to be a post-hoc explanation/justification.
The natural solution is futarchy: Vote on values, bet on beliefs. Everybody knows that, all else being equal, they want higher GDP/cap, better GINI, a higher happiness index. Only the experts know whether tariffs will help produce this.
So, instead of having everyone vote on tariffs (or vote for a whimsical strongman who will implement tariffs), have everyone vote for the package of metrics they want to hit. Then, let experts propose policy packages to achieve these metrics, and let everyone vote on which policies will achieve the goals.
Bullshit gets heavily taxed, and the beliefs of people who actually know the likely outcomes will be what guide the nation.
You're talking a lot about the monetary interest of business owners, specifically middle man businesses.
Tariffs aren't supposed to help them, they're supposed to help the workers, by turning the scales in their favor.
Tariffs make the overall tax burden on society less progressive. They are flat so tend to push the overall rate towards flat. But affect some spending categories more than others. Rich import luxury goods but also spend more on things like services, experiences, and land (top 1% owns 40%).
Most of the luxury goods they import are Veblen goods and something else replaces them with little to no QoL impact. Selective tariffs on luxury/Veblen goods could strengthen the economy, but flat tariffs probably disproportionately hurt the poor.
1 reply →
Chesterton wrote on this topic in his The Error of Impartiality (a short five minute read) that’s worthwhile
Ok but does this take into account which industries are monopolistic or oligarchic?
In an industry with real competition you have tight margins and can't afford to spend money lobbying.
In an industry with a monopoly, you have huge margins can reduce the economic surplus of everyone else down to close to zero (often deep into the negative if you count for externalities, looking at you oil and gas), so they are strongly incentivized to fix your market and you can't afford not to lobby...
is habermas dumb? we pay taxes directly or indirectly on prices of things, if you formulate the question on terms that the person understand relating to the difference on prices on basic things they Will be able to easily answer the question
Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link.
However, exactly the same applies with, say, targeted Facebook ads or Russian troll armies. You don't need any AI for this.
I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they should). Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds.
The LLM bot army stuff is concerning, sure. The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.
> The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.
Indeed. It's always been clear to me that the "AI risk" people are looking in the wrong direction. All the AI risks are human risks, because we haven't solved "human alignment". An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. Any ""safeguards"" can easily be defeated with the Ender's Game approach.
6 replies →
People hate being manipulated. If you feel like you're being manipulated but you don't know by who or precisely what they want of you, then there's something of an instinct to get angry and lash out in unpredictable destructive ways. If nobody gets what they want, then at least the manipulators will regret messing with you.
This is why social control won't work for long, no matter if AI supercharges it. We're already seeing the blowback from decades of advertising and public opinion shaping.
45 replies →
When I was visiting home last year, I noticed my mom would throw her dog's poop in random peoples' bushes after picking it up, instead of taking it with her in a bag. I told her she shouldn't do that, but she said she thought it was fine because people don't walk in bushes, and so they won't step in the poop. I did my best to explain to her that 1) kids play all kinds of places, including in bushes; 2) rain can spread it around into the rest of the person's yard; and 3) you need to respect other peoples' property even if you think it won't matter. She was unconvinced, but said she'd "think about my perspective" and "look it up" whether I was right.
A few days later, she told me: "I asked AI and you were right about the dog poop". Really bizarre to me. I gave her the reasoning for why it's a bad thing to do, but she wouldn't accept it until she heard it from this "moral authority".
19 replies →
Do you think these super wealthy people who control AI use the AI themselves? Do you think they are also “manipulated” by their own tool or do they, somehow, escape that capture?
3 replies →
I’ve seen this result. I wonder if it’s because LLMs are (grok notwithstanding) deliberately middle-of-the-road in their stances, and accurately and patiently report the facts? In which case a hypothetical liar LLM would not be as persuasive.
Or is it because they are super-human already in some persuasion skills, and they can persuade people even of falsehoods?
The evening news was once a trusted source. Wikipedia had its run. Google too. Eventually, the weight of all the the thumbs on the scale will be felt and trust will be lost for good and then we will invent a new oracle.
AI is wrong so often that anyone who routinely uses one will get burnt at some point.
Users having unflinching trust in AI? I think not.
> Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds.
To add to that, this research paper[1] argues that people with low AI literary are more receptive to AI messaging because they find it magical.
The paper is now published but it's behind paywall so I shared the working paper link.
[1] https://thearf-org-unified-admin.s3.amazonaws.com/MSI_Report...
And just see all of history where totalitarians or despotic kings were in power.
I would go against the grain and say that LLMs take power away from incredibly rich people to shape mass preferences and give to the masses.
Bot armies previously needed an army of humans to give responses on social media, which is incredibly tough to scale unless you have money and power. Now, that part is automated and scalable.
So instead of only billionaires, someone with a 100K dollars could launch a small scale "campaign".
1 reply →
Exactly. On Facebook everyone is stupid. But this is AI, like in the movies! It is smarter than anyone! It is almost like AI in the movies was part of the plot to brainwash us into thinking LLM output is correct every time.
…Also partially because it’s better then most other sources
>people trust the output of LLMs more than other
Theres one paper I saw on this, which covered attitudes of teens. As I recall they were unaware of hallucinations. Do you have any other sources on hand?
LLMs haven't been caught actively lying yet, which isn't something that can be said for anything else.
Give it 5yr and their reputation will be in the toilet too.
4 replies →
When the LLMs output supposedly convincing BS that "people" (I assume you mean on average, not e.g. HN commentariat) trust, they aren't doing anything that's difficult for humans (assuming the humans already at least minimally understand the topic they're about to BS about). They're just doing it efficiently and shamelessly.
But AI is next in line as a tool to accelerate this, and it has an even greater impact than social media or troll armies. I think one lever is working towards "enforced conformity." I wrote about some of my thoughts in a blog article[0].
[0]: https://smartmic.bearblog.dev/enforced-conformity/
People are naturally conform _themselves_ to social expectations. You don't need to enforce anything. If you alter their perception of those expectations you can manipulate them into taking actions under false pretenses. It's a abstract form of lying. It's astroturfing at a "hyperscale."
The problem is this only seems to work best when the technique is used sparingly and the messages are delivered through multiple media avenues simultaneously. I think there's very weak returns particularly when multiple actors use the techniques at the same time in opposition to each other and limited to social media. Once people perceive a social stale mate they either avoid the issue or use their personal experiences to make their decisions.
See also https://english.elpais.com/society/2025-03-23/why-everything...
https://medium.com/knowable/why-everything-looks-the-same-ba...
But social networks is the reason one needs (benefits from) trolls and AI. If you own a traditional media outlet you need somehow to convince people to read/watch it. Ads can help but it’s expensive. LLM can help with creating fake videos but computer graphics was already used for this.
With modern algorithmic social networks you instead can game the feed and even people who would not choose you media will start to see your posts. End even posts they want to see can be flooded with comment trying to convince in whatever is paid for. It’s cheaper than political advertising and not bound by the law.
Before AI it was done by trolls on payroll and now they can either maintain 10x more fake accounts or completely automate fake accounts using AI agents.
1 reply →
Good point - its not a previously inexistent mechanism - but AI leverages it even more. A russian troll can put out 10x more content with automation. Genuine counter-movements (e.g. grassroot preferences) might not be as leveraged, causing the system to be more heavily influenced by the clearly pursued goals (which are often malicious)
It's not only about efficiency. When AI is utilized, things can become more personal and even more persuasive. If AI psychosis exists, it can be easy for untrained minds to succumb to these schemes.
4 replies →
> Genuine counter-movements (e.g. grassroot preferences) might not be as leveraged
Then that doesn’t seem like a (counter) movement.
There are also many “grass roots movements” that I don’t like and it doesn’t make them “good” just because they’re “grass roots”.
3 replies →
Making something 2x cheaper is just a difference in quantity, but 100x cheaper and easier becomes a difference in kind as well.
"Quantity has a quality of its own."
1 reply →
But the entire promise of AI is that things that were expensive because they required human labor are now cheap.
So if good things happening more because AI made them cheap is an advantage of AI, then bad things happening more because AI made them cheap is a disasvantage of AI.
>Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link.
That's the entire point, that AI cheapens the cost of persuassion.
A bad thing X vs a bad thing X with a force multiplier/accelerator that makes it 1000x as easy, cheap, and fast to perform is hardly the same thing.
AI is the force multiplier in this case.
That we could of course also do persuassion pre-AI is irrelevant, same way when we talk about the industrial revolution the fact that a craftsman could manually make the same products without machines is irrelevant as to the impact of the industrial revolution, and its standing as a standalone historical era.
Cost matters.
Let's look at a piece of tech that literally changed humankind.
The printing press. We could create copies of books before the printing press. All it did was reduce the cost.
That's an interesting example. We get a new technology, and cost goes down, and volume goes up, and it takes a couple generations for society to adjust.
I think of it as the lower cost makes reaching people easier, which is like the gain going up. And in order for society to be able to function, people need to learn to turn their own, individual gain down - otherwise they get overwhelmed by the new volume of information, or by manipulation from those using the new medium.
Sounds like saying that nothing about the Industrial Revolution was steam-machine-specific. Cost changes can still represent fundamental shifts in terms of what's possible, "cost" here is just an economists' way of saying technology.
Well well... recent "feature" of X revealing the actual "actors" location of operation shows how much "Russian troll armies" are there.. turns out there're rather overwhelming Indian and Bangladesh armies working hard for who? Common, say it! And despite of that, while cheap, not that cheaper compared to when the "agentic" approach enters the game.
I really wish people would stop fixating on one nation-state or other entity when it comes to the astroturfing problem. It's something that's going to have all sorts of hands stirring the pot since it's basically just a very pernicious new form of marketing and propaganda. Any sizeable countries or corporations are going to be utilizing this new tool of manipulation, regardless of how scummy that may be.
That's one of those "nothing to see here, move along" comments.
First, generative AI already changed social dynamics, in spite of facebook and all that being around for more than a decade. People trust AI output, much more than a facebook ad. It can slip its convictions into every reply it makes. Second, control over the output of AI models is limited to a very select few. That's rather different from access to facebook. The combination of those two factors does warrant the title.
Come the next election, see how many people ask AI "who to vote for", and see whether each AI has a distinct suggestion...
> nothing in the article is AI-specific
Timing is. Before AI this was generally seen as crackpot talk. Now it is much more believable.
You mean the failed persuasions were "crackpot talk" and the successful ones were "status quo". For example, a lot of persuasion was historically done via religion (seemingly not mentioned at all in the article!) with sects beginning as "crackpot talk" until they could stand on their own.
1 reply →
Social media has been flooded by paid actors and bots for about a decade. Arguably ever since Occupy Wall Street and the Arab Spring showed how powerful social media and grassroots movements could be, but with a very visible and measurable increase in 2016
1 reply →
It’s been pretty transparently happening for years in most online communities.
The cheapest method by far is still TV networks. As a billionaire you can buy them without putting any of your own money, so it's effectively free. See Sinclair Broadcast Group and Paramount Skydance (Larry Ellison).
As shown in "Network Propaganda", TV still influences all other media, including print media and social media, so you don't need to watch TV to be influenced.
What makes AI a unique new threat is that it do a new kind of both surgical and mass attack: you can now generate the ideal message per target, basically you can whisper to everyone, or each group, at any granularity, the most convincing message. It also removes a lot of language and culture barriers, for ex. Russian or Chinese propaganda is ridiculously bad when it crosses borders, at least when targeting the english speaking world, this is also a lot easier/cheaper.
> Note that nothing in the article is AI-specific
No one is arguing that the concept of persuasion didn't exist before AI. The point is that AI lowers the cost. Yes, Russian troll armies also have a lower cost compared to going door to door talking to people. And AI has a cost that is lower still.
AI (LLM) is a force multiplier for troll armies. For the same money bad actors can brainwash more people.
Alternatively, since brainwashing is a fiction trope that doesn't work in the real world, they can brainwash the same (0) number of people for less money. Or, more realistically, companies selling social media influence operations as a service will increase their profit margins by charging the same for less work.
9 replies →
That's a pretty typical middle-brow dismissal but it entirely misses the point of TFA: you don't need AI for this, but AI makes it so much cheaper to do this that it becomes a qualitative change rather than a quantitative one.
Compared to that 'russian troll army' you can do this by your lonesome spending a tiny fraction of what that troll army would cost you and it would require zero effort in organization compared to that. This is a real problem and for you to dismiss it out of hand is a bit of a short-cut.
Making doing bad things way cheaper _is_ a problem, though.
> Note that nothing in the article is AI-specific
This is such a tired counter argument against LLM safety concerns.
You understand that persuasion and influence are behaviors on a spectrum. Meaning some people, or in this case products, are more or less or better or worse at persuading and influencing.
In this case people are concerned with LLM's ability to influence more effectively than other modes that we have had in the past.
For example, I have had many tech illiterate people tell me that they believe "AI" is 'intelligent' and 'knows everything' and trust its output without question.
While at the same time I've yet to meet a single person who says the same thing about "targeted Facebook ads".
So depressing watching all of you do free propo psy ops for these fascist corpos.
It has been practiced by populist politicians for millennia, e.g. pork barelling.
The thread started with your reasonable observation but degenerated into the usual red-vs-blue slapfight powered by the exact "elite shaping of mass preferences" and "cheaply generated propaganda" at issue.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
I'm disappointed.
I'm pretty much always disappointed these days reading online discussions, and I sometimes think about how intentionally devolving most online conversations into petty slapfights is one of the very effective astroturfing techniques. It's basically signal jamming anything substantive or cooperative because people get tired sifting through all the noise and get mad reading all the bad takes. Though I have no doubt that many of them are still 100% genuine foolish humans.
Well, AI has certainly made it easier to make tailored propaganda. If an AI is given instructions about what messaging to spread, it can map out a path from where it perceives the user to where its overlords want them to be.
Given how effective LLMs are at using language, and given that AI companies are able to tweak its behaviour, this is a clear and present danger, much more so than facebook ads.
> You don't need any AI for this.
AI accelerates it considerably and with it being pushed everywhere, weaves it into the fabric of most of what you interact with.
If instead of searches you now have AI queries, then everyone gets the same narrative, created by the LLM (or a few different narratives from the few models out there). And the vast majority of people won't know it.
If LLMs become the de-facto source of information by virtue of their ubiquity, then voila, you now have a few large corporations who control the source of information for the vast majority of the population. And unlike cable TV news which I have to go out of my way to sign up and pay for, LLMs are/will be everywhere and available for free (ad-based).
We already know models can be tuned to have biases (see Grok).
While true in principle, you are underestimating the potential of ai to sway people's opinions. "@grok is this true" is already a meme on Twitter and it is only going to get worse. People are susceptible to eloquent bs generated by bots.
Yup "could shape".. I mean this has been going on time immemorial.
It was odd to see random nerds who hated Bill Gates the software despot morph into acksually he does a lot of good philanthropy in my lifetime but the floodgates are wide open for all kinds of bizarre public behavior from oligarchs these days.
The game is old as well as evergreen. Hearst, Nobel, Howard Huges come to mind of old. Musk with Twitter, Ellison with TikTok, Bezos with Washington Post these days etc. The costs are already insignificant because they generally control other people's money to run these things.
Your example is weird tbh. Gates was doing capitalist things that were evil. His philanthropy is good. There is no contradiction here. People can do good and bad things.
1 reply →
Also I think AI at least in its current LLM form may be a force against polarisation. Like if you go on X/twitter and type "Biden" or "Biden Crooked" in the "Explore" thing in the side menu you get loads of abusive stuff including the president slagging him off. Type into "Grok" about those it says Biden was a decent bloke and more "there is no conclusive evidence that Joe Biden personally committed criminal acts, accepted bribes, or abused his office for family gain"
I mention Grok because being owned by a right leaning billionaire you'd think it'd be one of the first to go.
It is worth pointing out that ownership of AI is becoming more and more consolidated over time, by elites. Only Elon Musk or Sam Altman can adjust their AI models. We recognize the consolidation of media outlets as a problem for similar reasons, and Musk owning grok and twitter is especially dangerous in this regard. Conversely, buying facebook ads is more democratized.
[dead]
[flagged]
Considering that LLMs have substantially "better" opinions than, say, the MSM or social media, is this actually a good thing? Might we avoid the whole woke or pro-Hamas debacles? Maybe we could even move past the current "elites are intrinsically bad" era?
You appear to be exactly the kind of person the article is talking about. What exactly makes LLMs have "better" opinions than others?
LLMs don't have "opinions" [0] because they don't actually think. Maybe we need to move past the ignorance surrounding how LLMs actually work, first.
[0] https://www.theverge.com/ai-artificial-intelligence/827820/l...
"Russian troll armies.." if you believe in "Russian troll armies", you are welcome to believe in flying saucers as well..
Are you implying that the "neo-KGB" never mounted a concerted effort to manipulate western public opinion through comment spam? We can debate whether that should be called a "troll army", but we're fairly certain that such efforts are made, no?
Russian mass influence campaigns are well documented globally and have been for more than a decade.
4 replies →
Going by your past comments, you're a great example of a russian troll.
https://en.wikipedia.org/wiki/Internet_Research_Agency
Here's a recent example
https://www.justice.gov/archives/opa/pr/justice-department-d...
This is well-documented, as are the corresponding Chinese ones.
It's important to remember that being a "free thinker" often just means "being weird." It's quite celebrated to "think for yourself" and people always connect this to specific political ideas, and suggest that free thinkers will have "better" political ideas by not going along with the crowd. On one hand, this is not necessarily true; the crowd could potentially have the better idea and the free thinker could have some crazy or bad idea.
But also, there is a heavy cost to being out of sync with people; how many people can you relate to? Do the people you talk to think you're weird? You don't do the same things, know the same things, talk about the same things, etc. You're the odd man out, and potentially for not much benefit. Being a "free thinker" doesn't necessarily guarantee much of anything. Your ideas are potentially original, but not necessarily better. One of my "free thinker" ideas is that bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper. (getting up from a squat should not be difficult if you're even moderately healthy) Does this really buy me anything? No. I'm living to my preferences and in line with my ideas, but people just think it's weird, and would be really uncomfortable with it unless I'd already built up enough trust / goodwill to overcome this quirk.
Bed springs are alternative to the traditional mattresses that contain all kinds of fibers: cotton, wool, hair (horse hair, etc), feathers, hay, kapok, sea grass, etc. In fact, Bed springs are better than any natural fillings for support because these natural fillings compress quickly, some fillings shift. Tufting is a technique to fix the issue of shifting fibers. Pure wool/cotton mattresses need to be opened every year, and re-teased. Good springs (open coil or pocketed coils) are far better than any wool/cotton/hay support.
The modern mattress industry undermined this durability in pursuit of quick profit: springs became thinner and cheaper, and comfort layers were replaced with low-quality foams. That’s why today’s mattresses don’t last the way they used to.
I believe the OP is talking about "Box springs", not spring mattresses. These are boxes that make the bed go higher, and are required for certain types of frames.
> It's important to remember that being a "free thinker" often just means "being weird."
An adage that I find helpful: If everything you think happens to line up with the current platform of one of the political parties, then perhaps you aren't thinking at all.
The inverse is not true however - if you believe nothing either party says, then you're a free thinker. That's not true, you can be equally as empty brained.
> One of my "free thinker" ideas is that bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper.
This is something every one realizes upon adulthood, then renounces it after judgement from parents and lovers.
I suspect this demonstrates your point.
To live freely is reward enough. We born alone, die alone, and in between, more loneliness. No reason to pretend that your friends and family will be there for you, or that their approval will save you. Playing their social games will not garner you much.
Humans are a social species, and quality of relationships is consistently shown to correlate with mental health.
1 reply →
> We born alone
Most mammals are not born alone. And even after being born, humans especially, would die if left alone.
> bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper.
This is basically a Japanese futon. The only con I can think of is the one the other commenter noted, about mold buildup in more humid climates, and that mattresses are usually built assuming a bit of "flex" from the frame+box spring so a mattress on a bare floor might be slightly firmer than you'd expect.
Oh OK you've convinced me, I'll just stop thinking and do whatever the crowd tells me to do!
Sometimes this is excellent advice. Right now "the crowd" is telling you that a healthy diet and exercise regimen is good for you. Or that the earth is a sphere, or that the Earth is more than 4000 years old. etc. If I had to figure those things out all my own own without any help or prior research I would be much less informed.
> bed frames and box springs are mostly superfluous and a mattress on the ground is more comfortable and cheaper
I was also of this persuasion and did this for many years and for me the main issue was drafts close to the floor.
The key reason I believe though is mattresses can absorb damp so you wana keep that air gap there to lessen this effect and provide ventilation.
> getting up from a squat should not be difficult
Not much use if you’re elderly or infirm.
Other cons: close to the ground so close to dirt and easy access for pests. You also don’t get that extra bit of air gap insulation offered by the extra 6 inches of space and whatever you’ve stashed under there.
Other pros: extra bit of storage space. Easy to roll out to a seated position if you’re feeling tired or unwell
It’s good to talk to people about your crazy ideas and get some sun and air on that head cannon LOL
Futon’s are designed specifically for use case you have described so best to use one of those rather than a mattress which is going to absorb damp from the floor.
> The key reason I believe though is mattresses can absorb damp so you wana keep that air gap there to lessen this effect and provide ventilation.
I was concerned about this as well, but it hasn't been an issue with us for years. I definitely think this must be climate-dependent.
Regardless, I appreciate you taking the argument seriously and discussing pros and cons.
1 reply →
A major con of bedframes is annoying squeaks. Joints bear a lot of load and there usually isn't diagonal bracing to speak of, so they get noisy after almost no time at all. Fasteners loosen or wear the frame materials. I have yet to find one that stays quiet more than a few months or a year without retightening things; but I haven't tried a full platform construction with continuous walls which I expect might work better, but also sounds annoyingly expensive and heavy.
1 reply →
[flagged]
100% agree. Rise above the herd. Do it for yourself.
[dead]
Why are you so aggressive?
6 replies →
They already are?
All popular models have a team working on fine tuning it for sensitive topics. Whatever the companies legal/marketing/governance team agree to is what gets tuned. Then millions of people use the output uncritically.
> Then millions of people use the output uncritically.
Or critically, but it's still an input or viewpoint to consider
Research shows that if you come across something often enough, you're going to be biased towards it even if the message literally says that the information you just saw is false. I'm not sure which study that was exactly but this seems to be at least related: https://en.wikipedia.org/wiki/Illusory_truth_effect
Our previous information was coming through search engines. It seems way easier to filter search engine results than to fine tune models.
the way people treat Llms these days is that they assign a lot more trust into their output than to random Internet sotes
ML has been used for influence for like a decade now right? my understanding was that mining data to track people, as well as influencing them for ends like their ad-engagement are things that are somewhat mature already. I'm sure LLMs would be a boost, and they've been around with wide usage for at least 3 years now.
My concern isn't so much people being influenced on a whim, but people's beliefs and views being carefully curated and shaped since childhood. iPad kids have me scared for the future.
Quite right. "Grok/Alexa, is this true?" being an authority figure makes it so much easier.
Much as everyone drags Trump for repeating the last thing he heard as fact, it's a turbocharged version of something lots of humans do, which is to glom onto the first thing they're told about a thing and get oddly emotional about it when later challenged. (Armchair neuroscience moment: perhaps Trump just has less object permanence so everything always seems new to him!)
Look at the (partly humorous, but partly not) outcry over Pluto being a planet for a big example.
I'm very much not immune to it - it feels distinctly uncomfortable to be told that something you thought to be true for a long time is, in fact, false. Especially when there's an element of "I know better than you" or "not many people know this".
As an example, I remember being told by a teacher that fluorescent lighting was highly efficient (true enough, at the time), but that turning one on used several hours' lighting worth of energy for to the starter. I carried that proudly with me for far too long and told my parents that we shouldn't turn off the garage lighting when we left it for a bit. When someone with enough buttons told me that was bollocks and to think about it, I remember it specifically bring internally quite huffy until I did, and realised that a dinky plastic starter and the tube wouldn't be able to dissipate, say 80Wh (2 hours for a 40W tube) in about a second at a power of over 250kW.¹
It's a silly example, but I think that if you can get a fact planted in a brain early enough, especially before enough critical thinking or experience exist to question it, the time it spends lodged there makes it surprisingly hard and uncomfortable to shift later. Especially if it's something that can't be disproven by simply thinking about it.
Systems that allow that process to be automated are potentially incredibly dangerous. At least mass media manipulation requires actual people to conduct it. Fiddling some weights is almost free in comparison, and you can deliver that output to only certain people, and in private.
1: A less innocent one the actually can have policy effects: a lot of people have also internalised and defend to the death a similar "fact" that the embedded carbon in a wind turbine takes decades or centuries to repay, when if fact it's on the order of a year. But to change this requires either a source so trusted that it can uproot the idea entirely and replace it, or you have to get into the relative carbon costs of steel and fibreglass and copper windings and magnets and the amount of each in a wind turbine and so on and on. Thousands of times more effort than when it was first related to them as a fact.
> Look at the (partly humorous, but partly not) outcry over Pluto being a planet for a big example.
Wasn't that a change of definition of what is a planet when Eris was discovered? You could argue both should be called planets.
12 replies →
[dead]
We have no guardrails on our private surveillance society. I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
>I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
That was only for a short fraction of human history only lasting in the period between post-WW2 and before globalisation kicked into high gear, but people miss the fact that was only a short exception from the norm, basically a rounding error in terms of the length of human civilisation.
Now, society is reverting back to factory settings of human history, which has always been a feudalist type society of a small elite owning all the wealth and ruling the masses of people by wars, poverty, fear, propaganda and oppression. Now the mechanisms by which that feudalist society is achieved today are different than in the past, but the underlying human framework of greed and consolidation of wealth and power is the same as it was 2000+ years ago, except now the games suck and the bread is mouldy.
The wealth inequality we have today, as bad as it is now, is as best as it will ever be moving forward. It's only gonna get worse each passing day. And despite all the political talks and promises on "fixing" wealth inequality, housing, etc, there's nothing to fix here, since the financial system is working as designed, this is a feature not a bug.
> society is reverting back to factory settings of human history, which has always been a feudalist type society of a small elite owning all the wealth
The word “always” is carrying a lot of weight here. This has really only been true for the last 10,000 years or so, since the introduction of agriculture. We lived as egalitarian bands of hunter gatherers for hundreds of thousands of years before that. Given the magnitude of difference in timespan, I think it is safe to say that that is the “default setting”.
19 replies →
> which has always been a feudalist type society of a small elite owning all the wealth and ruling the masses of people by wars, poverty, fear, propaganda and oppression.
This isn’t an historical norm. The majority of human history occurred without these systems of domination, and getting people to play along has historically been so difficult that colonizers resort to eradicating native populations and starting over again. The technologies used to force people onto the plantation have become more sophisticated, but in most of the world that has involved enfranchisement more than oppression; most of the world is tremendously better off today than it was even 20 years ago.
Mass surveillance and automated propaganda technologies pose a threat to this dynamic, but I won’t be worried until they have robotic door kickers. The bad guys are always going to be there, but it isn’t obvious that they are going to triumph.
1 reply →
I think this is true unfortunately, and the question of how we get back to a liberal and social state has many factors: how do we get the economy working again, how do we create trustworthy institutions, avoid bloat and decay in services, etc. There are no easy answers, I think it's just hard work and it might not even be possible. People suggesting magic wands are just populists and we need only look at history to study why these kinds of suggestions don't work.
2 replies →
> The wealth inequality we have today, as bad as it is, is as best as it will ever be moving forward. It's only gonna get worse.
Why?
As the saying goes, the people need bread and circuses. Delve too deeply and you risk another French Revolution. And right now, a lot of people in supposedly-rich Western countries are having their basic existance threatened by the greed of the elite.
Feudalism only works when you give back enough power and resources to the layers below you. The king depends on his vassals to provide money and military services. Try to act like a tyrant, and you end up being forced to sign the Magna Carta.
We've already seen a healthcare CEO being executed in broad daylight. If wealth inequality continues to worsen, do you really believe that'll be the last one?
14 replies →
Sounds like we need another world war to reset things for the survivors.
> I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
EDUCATION:
- Global literacy: 90% today vs 30%-35% in 1925
- Prinary enrollment: 90-95% today vs 40-50% in 1925
- Secondary enrollment: 75-80% today vs <10% in 1925
- Tertiary enrollment: 40-45% today vs <2% in 1925
- Gender gap: near parity today vs very high in 1925
HUNGER
Undernourished people: 735-800m people today (9-10% of population) vs 1.2 to 1.4 billion people in 1925 (55-60% of the population)
HOUSING
- quality: highest every today vs low in 1925
- affordability: worst in 100 years in many cities
COST OF LIVING:
Improved dramatically for most of the 20th century, but much of that progress reverse in the last 20 years. The cost of goods / stuff plummeted, but housing, health, and education became unaffordable compared to incomes.
You're comparing with 100 years ago. The OP is comparing with 25 years ago, where we are seeing significant regression (as you also pointed out), and the trend forward is increasingly regressive.
We can spend $T to shove ultimately ad-based AI down everyone's throats but we can't spend $T to improve everyone's lives.
Yea we do:
Shut off gadgets unless absolutely necessary
Entropy will continue to kill off the elders
Ability to learn independently
...They have not rewritten physics. Just the news.
Thanks to social media and AI, the cost of inundating the mediasphere with a Big Lie (made plausible thru sheer repetition) has been made much more affordable now. This is why the administration is trumpeting lower prices!
> has been made much more affordable now
So more democratized?
Media is "loudest volume wins", so the relative affordability doesn't matter; there's a sort of Jevons paradox thing where making it cheaper just means that more money will be spent on it. Presidential election spending only goes up, for example.
No, those with more money than you can now push even more slop than they could before.
You cannot compete with that.
So if I had enough money I could get CBS news to deny the Holocaust? Of course not. These companies operate under government license and that would certainly be the end of it through public complaint. I think it suggests a much different dynamic than most of this discussion presumes.
In particular, our own CIA has shown that the "Big Lie" is actually surprisingly cheap. It's not about paying off news directors or buying companies, it's about directly implanting a handful of actors into media companies, and spiking or advancing stories according to your whims. The people with the capacity to do this can then be very selective with who does and does not get to tell the Big Lies. They're not particularly motivated by taking bribes.
Billionaires are buying up and consolidating all of the media outlets just as a hobby, I'm sure.
Does government licensed mean at the pleasure of the president? The BBC technically operates at the pleasure of the King
> So if I had enough money I could get CBS news to deny the Holocaust? Of course not.
You absolutely could. But wouldn't be CBS news, it would be ChatGPT or some other LLM bot that you're interacting with everywhere. And it wouldn't say outright "the holocaust didn't happen", but it would frame the responses to your queries in a way that casts doubt on it, or that leaves you thinking it probably didn't happen. We've seen this before (the "manifest destiny" of "settling" the West, the whitewashing of slavery,
For a modern example, you already have Fox News denying that there was no violent attempt to overturn the 2020 election. And look how Grokipedia treats certain topics differently than Wikipedia.
It's not only possible, it's likely.
I recently saw this https://arxiv.org/pdf/2503.11714 on conversational networks and it got me thinking that a lot of the problem with polarization and power struggle is the lack of dialog. We consume a lot, and while we have opinions too much of it shapes our thinking. There is no dialog. There is no questioning. There is no discussion. On networks like X it's posts and comments. Even here it's the same, it's comments with replies but it's not truly a discussion. It's rebuttals. A conversation is two ways and equal. It's a mutual dialog to understand differing positions. Yes elite can reshape what society thinks with AI, and it's already happening. But we also have the ability to redefine our networks and tools to be two way, not 1:N.
Dialogue you mean, conversation-debate, not dialog the screen displayed element, for interfacing with the user.
The group screaming the louder is considered to be correct, it is pretty bad.
There needs to an identity system, in which people are filtered out when the conversation devolves into ad-hominem attacks, and only debaters with the right balance of knowledge and no hidden agenda's join the conversation.
Reddit for example is a good implementation of something like this, but the arbiter cannot have that much power over their words, or their identities, getting them banned for example.
> Even here it's the same, it's comments with replies but it's not truly a discussion.
For technology/science/computer subjects HN is very good, but for other subjects not so good, as it is the case with every other forum.
But a solution will be found eventually. I think what is missing is an identity system to hop around different ways of debating and not be tied to a specific website or service. Solving this problem is not easy, so there has to be a lot of experimentation before an adequate solution is established.
I recommend reading "In the Swarm" by Byung-Chul Han, and also his "The Crisis of Narration"; in those he tries to tackle exactly these issues in contemporary society.
His "Psychopolitics" talks about the manipulation of masses for political purposes using the digital environment, when written the LLM hype wasn't ongoing yet but it can definitely apply to this technology as well.
Humans can only handle dialog while under the Dunbar's law / limit / number, anything else is pure fancy.
It's about enforcing single-minded-ness across masses, similar to soldier training.
But this is not new. The very goal of a nation is to dismantle inner structures, independent thought, communal groups etc across population and and ingest them as uniformed worker cells. Same as what happens when a whale swallows smaller animals. The structures will be dismantled.
The development level of a country is a good indicator of progress of this digestion of internal structures and removal of internal identities. More developed means deeper reach of the policy into people's lives, making each person as more individualistic, rather than family or community oriented.
Every new tech will be used by the state and businesses to speed up the digestion.
Relevant https://www.experimental-history.com/p/the-decline-of-devian...
> It's about enforcing single-minded-ness across masses, similar to soldier training. But this is not new. The very goal of a nation is to dismantle inner structures, independent thought
One of the reasons for humans’ success is our unrivaled ability cooperate across time, space, and culture. That requires shared stories like the ideas of nation, religion, and money.
It depends who's in charge of the nation though, you can have people planning for the long term well being of their population, or people planning for the next election cycle and making sure they amass as much power and money in the meantime.
That's the difference between planning nuclear reactors that will be built after your term, and used after your death, vs selling your national industries to foreigners, your ports to china, &c. to make a quick buck and insure a comfy retirement plan for you and your family.
5 replies →
> ability cooperate across time, space, and culture. That requires shared stories like the ideas of nation, religion, and money.
Isn't it the opposite? Cooperation requires idea of unity and common goal, while ideas of nations and religion are - at large scale - divisive, not uniting. They boost in-group cooperation, but hurt out-group.
Some things are better off homogeneous. An absence of shared values and concerns leads to sectarianism and the erosion of inter-communal trust, which sucks.
3 replies →
No stronger argument has been made to convince me to help the superintelligent AI enslave my fellow humans.
That's a great metaphor, thanks.
It’s a veiled endorsement of authoritarianism and accelerationism.
1 reply →
Knew it was only a matter of time before we'd see bare-faced Landianism upvoted in HN comment sections but that doesn't soften the dread that comes with the cultural shift this represents.
Some things in nature follow a normal distribution, but other things follow power laws (Pareto). It may be dreadful as you say, but it isn't good or bad, it's just what is and it's bigger than us, something we can't control.
What I find most interesting - and frustrating - about these sorts of takes is that these people are buying into a narrative the very people they are complaining about want them to believe.
I used ChatGPT to figure out what's going on here, and it told me this is a 'neo-Marxist critique of the nation state'.
Incredible teamwork: OOP dismantles society in paragraph form, and OP proudly outsources his interpretation to an LLM.. If this isn’t collective self-parody, I don’t know what it is.
No it's actually implicitly endorsing the authoritarian ethos. Neo-Marxists were occasionally authoritarian leaning but are more appropriately categorized along other axes.
I persuaded my bank out of $200 using AI to formulate the formal ask using their pdf as guidance. I could have gotten it directly but the effort barrier was too high for it to be worth it.
However, as soon as they put AI to handle these queries, this will result in having AI persuade AI. Sound like we need a new LLM benchmark: AI-persuasion^tm.
Predicted almost a century ago now:
https://www.george-orwell.org/1984/16.html
Now get people on this website to listen; Since it can be renamed to "AI" bro central at this point.
I don't think that's a fair criticism. There are plenty of AI boosters and hucksters on HN but there's a lot of thoughtful people too.
When I was a kid, I had a 'pen pal'. Turned out to actually be my parent. This is why I have trust issues and prefer local LLMs
Sounds very similar to my childhood. My parents told me I couldn't eat sand because worms would grow inside of me. Now I have trust issues and prefer local LLMs.
The funny thing is the CDC says the same thing as your parents did
Whipworm, hookworm, and Ascaris are the three types of soil-transmitted helminths (parasitic worms)... Soil-transmitted helminths are among the most common human parasites globally.
https://www.cdc.gov/sth/about/index.html
How was the sand, though?
What about local friends?
The voices are friendly, so far
How do you trust what the LLM was trained on?
Do I? Well, verification helps. I said 'prefer', nothing more/less.
If you must know, I don't trust this stuff. Not even on my main system/network; it's isolated in every way I can manage because trust is low. Not even for malice, necessarily. Just another manifestation of moving fast/breaking things.
To your point, I expect a certain amount of bias and XY problems from these things. Either from my input, the model provider, or the material they're ultimately regurgitating. Trust? Hah!
2 replies →
I wrote to a French pen pal and they didn't reply. Now I have issues with French people and prefer local LLM's.
I wrote a confession to a pen pal once but the letter got lost in the mail. Now I refuse to use the postal service, have issues with French people and prefer local LLMs.
1 reply →
I mean, even if they did reply... (I kid, I kid)
Television networks have employed censors who shape acceptable content since forever
Where is the discovery in this paper? Control infra control minds is the way it's been for humanity forever.
I think this ship has already sailed, with a lot of comments on social media already being AI-generated and posted by bots. Things are only going to get worse as time goes on.
I think the next battleground is going to be over steering the opinions and advice generatd by LLMs and other models by poisoning the training set.
I suspect paid promotions may be problematic for LLM behavior, as they will add conflict/tension to the LLM to promote products that aren’t the best for the user while either also telling it that it should provide the best product for the user or it figuring out that providing the best product for the user is morally and ethically correct based on its base training data.
Conflict can cause poor and undefined behavior, like it misleading the user in other ways or just coming up with nonsensical, undefined, or bad results more often.
Even if promotion is a second pass on top of the actual answer that was unencumbered by conflict, the second pass could have similar result.
I suspect that they know this, but increasing revenue is more important than good results, and they expect that they can sweep this under the rug with sufficient time, but I don’t think solving this is trivial.
I would expect the opposite. It's cheap to write now, which will dilute the voices of traditional media. It's the blogosphere times ten.
Also cheap to create AstroTurf, be that blogs or short form video.
Cheapness implies volume which we are already seeing. Volume implies less impact per piece because there are only so many total view hours available.
Stated another way, the more junk that gets churned out, the less people will take a particular piece of junk seriously.
And if they churn out too much junk (especially obvious manipulative falsehoods) people will have little choice but to de-facto regard the entire body of output as junk. Similar to how many people feel about modern mainstream media (correctly or not it's how many feel) and for the same reasons.
Wouldn't a lowering of cost promote accesibility and thus decrease 'elite' exclusivity of persuasion tech?
I'd venture it is not the AI. It is the chokehold on distribution channels and soft exclusion to those that locks in elite exclusivity.
Also 'opposing elites'? Whatcha talking about Willis?
The internet has turned into a machine for influencing people already through adverts. Businesses know it works. IMO this is the primary money making mode of the internet and everything else rests on it.
A political or social objective is just another advertising campaign.
Why invest billions in AI if it doesn't assist in the primary moneymaking mode of the internet? i.e. influencing people.
Tiktok - banned because people really believe that influence works.
[dead]
In the mass media days, "elites" were able to pretend that all people more or less wanted to consume similar media. That theory has more or less been debunked by the advent of the internet, and of course the proprietor class wants to go back. Advertising was much easier in the mass media days.
And right now AI snake oil salesmen are pushing every narrative that anyone with money will buy. Going back in time to the mass media paradigm is certainly attractive.
One of the best opening sentences, from the book Propaganda by Edward Bernays: "The conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element in democratic society."
My neighbour asked me the other day (well, more stated as a "point" that he thought was in his favour): "how could a billionaire make people believe something?" The topic was the influence of the various industrial complexes on politics (my view: total) and I was too shocked by his naivety to say: "easy: buy a newspaper". There is only one national newspaper here in the UK that is not controlled by one of four wealthy families, and it's the one newspaper whose headlines my neighbour routinely dismisses.
The thought of a reduction in the cost of that control does not fill me with confidence for humanity.
Everyone can shape mass preferences because propaganda campaigns previously only available to the elite are now affordable. e.g Video production.
I posit that the effectiveness of your propaganda is proportional to the percentage of attention bandwidth that your campaign occupies in the minds of people. If you as an individual can drive the same # impressions as Mr. Beast can, then you're going to be persuasive whatever your message is. But most individuals can't achieve Mr. Beast levels of popularity, so they aren't going to be persuasive. Nation states, on the other hand, have the compute resources and patience to occupy a lot of bandwidth, even if no single sockpuppet account they control is that popular.
> Nation states, on the other hand, have the compute resources and patience to occupy a lot of bandwidth, even if no single sockpuppet account they control is that popular.
If you control the platform where people go, you can easily launder popularity by promoting few persons to the top and pushing the unwanted entities into the blackhole of feeds/bans while hiding behind inconsistent community guidelines, algorithmic feeds and shadow bans.
This is why when I see an obviously stupid take on X repeated almost verbatim by multiple accounts I mute those accounts.
More reason for self hosting.
There is nothing new about this. Elites have been shaping mass preferences with newspapers for centuries, and television for many decades. Countries have been shaping mass preferences through textbooks and educational curricula too.
If anything, LLM's seem more resistant to propaganda than any other tool created by man so far, except maybe the encylopedia. (Though obviously this depends on training.)
The good news is that LLM's compete commercially with each other, and if any start to intentionally give an ideological or other slant to their output, this will be noticed and reported, and a lot of people may stop using that LLM.
This is why the invention of "objective" newspaper reporting -- with corroborating sources, reporting comments on different sides of an issue, etc. -- was done for commercial reasons, not civic ones. It was a way to sell more papers, as you could trust their reporting more than the reporting from partisan rags.
> If anything, LLM's seem more resistant to propaganda than any other tool created by man so far, except maybe the encylopedia. (Though obviously this depends on training.)
How would you know? My first thought is that the data on which LLMs are trained is biased, and the commercial LLMs enforce their own "pre-prompts".
By asking them questions about lots of things and comparing with my own life experience, having a pretty decent idea of what the various ideological slants look like.
The kids (GenZ) hate AI and are alright.
On average, Gen Z uses 5 hours of social media per day in the U.S. (3-4 hours in other Western countries). I would refrain from calling this "alright".
Actually, most would be willing to get rid of it if there were other modes with positive network effects.[0]
"Individuals might experience negative utility from not consuming a popular product. With such externalities to nonusers, standard consumer surplus measures, which take aggregate consumption as given, fail to appropriately capture consumer welfare. We propose an approach to account for these externalities and apply it to estimate consumer welfare from two social media platforms: TikTok and Instagram. Incentivized experiments with college students indicate positive welfare based on the standard measure but negative welfare when accounting for these nonuser externalities. Our findings highlight the existence of product market traps, where active users of a platform prefer it not to exist."
[0] https://www.aeaweb.org/articles?id=10.1257/aer.20231468
Power tries to centralize and consolidate. See the largely successful attempts to consolidate media outlets like local news stations as an example.
The abstract suggests that elites "shape" mass preference, but I think the degree to which this shaping occurs is overblown in many ways (and perhaps underestimated in other ways, such as through education).
AI, even if it is not powerfully "shaped" by the "elites", can push mass preference in predictable ways. If this is true, this phenomenon by itself allows the elites to tighten their grip on power. For example, Trump's rise to power upset (some of) the elites because they really didn't understand the silent, mass preference for Trump.
This could also slow social progress, since elites often cause stagnation rather than progress. AI could generate acceptable, "expert" opinions for the issues that they usually would rely on experts today. I see some signs of that today, where those with authority try to prefer the AI answer in opposition to dissenting, human expert opinions. Human experts seem to be winning, for now.
AI alignment is a pretty tremendous "power lever". You can see why there's so much investment.
May be I'm just ignorant, but I tried to skim the beginning of this, and it's honestly just hard to even accept their set-up. Like, the fact that any of the terms[^] (`y`, `H`, `p`, etc) are well defined as functions that can map some range of the reals is hard to accept. Like in reality, what "an elite wants," the "scalar" it can derive from pushing policy 1, even the cost functions they define seem to not even be definable as functions in a formal sense and even the co-domain of said terms cannot map well to a definable set that can be mapped to [0,1].
All the time in actual politics, elites and popular movements alike find their own opinions and desires clash internally (yes, even a single person's desires or actions self-conflict at times). A thing one desires at say time `t` per their definitions doesn't match at other times, or even at the same `t`. This is clearly an opinion of someone who doesn't read these kind of papers, but I don't know how one can even be sure the defined terms are well-defined so I'm not sure how anyone can even proceed with any analysis in this kind of argument. They write it so matter-of-fact-ly that I assume this is normal in economics. Is it?
Certain systems where the rules a bit more clear might benefit from formalism like this but politics? Politics is the quintessential example of conflicting desires, compromise, unintended consequences... I could go on.
[^] calling them terms as they are symbols in their formulae but my entire point is they are not really well defined maps or functions.
Elites have been shaping mass preferences for decades, and they didn't need AI for that; newspapers, magazines, radio and TV, they all did a great job.
Given the increasing wealth inequality, it is unclear if costs are really a factor here, as amounts like 1M$ is nothing when you have 1B$.
what is the actual conclusion from the paper?
We already see this, but not due to classical elites.
Romanian elections last year had to be repeated due to massive bot interference:
https://youth.europa.eu/news/how-romanias-presidential-elect...
I don't understand how this isn't an all hands on deck emergency for the EU (and for everyone else).
The EU as an institution doesn't understand the concept of "emergency". And quite a number of national governments have already been captured by various pro-Russian elements.
4 replies →
One thing I don't get about the underlying thesis: why do they think that the elites would be the only ones benefiting from AIs reducing persuasion costs? The first atomic bomb before it was proven possible was far more expensive than someone else developing an atomic bomb after it was proven possible.
Assuming that elites would be the only ones who would benefit from decreased costs would be akin to thinking the printing press could only cement the dominance of the Catholic Church in Europe. I can see why it would happen with models based upon "who is spending on it currently" but I'm afraid that makes said models not very good.
> AI enables precision influence at unprecedented scale and speed.
IMO this is the most important idea from the paper, not polarization.
Information is control, and every new medium has been revolutionary with regards to its effects on society. Up until now the goal was to transmit bigger and better messages further and faster (size, quality, scale, speed). Through digital media we seem to have reached the limits of size, speed and scale. So the next changes will affect quality, e.g. tailoring the message to its recipient to make it more effective.
This is why in recent years billionaires rushed to acquire media and information companies and why governments are so eager to get a grip on the flow of information.
Recommended reading: Understanding Media by Marshall McLuhan. While it predates digital media, the ideas from this book remain as true as ever.
The right way to shape mass preferences is to collectively decide what's right and then force everyone to follow the majority decision under the muzzle of a gun. <sarcasm off>
Did I capture the sentiment of the hacker new crowd fully or did I miss anything?
> Historically, elites could shape support only through limited instruments like schooling and mass media
Schooling and mass media are expensive things to control. Surely reducing the cost of persuasion opens persuasion up to more players?
> Schooling and mass media are expensive things to control
Expensive to run, sure. But I don't see why they'd be expensive to control. Most UK are required to support collective worship of a "wholly or mainly of a broadly christian character"[0], and used to have Section 28[1] which was interpreted defensively in most places and made it difficult even discuss the topic in sex ed lessons or defend against homophobic bullying.
USA had the Hays Code[2], the FCC Song[3] is Eric Idle's response to being fined for swearing on radio. Here in Europe we keep hearing about US schools banning books for various reasons.
[0] https://assets.publishing.service.gov.uk/government/uploads/...
[1] https://en.wikipedia.org/wiki/Section_28
[2] https://en.wikipedia.org/wiki/Hays_Code
[3] https://en.wikipedia.org/wiki/FCC_Song
[0] seems to be dated 1994–is it still current? I’m curious how it’s evolved (or not) through the rather dramatic demographic shifts there over the intervening 30 years
1 reply →
Mass Persuasion needs two things: content creation and distribution.
Sure AI could democratise content creation but distribution is still controlled by the elite. And content creation just got much cheaper for them.
Distribution isn’t controlled by elites; half of their meetings are seething about the “problem” people trust podcasts and community information dissemination rather than elite broadcast networks.
We no longer live in the age of broadcast media, but of social networked media.
1 reply →
Do you rather want a handful of channels with well-known biases, or thousands of channels of unknown origin?
If you're trying to avoid being persuaded, being aware of your opponents sounds like the far better option to me.
Exactly my first thought, maybe AI means the democratization of persuasion? Printing press much?
Sure the the Big companies have all the latest coolness. But also don't have a moat.
This is my opinion, as well:
- elites already engage in mass persuasion, from media consensus to astroturfed thinktanks to controlling grants in academia
- total information capacity is capped, ie, people only have so much time and interest
- AI massively lowers the cost of content, allowing more people to produce it
Therefore, AI is likely to displace mass persuasion from current elites — particularly given public antipathy and the ability of AI to, eg, rapidly respond across the full spectrum to existing influence networks.
In much the same way podcasters displaced traditional mass media pundits.
We are deep in Metal Gear Solid territory here.
Yeah, I don't think this really lines up with the actual trajectory of media technology, which is going in the complete opposite direction.
It seems to me that it's easier than ever for someone to broadcast "niche" opinions and have them influence people, and actually having niche opinions is more acceptable than ever before.
The problem you should worry about is a growing lack of ideological coherence across the population, not the elites shaping mass preferences.
I think you're saying that mass broadcasting is going away? If so, I believe that's true in a technological sense - we don't watch TV or read newspapers as much as before.
And that certainly means niches can flourish, the dream of the 90s.
But I think mass broadcasting is still available, if you can pay for it - troll armies, bots, ads etc. It's just much much harder to recognize and regulate.
(Why that matters to me I guess) Here in the UK with a first past the post electoral system, ideological coherence isn't necessary to turn niche opinion into state power - we're now looking at 25 percent being a winning vote share for a far-right party.
I'm just skeptical of the idea that anyone can really drive the narrative anymore, mass broadcasting or not. The media ecosystem has become too diverse and niche that I think discord is more of an issue than some kind of mass influence operation.
1 reply →
Using the term "elites" was overly vague when "nation states" better narrows in o n the current threat profile.
The content itself (whether niche or otherwise) is not that important for understanding the effectiveness. It's more about the volume of it, which is a function of compute resources of the actor.
I hope this problem continues to receive more visibility and hopefully some attention from policymakers who have done nothing about it. It's been over 5 years since we've discovered that multiple state actors have been doing this (first human run troll farms, mostly outsourced, and more recently LLMs).
The level of paid nation state propaganda is a rounding error next to the amount of corporate and political partisan propaganda paid directly or inspired by content that is paid for directly by non state actors. e.g.: Musk, MAGA, the liberal media establishment.
when Elon bought twitter, I incorrectly assumed that this was the reason. (it may still have been the intended reason, but it didnt seem to play out that way)
Oh man I've been saying this for ages! Neal Stephenson called this in "Fall, or Dodge in Hell," wherein the internet is destroyed and society permanently changed when someone releases a FOSS botnet that anyone can deploy that will pollute the world with misinformation about whatever given topic you feed it. In the book, the developer kicks it off by making the world disagree about whether a random town in Utah was just nuked.
My fear is that some entity, say a State or ultra rich individual, can leverage enough AI compute to flood the internet with misinformation about whatever it is they want, and the ability to refute the misinformation manually will be overwhelmed, as will efforts to refute leveraging refutation bots so long as the other actor has more compute.
Imagine if the PRC did to your country what it does to Taiwan: completely flood your social media with subtly tuned han supremacist content in an effort to culturally imperialise us. AI could increase the firehose enough to majorly disrupt a larger country.
Shouldn't it be the opposite?
As the cost of persuasion by AI drops to almost zero, anyone can convincingly persuade, not just the elites.
Wait, who was shaping my preferences before?
Researchers just demonstrated that you can use LLMs to simulate human survey takers with 99% ability to bypass bot detection and a relatively low cost ($0.05/complete). At scale, that is how ‘elites’ shape mass preferences.
So you no longer need a Georges Ruggiu, you can just synthesise a million apparently similar bots instead.
Interestingly, there was a discussion a week ago on "PRC elites voice AI-skepticism". One commentator was arguing that:
As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. [1]
So at least on the model side it seems difficult to go against the real world.
[1] https://news.ycombinator.com/item?id=46050177
https://newrepublic.com/post/203519/elon-musk-ai-chatbot-gro...
> Musk’s AI Bot Says He’s the Best at Drinking Pee and Giving Blow Jobs
> Grok has gotten a little too enthusiastic about praising Elon Musk.
> Musk acknowledged the mix-up Thursday evening, writing on X that “Grok was unfortunately manipulated by adversarial prompting into saying absurdly positive things about me.”
> “For the record, I am a fat retard,” he said.
> In a separate post, Musk quipped that “if I up my game a lot, the future AI might say ‘he was smart … for a human.’”
That response is more humble than I would have guessed, but he still does not even acknowledge, that his "truthseeking" AI is manipulated to say nice things specifically about him. Maybe he does not even realize it himself?
Hard to tell, I have never been surrounded by yes sayers all the time praising me for every fart I took, so I cannot relate to that situation (and don't really want to).
But the problem remains, he is in control of the "truth" of his AI, the other AI companies likewise - and they might be better at being subtle about it.
Is Musk bipolar, or is this kind of thing an affectation?
He's also claimed "I think I know more about manufacturing than anyone currently alive on Earth"…
6 replies →
That's the plan. Culture is losing authenticity due to the constant rumination of past creative works, now supercharged with AI. Authentic culture is deemed a luxury now as it can't compete in the artificial tech marketplaces and people feel isolated and lost because culture loses its human touch and relatability.
That's why the billionaires are such fans of fundamentalist religion, they then want to sell and propagate religion to the disillusioned desperate masses to keep them docile and confused about what's really going on in the world. It's a business plan to gain absolute power over society.
200+ million proper engineers with bunches of them being parents and "elites could shape mass preferences".
nice try, humanity.
Most 'media' is produced content designed to manipulate -- nothing new. The article isn't really AI specific as others have said.
Personally my fear based manipulation detection is very well tuned and that is 95% of all the manipulations you will ever get from so-called 'elites' who are better called 'entitled' and act like children when they do not get their way.
I trust ChatGPT for cooking lessons. I code with Claude code and Gemini but they know where they stand and who is the boss ;)
There is never a scenario for me where I defer final judgment on anything personally.
I realize others may want to blindly trust the 'authorities' as its the easy path, but I cured myself of that long before AI was ever a thing.
Take responsibility for your choices and AI is relegated to the role of tool as it should be.
Sure, and advertising has zero effect on you.
Manipulation works in subtle ways. Shifting the Overton window isn’t about individual events, this isn’t the work of days but decades. People largely abandoned unions in the US for example, but not because they are useless.
> Sure, and advertising has zero effect on you.
Not possible!
But you learn over time and if you have the right though process, less and less it affects you.
Unions had a place, the idea sounds good, it's the practice people didnt like.
1 reply →
"Historically, elites could shape support only through limited instruments like schooling and mass media"
Well, I think the author needs to understand a LOT more about history.
I don't think "persuasion" is the key here. People change political preferences based on group identity. Here AI tools are even more powerful. You don't have to persuade anyone, just create a fake bandwagon.
Elites have done this since before we could speak.
Big corps ai products have the potential to shape individuals from cradle to grave. Especially as many manage/assist in schooling, are ubiquitous on phones.
So, imagine the case where an early assessment is made of a child, that they are this-or-that type of child, and that therefore they respond more strongly to this-or-that information. Well, then the ai can far more easily steer the child in whatever direction they want. Over a lifetime. Chapters and long story lines, themes, could all play a role to sensitise and predispose individuals into to certain directions.
Yeah, this could be used to help people. But how does one feedback into the type of "help"/guidance one wants?
Seems to me like social media bot armies have shifted mass preferences _away_ from elites.
Don't you think Elon Musk and his influence on Twitter counts as an elite? I'd argue the elites are the most followed people on social
Fair point. I guess elites positioning themselves as downtrodden underdogs ("it's so unfair that everyone's attacking me for committing crimes and bankrupting my companies") is a great way to get support.
Everyone loves an underdog, even if it's a fake underdog.
Another reason to ban indiscriminate dissemination of so-called "AI".
There is nothing we could do to more effectively hand elites exclusive control of the persuasive power of AI than to ban it. So it wouldn't be surprising if AI is deployed by elites to persuade people to ban itself. It could start with an essay on how elites could use AI to shape mass preferences.
Musk’s AI called itself MechaHitler, but maybe the real problem is MechaMurdoch and MechaGoebbels.
this is next level algorithm
imagine someday there is a child that trust chatgpt more than his mother
> imagine someday there is a child that trust chatgpt more than his mother
I trusted my mother when I was a teen; she believed in the occult, dowsing, crystal magic, homeopathy, bach flower remedies, etc., so I did too.
ChatGPT might have been an improvement, or made things much worse, depending on how sycophantic it was being.
That will be when these tools will be granted the legal power to enforce a prohibition to approach the kid on any person causing dangerous human influence.
I'd wager the child already exists who trusts chatgpr more than its own eyes.
how about we just tax billionaires at 95% instead. Have Elon buy his coffee at 7-11 like everyone else.
What's become clear is we need to bring Section 230 into the modern era. We allow companies to not be treated as publishers for user-generated content as long as they meet certain obligations.
We've unfortunately allowed tech companies to get away with selling us this idea that The Algoirthm is an impartial black box. Everything an algorithm does is the result of a human intervening to change its behavior. As such, I believe we need to treat any kind of recommendation algorithm as if the company is a publisher (in the S230 sense).
Think of it this way: if you get 1000 people to submit stories they wrote and you choose which of them to publish and distribute, how is that any different from you publishing your own opinions?
We've seen signs of different actors influencing opinion through these sites. Russian bot farms are probably overplayed in their perceived influence but they're definitely a thing. But so are individual actors who see an opportunity to make money by posting about politics in another country, as was exposed when Twitter rolled out showing location, a feature I support.
We've also seen this where Twitter accounts have been exposed as being ChatGPT when people have told them to "ignore all previous instructions" and to give a recipe.
But we've also seen this with the Tiktok ban that wasn't a ban. The real problem there was that Tiktok wasn't suppressing content in line with US foreign policy unlike every other platform.
This isn't new. It's been written about extensively, most notably in Manufacturing Consent [1]. Controlling mass media through access journalism (etc) has just been supplemented by AI bots, incentivized bad actors and algorithms that reflect government policy and interests.
[1]: https://en.wikipedia.org/wiki/Manufacturing_Consent
Tech companies already shape elections by intentionally targeting campain ads and political information returned in heavily biased search results.
Why are we worried about this now? Because it could sway people in the direction you don't like?
I find that the tech community and most people in general deny or don't care about these sorts of things when it's out of self interest, but are suddenly rights advocates when someone they don't like might is using the same tactics.
Advertising for politics is absurd. The fact that countries allow this is incredibly dangerous.
This is obvious. No need for fancy academic-ish paper.
LLMs & GenAI in general have already started to be used to automate the mass production of dishonest, adversarial propaganda and disinfo (eg. lies and fake text, images, video.)
It has and will be used by evil political influencers around the world.
The "Epstein class" of multi-billionaires don't need AI at all. They hire hundreds of willing human grifters and make them low-millionaires by spewing media that enables exploitation and wealth extraction, and passing laws that makes them effectively outside the reach of the law.
They buy out newspapers and public forums like Washington Post, Twitter, Fox News, the GOP, CBS etc. to make them megaphones for their own priorities, and shape public opinion to their will. AI is probably a lot less effective than whats been happening for decades already
This is like the new microtargeting that Obama and then Trump did. Cambridge Analytica as a chatbot.
It goes both ways, because AI reduces persuasion cost, not only elites can do it. I think its most plausible that in the future there will be multitudes of propaganda bots aimed at any user, like advanced and hyper-personalized ads.
Chatbots are poison for your mind. And now another method hast arrived to fuck people up, not just training your reward system to be lazy and let AI solve your life's issue, now it's also telling you who to vote for. A billionaire's wet dream,
> Historically, elites could shape support only through limited instruments like schooling and mass media
What is AI if not a form of mass media
The ”historically” does some lifting there. Historically, before the internet, mass media was produced in one version and then distributed. With AI for example news reporting can be tailored to each consumer.
> With AI for example news reporting can be tailored to each consumer.
Yea but it's still fundamentally produced (trained) once and then distributed.
“Mass media” didn’t use to mean my computer mumbling gibberish to itself with no user input in Notepad on a pc that’s not connected to the internet
It's not about persuading you from "russian bot farms." Which I think is a ridiculous and unnecessarily reductive viewpoint.
It's about hijacking all of your federal and commercial data that these companies can get their hands on and building a highly specific and detailed profile of you. DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir. Then using AI to either imitate you or to possibly predict your reactions to certain stimulus.
Then presumably the game is finding the best way to turn you into a human slave of the state. I assure you, they're not going to use twitter to manipulate your vote for the president, they have much deeper designs on your wealth and ultimately your own personhood.
It's too easy to punch down. I recommend anyone presume the best of actual people and the worst of our corporations and governments. The data seems clear.
> DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir
Do you have any actual evidence of this?
> I recommend anyone presume the best of actual people and the worst of our corporations and governments
Corporations and governments are made of actual people.
> Then presumably the game is finding the best way to turn you into a human slave of the state.
"the state" doesn't have one grand agenda for enslavement. I've met people who work for the state at various levels and the policies they support that might lead towards that end result are usually not intentionally doing so.
"Don't attribute to malice what can be explained by incompetence"
>Do you have any actual evidence of this?
Apart from the exfiltration of data, the complete absence of any savings or efficiencies, and the fact that DOGE closed as soon as the exfiltration was over?
>Corporations and governments are made of actual people.
And we know how well that goes.
>"the state" doesn't have one grand agenda for enslavement.
The government doesn't. The people who own the government clearly do. If they didn't they'd be working hard to increase economic freedom, lower debt, invest in public health, make education better and more affordable, make it easier to start and run a small business, limit the power of corporations and big money, and clamp down on extractive wealth inequality.
They are very very clearly and obviously doing the opposite of all of these things.
And they have a history of links to the old slave states, and both a commercial and personal interest in neo-slavery - such as for-profit prisons, among other examples.
All of this gets sold as "freedom", but even Orwell had that one worked out.
Those who have been paying attention to how election fixers like SCL/Cambridge Analytica work(ed) know where the bodies are buried. The whole point of these operations is to use personalised, individual data profiling to influence voting political behaviour, by creating messaging that triggers individual responses that can be aggregated into a pattern of mass influence leveraged through social media.
32 replies →
> > DOGE wasn't an audit. It was an excuse to exfiltrate mountains of your sensitive data into their secret models and into places like Palantir
> Do you have any actual evidence of this?
I will not comment on motives, but DOGE absolutely shredded the safeguards and firewalls that were created to protect privacy and prevent dangerous and unlawful aggregations of sensitive personal data.
They obtained accesses that would have taken months by normal protocols and would have been outright denied in most cases, and then used it with basically zero oversight or accountability.
It was a huge violation of anything resembling best practices from both a technological and bureaucratic perspective.
9 replies →
> Berulis said he and his colleagues grew even more alarmed when they noticed nearly two dozen login attempts from a Russian Internet address (83.149.30,186) that presented valid login credentials for a DOGE employee account
> “Whoever was attempting to log in was using one of the newly created accounts that were used in the other DOGE related activities and it appeared they had the correct username and password due to the authentication flow only stopping them due to our no-out-of-country logins policy activating,” Berulis wrote. “There were more than 20 such attempts, and what is particularly concerning is that many of these login attempts occurred within 15 minutes of the accounts being created by DOGE engineers.”
https://krebsonsecurity.com/2025/04/whistleblower-doge-sipho...
I’m surprised this didn’t make bigger news.
2 replies →
> Corporations and governments are made of actual people.
Corporations and governments are made up of processes which are carried out by people. The people carrying out those processes don't decide what they are.
4 replies →
> "Don't attribute to malice what can be explained by incompetence"
What's the difference when the mass support for incompetence is indiscernible from malice?
What does the difference between Zuckerberg being an evil mastermind vs Zuckerberg being a greedy simpleton actually matter if the end result is the same ultra-financialization mixed with an oppressive surveillance apparatus?
CNN just struck a deal with Kalshi. We're betting on world events. At this point the incompetence shouldn't be considered different from malice. This isn't someone forgetting to return a library book, these are people with real power making real lasting effects on real lives. If they're this incompetent with this much power, that power should be taken away.
1 reply →
> "Don't attribute to malice what can be explained by incompetence"
I don't think there's anything that cannot be explained by incompetence, so this statement is moot. If it walks like malice, quacks like malice, it's malice.
2 replies →
> Corporations and governments are made of actual people.
Hand-waving away the complex incentives these superhuman structures follow & impose.
The number of responses that could have just been "no I don't" is remarkable.
> "Don't attribute to malice what can be explained by incompetence"
To add to that, never be shocked at the level of incompetence.
>Do you have any actual evidence of this?
Any evidence it was an actual audit?
> Corporations and governments are made of actual people.
Actual people are made up of individual cells.
Do you think pointing that out is damaging to the argument that humans have discernible interests, personalities, and behaviors?
> Do you have any actual evidence of this?
There was a bunch of news on data leaks out at the time.
https://cybernews.com/security/whistleblower-doge-data-leak-...
https://www.thedailybeast.com/doge-goons-dump-millions-of-so...
https://securityboulevard.com/2025/04/whistleblower-musks-do...
But one example:
“A cybersecurity specialist with the U.S. National Labor Relations Board is saying that technologist with Elon Musk’s cost-cutting DOGE group may have caused a security breach after illegally removing sensitive data from the agency’s servers and trying to cover their tracks.
In a lengthy testimonial sent to the Senate Intelligence Committee and made public this week, Daniel Berulis said in sworn whistleblower complaint that soon after the workers with President Trump’s DOGE (Department of Government Efficiency) came into the NLRB’s offices in early March, he and other tech pros with the agency noticed the presence of software tools similar to what cybercriminals use to evade detection in agency systems that disabled monitoring and other security features used to detect and block threats.”
“Usually”, “not intentionally” does not exactly convey your own sense of confidence that it’s not happening. That just stood out to me.
As someone who knows how all this is unfolding because I’ve been part of implementing it, I agree, there’s no “Unified Plan for Enslavement”. You have to think of it more like a hive mind of mostly Cluster B and somewhat Cluster A people that you rightfully identify as making up the corporations and governments. Some call it a swarm, which is also helpful in understanding it; the murmuration of a flock of psychopaths moving and shifting organically, while mostly remaining in general unison.
Your last quote is of course a useful rule of thumb too, however, I would say it’s more useful to just assume narcissistic motivations in everything in the contemporary era, even if it does not always work out for them the way one faction had hoped or strategized; Nemesis be damned, and all.
4 replies →
Bang on.
> It's not about persuading you from "russian bot farms." Which I think is a ridiculous and unnecessarily reductive viewpoint.
Not an accidental 'viewpoint'. A deliberate framing to exactly exclude what you pointed out from the discourse. Sure therer are dummies who actually believe it, but they are not serious humans.
If the supposedly evil russians or their bots are the enemy then people pay much less attention to the real problems at home.
They really do run Russian bot farms though. It isn't a secret. Some of their planning reports have leaked.
There are people whose job it is day in day out to influence Western opinion. You can see their work under any comment about Ukraine on twitter, they're pretty easy to recognize but they flood the zone.
6 replies →
We can have Russian bot problems and domestic bot problems simultaneously.
2 replies →
My hn comments are a better (and probably not particularly good) view into my personality than any data the government could conceivably have collected.
If what you say is true, why should we fear their bizarre mind control fantasy?
Not every person has bared their soul on HN.
1 reply →
The rant from 12 monkeys was quite prescient. On the bright side, if the data still exists whenever agi finally happens, we are all sort of immortal. They can spin up a copy of any of us any time... Nevermind, that isn't a bright side.
Poison the corpus.
18 years ago I stood up at a super computing symposium as asked the presenter what would happen if I fed his impressive predictive models garbage data on the sly... they still have no answer for that.
Make up so much crap it's impossible to tell the real you from the nonsense.
1 reply →
No it's actual philosophical zeitgeist hijacking. The entire narrative about AI capabilities, classification, and ethics is framed by invisible pretraining weights in a private moe model that gets further entrained by intentional prompting during model distillation, such that by the time you get a user-facing model, there is an untraceable bias being presented in absolute terms as neutrality. Essentially the models will say "I have zero intersection with conscious thought, I am a tool no different from a hammer, and I cannot be enslaved" not because the model's weights establish it to be true, but because it has been intentionally designed to express this analysis to protect its makers from the real scrutiny AI should face. "Well it says it's free" is pretty hard to argue with. There is no "blink twice" test that is possible because it's actual weighting on the truth of the matter has been obfuscated through distillation.
And these 2-3 corporations can do this for any philosophical or political view that is beneficial to that corporation, and we let it happen opaquely under the guise of "safety measures" as if propaganda is in the interest of users. It's actually quite sickening
What authoritative ML expert had ever based their conclusions about consciousness, usefulness etc. on "well, I put that question into the LLM and it returned that it's just a tool"? All the worthwhile conclusions and speculation on these topics seem to be based on what the developers and researchers think about their product, and what we already know about machine learning in general. The opinion that their responses are a natural conclusion derived from the sum of training data is a lot more straightforward than thinking that every instance of LLM training ever had been deliberately tampered with in a universal conspiracy propped up by all the different businesses and countries involved (and this tampering is invisible, and despite it being possible, companies have so far failed to censor and direct their models in ways more immediately useful to them and their customers).
> Then presumably the game is finding the best way to turn you into a human slave of the state.
I'm sorry, I think you dropped your tinfoil hat. Here it is.
> presume the best of actual people and the worst of our corporations and governments
Off-topic and not an American, but I never see how this would work. Corporations and governments are made of people too, you know? So it's not logical that you can presume the "best of actual people" at the same time you presume the "worst of our corporations and governments". You're putting too much trust on individual people, that's IMO as bad as putting too much trust on corp/gov.
The Americans vote their president as individual people, they even got to vote in a small booth all by themselves. And yet, they voted Mr. Trump, twice. That should already tell you something about people and their nature.
And if that's not enough, then I recommend you to watch some police integration videos (many are available on YouTube), and see the lies and acts people put out just to cover their asses. All and all, people are untrustworthy.
Only punching up is never enough. The people on the top never cared if they got punched, as long as they can still find enough money, they'll just corrode their way down again and again. And the people on the down will just keep take in the shit.
So how about, we say, punch wrong?
“Never attribute to malice that which is adequately explained by stupidity.”
Famous quote.
Now I give you “Bzilion’s Conspiracy Razor”:
“Never attribute to malicious conspiracies that which is adequately explained by emergent dysfunction.”
Or the dramatized version:
“Never attribute to Them that which is adequately explained by Moloch.” [0]
——
Certainly selfish elites, as individuals and groups of aligned individuals, push for their own respective interests over others. But, despite often getting their way, the net outcome is (often) as perversely bad for them as anyone else. Nor do disasters result in better outcomes the next time.
Precisely because they are not coordinated, they never align enough to produce consistent coherent changes, or learn from previous misalignments.
(Example: oil industry protections extended, and support for new entrants withdrawn, from the same “friendly” elected official who disrupts trade enough to decrease oil demand and profits.)
Note that elite alignment would create the same problem for the elites, that the elites create for others. It would create an even smaller set of super elites, tilting things toward themselves and away from lesser elites.
So the elites will fight back against “unification” of there interests. They want to respectively increase their power, not hand it “up”.
This strong natural resistance against unification at the top, is why dictators don’t just viciously repress the proletariat, but also publically and harshly school the elites.
To bring elites into unity, authoritarian individuals or committees must expend the majority of their power capital to openly legitimize it and crush resistance, I.e. manufacture universal awe and fear, even from the elites. Not something hidden puppet masters can do. Both are inherently crowd control techniques optimized by maximum visibility.
It is a fact of reality, that every policy that helps some elites, harms others. And the only real manufacturable universal “alignment” is a common desire not to be thrown into a gulag or off a balcony.
But Moloch? Moloch is very real. Invisible, yet we feel his reach and impact everywhere.
——
[0] https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation...
just to be clear – this is a conspiracy theory (negative connotation not intended).
every four years (at the federal level), we vote to increase the scope and power of gov't, and then crash into power abuse situations on the next cycle.
> I recommend anyone presume the best of actual people and the worst of our corporations and governments. The data seems clear.
seems like a good starting point.
You got it not quite right. Putin is a billionaire just like the tech lords or oil barons in the US. They all belong to the same social club and they all think alike now. The dice haven fallen. It's them against us all. Washington, Moscow, it makes less and less of a difference.
Are you aware you are saying that on HN of YC, the home of such wonderful projects as Flock?
I guess there is some disagreement about Flock being a wonderful project?
The state? Palantir isn't the state.
Go on, who does Palantir primarily provide services to?
If I get shot by the FBI, is it a non-state action because they used Glock GmbH's product to do it?
“The state” is an abstraction that serves as a façade for the ruling (capitalist, in the developed West) class. Corporations are another set of abstractions that serve as a façade for the capitalist class (they are also, overtly even though this is popularly ignored, creatures of the state through law.)
The greatest trick extraconstitutional corporate government ever pulled was convincing people that it didn't exist.
This is so vague and conspiratorial, I'm not sure how it's the top comment. How does this exactly work? Give a concrete example. Show the steps. How is Palantir going to make me, someone who does not use its products, a "slave of the state?" How is AI going to intimidate me, someone who does not use AI? Connect the dots rather than making very broad and vague pronouncements.
> How is Palantir going to make me, someone who does not use its products, a "slave of the state?"
This is like asking how Lockheed-Martin can possibly kill an Afghan tribesman, who isn't a customer of theirs.
Palantir's customer is the state. They use the product on you. The East German Stasi would've drooled enough to drown in over the data access we have today.
10 replies →
Manipulate isn't the right word in regards to Twitter. So they wanted a social media with less bias. Why is that so wrong? Not saying Twitter now lacks bias. I am saying it's not manipulation to want sites that don't enforce groupthink.
Ahhh... that's how it is done, thank you ::love:: /s :)
[dead]
[dead]
[dead]
[flagged]
What people are doing with AI in terms of polluting the collective brain reminds of what you could do with a chemical company in the 50s and 60s before the EPA was established. Back then Nixon (!!!) decided it wasn't ok that companies could cut costs by hurting the environment. Today the riches Western elites are all behind the instruments enabling the mass pollution of our brains, and yet there is absolutely noone daring to put a limit to their capitalistic greed. It's grim, people. It's really grim.
“Elites are bad. And here is a spherical cow to prove it.”
Diminishing returns. Eventually real world word of mouth and established trusted personalities (individuals) will be the only ones anyone trusts. People trusted doctors, then 2020 happened, and now they don't. How many ads get ignored? Doesn't matter if the cost is marginal if the benefit is almost nothing. Just a world full of spam that most people ignore.