Comment by crote

More than one danger from any given tech can be true at the same time. Coal plants can produce local smog as well as global warming.

There's certainly some AI risks that are the same as human risks, just as you say.

But even though LLMs have very human failures (IMO because the models anthropomorphise themselves as part of their training, thus leading to the outward behaviours of our emotions and thus emit token sequences such as "I'm sorry" or "how embarrassing!" when they (probably) didn't actually create any internal structure that can have emotions like sorrow and embarrassment), that doesn't generalise to all AI.

Any machine learning system that is given a poor quality fitness function to optimise, will optimise whatever that fitness function actually is, not what it was meant to be: "Literal minded genie" and "rules lawyering" may be well-worn tropes for good reason, likewise work-to-rule as a union tactic, but we've all seen how much more severe computers are at being literal-minded than humans.

I think people who care about superintelligent AI risk don't believe an AI that is subservient to humans is the solution to AI alignment, for exactly the same reasons as you. Stuff like Coherent Extrapolated Volition* (see the paper with this name) which focuses on what all mankind would want if they know more and they were smarter (or something like that) would be a way to go.

*But Yudkowsky ditched CEV years ago, for reasons I don't understand (but I admit I haven't put in the effort to understand).

What’s the “Ender’s Game Approach “? I’ve read the book but I’m not sure which part you’re referring to.

>An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human.

"Obedient" is anthropomorphizing too much (as there is no volition), but even then, it only matters according to how much agency the bot is extended. So there is also risk from neglectful humans who opt to present BS as fact due to an expectation of receiving fact and a failure to critique the BS.

People don't know they are being manipulated. Marketing does that all of the time and nobody complain. They complain about "too much advert" but not about "too much manipulation".

Example: in my country we often hear "it costs too much to repair, just buy a replacement". That's often not true, but we do pay. Mobile phone subscription are routinely screwing you, many complain but keep buying. Or you hear "it's because of immigration" and many just accept it, etc.

The longstanding existence of religions and the continual birth of new cults, the popularity of extremist political groups of all types, and the ubiquity of fortune-telling across cultures, seem to stand in opposition to your assertion that people hate being manipulated. At least, people enjoy belonging to something far more than they hate being manipulated. Most successful versions of fortune-telling, religious conversion, and cult recruitment do utilize confirmation bias affirmation, love-bombing, and other techniques to increase people's agreeableness before getting to the manipulation part, but they still successfully do that. It's also like saying that advertising is pointless because it's manipulating people into buying things, and while people dislike ads it's also still a very successful part of getting people to buy products or else corporations wouldn't still spend vast amounts of money on marketing.

People hate feeling manipulated, but they love propaganda that feeds their prejudices. People voluntarily turn on Fox News - even in public spaces - and get mad if you turn it off.

Sufficiently effective propaganda produces its own cults. People want a sense of purpose and belonging. Sometimes even at the expense of their own lives, or (more easily) someone else's lives.

No they hate feeling manipulated. They not only expect social manipulation they think you are downright rude, unsocialized, and untrustworthy if you don't manipulate them reflexively. Just look at mirroring alone.

https://en.wikipedia.org/wiki/Mirroring

I hated to come to this conclusion, but the average neurotypical person is fundamentally so batshit insane they think that not manipulating them is a sign you aren't trustworthy and ability to conceal your emotions and put on an appropriate emotional kabuki dance is a sign of trustworthiness.

  > People hate being manipulated.

The crux is whether the signal of abnormality will be perceived as such in society.

- People are primarily social animals, if they see their peers accept affairs as normal, they conclude it is normal. We don't live in small villages anymore, so we rely on media to "see our peers". We are increasingly disconnected from social reality, but we still need others to form our group values. So modern media have a heavily concentrated power as "towntalk actors", replacing social processing of events and validation of perspectives.

- People are easily distracted, you don't have to feed them much.

- People have on average an enormous capacity to absorb compliments, even when they know it is flattery. It is known we let ourselves being manipulated if it feels good. Hence, the need for social feedback loops to keep you grounded in reality.

TLDR: Citizens in the modern age are very reliant on the few actors that provide a semblance of public discourse, see Fourth Estate. The incentives of those few actors are not aligned with the common man. The autonomous, rational, self-valued citizen is a myth. Undermine the man's groups process => the group destroys the man.

Knowing one is manipulated, requires having some trusted alternate source to verify against.

If all your trusted sources are saying the same thing, then you are safe.

If all your untrusted sources are telling you your trusted sources are lying, then it only means your trusted sources are of good character.

Most people are wildly unaware of the type of social conditioning they are under.

I don't find your mother's reaction bizarre. When people are told that some behavior they've been doing for years is bad for reasons X,Y,Z, it's typical to be defensive and skeptical. The fact that your mother really did follow up and check your reasons demonstrates that she takes your point of view seriously. If she didn't, she wouldn't have bothered to verify your assertions, and she wouldn't have told you you were right all along.

As far as trusting AI, I presume your mother was asking ChatGPT, not Llama 7B or something. The LLM backed up your reasoning rather than telling her that dog feces in bushes is harmless isn't just happenstance, it's because the big frontier commercial models really do know a lot.

That isn't to say the LLMs know everything, or that they're right all the time, but they tend to be more right than wrong. I wouldn't trust an LLM for medical advice over, say, a doctor, or for electrical advice over an electrician. But I'd absolutely trust ChatGPT or Claude for medical advice over an electrician, or for electrical advice over a medical doctor.

But to bring the point back to the article, we might currently be living in a brief period where these big corporate AIs can be reasonably trusted. Google's Gemeni is absolutely going to become ad driven, and OpenAI seems on the path to following the same direction. Xai's Grok is already practicing Elon-thought. Not only will the models show ads, but they'll be trained to tell their users what they want to hear because humans love confirmation bias. Future models may well tell your mother that dog feces can safely be thrown in bushes, if that's the answer that will make her likelier to come back and see some ads next time.

On the one hand, confirming a new piece of information with a second source is good practice (even if we should trust our family implicitly on such topics). On the other, I'm not even a dog person and I understand the etiquette here. So, really, this story sounds like someone outsourcing their common sense or common courtesy to a machine, which is scary to me.

However, maybe she was just making conversation & thought you might be impressed that she knows what AI is and how to use it.

Quite a tangent, but for the purpose of avoiding anaerobic decomposition (and byproducts, CH4, H2S etc) of the dog poo and associated compostable bag (if you’re in one of those neighbourhoods), I do the same as your mum. If possible, flick it off the path. Else use a bag. Nature is full of the faeces of plenty of other things which we don’t bother picking up.

I don't know how old your mom is, but my pet theory of authority is that people older than about 40 accept printed text as authoritative. As in, non-handwritten letters that look regular.

When we were kids, you had either direct speech, hand-written words, or printed words.

The first two could be done by anybody. Anything informal like your local message board would be handwritten, sometimes with crappy printing from a home printer. It used to cost a bit to print text that looked nice, and that text used to be associated with a book or newspaper, which were authoritative.

Now suddenly everything you read is shaped like a newspaper. There's even crappy news websites that have the physical appearance of a proper newspaper website, with misinformation on them.

Welcome to my world. People don't listen to reason or arguments, they only accept social proof / authority / money talks etc. And yes, AI is already an authority. Why do you think companies are spending so much money on it? For profit? No, for power, as then profit comes automatically.

Wow, that is interesting! We used to go to elders, oracles, and priests. We have totally outsourced our humanity.

Well, I prefer this to people who bag up the poop and then throw the bag in the bushes, which seems increasingly common. Another popular option seems to be hanging the bag on a nearby tree branch, as if there's someone who's responsible for coming by and collecting it later.

It's fairly clear from Twitter that it's possible to be a victim of your own system. But sycophancy has always been a problem for elites. It's very easy to surround yourselves with people who always say yes, and now you can have a machine do it too.

This is how you get things like the colossal Facebook writeoff of "metaverse".

Isn't Grok just built as "the AI Elon Musk wants to use"? Starting from the goals of being "maximally truth seeking" and having no "woke" alignment and fewer safety rails, to the various "tweaks" to the Grok Twitter bot that happen to be related to Musk's world view

Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. Not something that's healthy or that he would likely prefer when asked, but something that would produce answers that he personally likes when using it

"someone with 100k dollars" is not exactly "the masses". It is a larger set, but it's just more rich/powerful people. Which I would not describe as the "masses".

I know what you mean, but that descriptor seems off

LLMs can't lie: they aren't alive.

The text they produce contains lies, constantly, at almost every interaction.

> LLMs haven't been caught actively lying yet…

Any time they say "I'm sorry" - which is very, very common - they're lying.

>Once people perceive a social stale mate they either avoid

This is called the Firehose of Falsehood and it's a very effective way of killing public participation.

>use their personal experiences to make their decisions

Yes they can if they have them. But people use other peoples personal experiences when they don't, which means all you have to do is become their facebook friend and then tell them that 'trans mexican aliens from mars stole their job' and they'll start repeating it as a personal experience.

Social networks are not a prerequisite for sentiment shaping by AI.

Every time you interact with an AI, its responses and persuasive capabilities shape how you think.

> If AI psychosis exists, it can be easy for untrained minds to succumb to these schemes.

Evolution by natural selection suggests that this might be a filter that yield future generations of humans that are more robust and resilient.

In this context grass roots would imply the interests of a group of common people in a democracy (as opposed to the interests of a small group of elites) which ostensibly is the point.

Propaganda with books could start wars in decades.

Propaganda with radio could start wars in years.

Propaganda with TV could start wars in months.

Propaganda with Internet/AI could start war in _____?

What I mean is that talking about mass persuation was (and to a certain degree, it still is) crackpot talk.

I'm not talking about the persuations themselves, it's the general public perception of someone or some group that raises awareness about it.

This also excludes ludic talk about it (people who just generally enjoy post-apocalyptic aesthetics but doesn't actually consider it to be a thing that can happen).

5 years ago, if you brought up serious talk about mass systemic persuation, you were either a lunatic or a philosopher, or both.

I'm not talking about whether it exists or not. I'm talking about how AI makes it more believable to say that it exists.

It seems very related, and I understand it's a very attractive hook to start talking about whether it exists or not, but that's definitely not where I'm intending to go.

And yet denied massively. This said, if I were one of the people performing mass manipulation, I would also make bots to say mass manipulation is not really.

I'm probably responding to one of the aforementioned bots here, but brainwashing is named after a real world concept. People who pioneered the practice named it themselves. [1] Real brainwashing predates fictional brainwashing.

[1] https://en.wikipedia.org/wiki/Brainwashing#China_and_the_Kor...

So your thesis is that marketing doesn't work?

[dead]

The "philanthropy" worked on you.

It is also right in their military strategy text that you can read yourself.

Even beyond that, why would an adversarial nation state to the US not do this? It is extremely asymmetrical, effective and cheap.

The parent comment shows how easy it is to manipulate smart people away from their common sense into believing obvious nonsense if you use your brain for 2 seconds.

Of course, of course.. still, strangely I see online other kinds of "armies" much more often.. and the scale, in this case, is indeed of armies..