There's a saddening effect that this kind of garbage foists onto society, and onto individual humans, that costs drastically more than the money the content itself scrapes up. On a societal level, you have the net effect of millions of individuals being duped into throwing their empathy at something absurdly false. That is a recipe for wild and random backlashes as people become convinced that the world is something it isn't. On an individual level, liking some fake bullshit, then realizing it's fake, causes people to feel humiliated. Ultimately, being humiliated by giving away your empathy to the wrong subject causes people to become angry and reject their natural empathy towards legitimate other people, having been fooled so many times already.
Maybe a better way of saying this is: Bad currency drives out good. And if the currency is empathy and sympathy itself, then we are racing toward a society that will no longer be able to cast itself into anyone else's shoes.
Good, people should learn that all the stuff they read online is fake. It's been like that for a few decades, the only thing that's changed is that now lying is cheap enough to be ubiquitous, and thus our collective immune system can finally work.
I'm not sure it's good to sharpen the collective immune system to the point where it becomes allergic to everything. This article decries the stupid blind trust that late 20th century humans still put into AI generated facebook posts. One shouldn't expect more discernment once that audience's trust is fully shattered. More likely is that once someone is scammed enough times to give up their sympathy, any genuine plea for sympathy will be met with distrust.
We do have a collective immune system, and it functions based on a collective understanding of reality. The hijacking of individual empathy for a few pennies at a time, using fake stories that trick and humiliate the targets, is like a retrovirus. It attacks the collective immune system that normally allows people to feel empathy for one another, by dissolving the sense that anyone else out there is real or honest.
It isn't though? But the stuff that is fake has a corrosive effect on out ability to interact with that which isn't.
For example the fine article.. reading it, it didn't seem off to me, at all. Then I see a lot of people in the comments saying it "feels generated". The author is also in the comments saying it's not generated.
> So braindead and stereotypical are these comments that you might think they are themselves AI generated. But, picking a few at random, I checked out their profiles and they seem genuine.
Maybe I don't read enough LLM slop, but that doesn't read like LLM slop to me. I have a hard time imagining the prompt that would create it, I have an easy time seeing a human write it, so to me "it doesn't feel generated". But that's all just us "feeling" something. Like in that well known Carl Sagan quote: "unable to distinguish between what feels good and what’s true"
Can I or the author prove that it's not generated? No, so that is enough to cast anything into doubt. And I don't think that destroying all trust between humans would make them super-resilient and rational and experts in spotting disinfo. I think it could isolate people completely, not even cut the bonds between them but singe the ability to even make bonds, and that might be to the horrors that the rootlessness of industrialized mass societies gave birth to what blindness is to darkness. Without the village it takes to raise a child, and also to "make" a mind, and a heart, there might be no more hearts and minds as we understand and cherish them.
When I talk to some old folks I know and they insist on "knowing" something from Facebook, and have to step on egg shells to not hurt their feelings, while my alarms are tingling because of what they tell me, and all of that without "AI" crap involved... man, there's so many vulnerable people out there, who are lonely enough as is. They will learn nothing from being assaulted like this, it will just make life more painful and confusing and misleading for them. All because their "engagement" is worth a few cents to someone. As if the brains and lifespans of people are just fruit to squeeze juice and pulp from.
Do I seem down on LLM slop? Because I'm super down on it and I want that to come across :)
That's the second time I've read this phrase on HN this week (earlier, it was in reference to the ad-supported Internet).
The race-to-the-bottom of zeroing out people's empathy to grab fractions of a penny (at scale) also reminds me of the Malthusian trap (aka "rats on the island") in slatestarcodex's "Meditations on Moloch". Whatever little nooks exist where some micro-pennies can still be found, will be farmed (at scale).
Empathy farming is what people with a religious or moral framework would call wicked. The reason the writer here observed that most of the suckers writing sympathy notes appear to be religious is not that religious people are stupid, it's that religious people are the last to completely abandon the notion that other people are real and need their support. They are the last to realize when a false messiah or a fake 105 year old woman baking fake cakes needs their help. It hijacks their moral framework, because they have invested a lifetime in believing certain tropes and terms and what (I as an atheist) would call sappy horseshit, which nonetheless imbues meaning into all aspects of their daily life. They don't suspect that the attack on their dignity will come disguised as an appeal to their faith.
I think you're giving people too much credit. Quite often they say empathetic things not because they're empathetic, but because that's the socially expected thing to do. They are so well-versed at this that they do it without even realizing it. Actual empathy is rare. Think about the corporate lingo which will kill you with kindness, but when there's an actual problem, you'll get thrown under the bus instantly. That's how people naturally behave.
To add on the point above: the whole panic is overblown. Majority of people have always been susceptible to blatant lies and manipulation. In my country there was an internet meme "Capital's sewer system as a tool of Judaism and quackery aimed at destroying the farming industry and extermination of local population". The thing is, that's a title of an actual essay printed 100 years ago. Someone genuinely believed this. In my father's village local peasants burnt down one guy's house because he used a scythe instead of a sickle on his own field, believing that this would lead to some curse or whatever.
The point is, people are stupid, have always been, and finding yet another way of generating obvious lies won't change anything in the big scheme of things. Reminds me of a South Park quote that goes along like this:
> I think you're giving people too much credit. Quite often they say empathetic things not because they're empathetic, but because that's the socially expected thing to do. They are so well-versed at this that they do it without even realizing it. Actual empathy is rare. Think about the corporate lingo which will kill you with kindness, but when there's an actual problem, you'll get thrown under the bus instantly. That's how people naturally behave.
I mostly believe the psychology studies that suggest that humans are generally empathetic by default. But then it seems to be fragile. Like you say, stress and culture can inhibit it. And that's without mentioning inborn cognitive biases like tribalism, which I guess works great if you're part of a desperate tribe fighting to survive and where the goal is winning Darwinism, and not so great when you're in a multicultural society at peace and there are a million and one problems to blame on others.
> The point is, people are stupid, have always been, and finding yet another way of generating obvious lies won't change anything in the big scheme of things.
But as we get more advanced levels of delusion (it can always get worse, when the metacognition kicks in), more powerful ways of influencing others, and bigger problems for society to confront, that means all the less capability to address those problems. Although I guess our lack of capability has not changed in the grand scheme of things...
Some are better than others. About three months ago I started seeing channels that were clearly stealing content from other channels and splicing it over a text-to-speech narration. I knew they were doing this because I had seen the original videos they were stealing from.
But then I've also seen a far larger number of channels which use text-to-speech to make audiobooks from public domain works. I recently listened to a couple of Plato's works on a channel like this. The text-to-speech did a fairly convincing reading, with Socrates and his companions each having their own distinct voices. These videos had tens of thousands of views, which might not be great, but if you can put out a few hundred videos like that in less than a year it might be worth it to the right person.
I also suspect most of the film summary channels have been using AI in similar ways since before it was even popular. Those videos get into the millions of views and in my opinion they are not bad as far as low-effort content goes.
I can't read the whole article, but one important point is that Facebook subsidizes users from these areas. It's cheaper to get 'Facebook data' than regular data.
As a result, life revolves around Facebook. All day, every day. It's how you shop, you contact each other, you 'google things', it's everything. And being rather high unemployment areas, people tend to spend all day there.
So given that you've essentially captured an entire demographic with little to no money that spend all their time on your platform, it's no surprise that's where the scams come from, whether it's outright or through such slop. Who can blame them?
And FB doesn't care, they just report 10 million user growth, without telling the actual truth that they're in a roundabout way, paying those users.
a very small amount of US dollars paid out to these people could be considered a "significant" amount of income when converted to those lower cost of living places.
A form that is particularly annoying to me is where official-sounding accounts are posting about plausible scientific advances or discoveries. Just yesterday, I saw some "James Webb Space Telescope" account gushing about new photos that dropped. They were beautiful, though obviously a little too nice, and of course NASA/JPL had released no such photos.
I can't put my finger on why this bugs me--and my wife thinks I'm being a pedantic when I comment that they're fake ("it's a nice picture, they like it, let them be")--but I think it has to do with the targeting of folks that absolutely would like that photo to be real.
Disinformation is a very real thing to get angry about. At the same time it's a blessing, because it gives away if people fact check the stuff they see
In recent India's general election. A deepfake video of a leading politician costed a massive vote shift and they almost lost an election. The damage was done by the time they have clarified it is fake.
Larger populations are hurt the most of this. Think of some small town with say 10 people in it and a local election. Someone puts out a deepfake. If it converts 10% of people to believe it that is just one crazy person in the town of ten people. Easily ignored. Now if you have an indian city of 10 million and convert 10% to believe your deepfake, now that is 1 million people on your side and that can’t so easily be ignored.
Propaganda spreads faster and affects more people in denser and larger population sizes. And when propaganda affects more people it starts feeling less like propaganda and starts feeling believable.
In Romania it's full of these pages that I never understood, but apparently they are used for 2 things:
- filter people that can be scammed, you can see that people commenting on these posts and congratulating whatever AI slop you have in the picture are targeted by bots that try to tell them they won something or getting them to buy various useless stuff online
- gather real likes and people in groups and for pages that later turn up to promote certain political candidates (see the recent annulled Romanian elections) or just change their names from "Beautiful Romania" to "Calin Georgescu for president"
He forgot to mention the reverse bait effect where loads of people engage due to their need to let us know that this is obviously ai. But in reallity these comments play a role in boosting the content
Yeah this is a bit part of youtube too. Some accounts are masterclasses in putting out just slightly frustrating quirks solely designed to drive a subset of people to the comments and boost engagement. The whole performance is a carefully rehearsed act from some of these content creators.
Damn two of those would have fooled me too if I wasn't looking too carefully.. the one like the one with the old lady and her cake and the kittens one.
The one where the cake is decorated with the phrase "Happy Birthday Bithiday"? Maybe that's her name.
Seriously though, I completely understand. AI slop tends to have a "when you see it, you can't unsee it" issue, but on first glance they're pretty compelling. Only when writing this did I notice how uncanny the 42 year old woman's thumbs were.
Yeah the two Birthday's gave it away. If it had some name, I would have believed it . Even the ones with the statue thing, the rose one looks realistic whereas the other two with the wooden lady is too perfect for a wooden statue (though I am no carpenter ) and the one with the sheep in the background, they just seem a little too small Idk why.
Those worthless posts on Facebook are all bot engagement farms. You can recognize them for having nothing to do with anything, and the commenters are all bots as well. You can click on their profile because it has to be public. If you look at their friends, they won't be mostly from someplace like real friends would, they'll be from all over. Each one of them will have a cover photo that is a low quality picture of...some people, doing nothing. Their posts will all be bot engagement, with a few nothing pictures (such as a sunset with the caption "Nice sunset.")
And they make money. Zuck makes money because he can claim users. Facebook ad metrics (what they report as clicks) were always inflated to begin with and now its just ai slop.
What’s the end game here for Facebook? Surely people will just stop going to these platforms or is the thinking Ai + the algorithm will be even more effective… shutter at the thought
FB is functionally dead for many people, unlikely to meaningfully grow. So they’re trying to reduce costs and make sure new platforms and ones that still have some growth potential get the effort instead. I’d rather work on the Threads or Instagram Reels teams than the FB one. And who knows, maybe the VR revolution Zuck wants really will happen.
VR revolution lacks the physics for it to ever happen. People need both time to plug into the vr and also money to buy the vr device and subscription. Two things that every business in the world wants from people which are in short supply these days due to how many hands want a finger in the finite pie of the consumers available time and money. Not to mention how the pie is generally shrinking due to increases to cost of living and wage depression.
It is kind of interesting how the incentive to profit leads to an outcome of lessened potential for profit. Someone who lives paycheck to paycheck is economically useless in the eyes of the capitalist: they have no money to part with essentially as whatever they are paid flows right out of their hands. What happens to capitalism when the numbers of paycheck to paycheck people continue to rise? A lot of the economic success of this country since WWII relative to other countries has depended on americans having more available disposable income to spend on consumer products than any other population in the world. We see that changing now. Strange times ahead.
Better this AI generated crap than the cute animal stories and videos where the animals are abused and hurt and then the abused pretends to heal them to gain sympathy views and money.
I saw this kind of posts since months ago and I wondered what is the point of "sadcore".
At first I thought they want to raise the engagement of some accounts and gather a following. But looking at the accounts, they doesn't seem to sell or promote anything.
Are they just sad trolls with a mission to amplify the amount of sadness in the world?
telegram also has stars, and reddit wanted to let you cash out gold, not sure what happened with that. let's also not forget tiktok's gifts. i believe instagram also has some kind of gifts but i can't find them
Even before AI images became viable, you had folks in low cost countries doing the same thing with local talent. AI might make it easier but humans in those countries can give you art for a small enough amount that scale makes up the expense enough to profit.
This is even present in fandom communities where young artists from low cost countries charge a lot less than their high cost country counterparts, leading to popularity and exposure on top of just revenue.
The arts have so few barriers to competition that it's quite hard to compete unless your art is just that novel. Most artists that make a living do it by outputting boring things like commercial voiceovers, corporate illustrations, or headshot photography.
There is plenty of human in the article. I suspect your ai-detector senses are heightened due to the content, and the writing style is mostly straightforward with a few needless flowery words, which makes it ripe for false detection.
But when I re-read it with "did AI write this?" in mind, there's plenty of stuff in there that I would find it rather difficult to get an LLM to write if I tried.
I was sent a YouTube video that was blatantly fake. Five seconds in I closed it and moved on. But I got curious.
I went back to the video and checked the channel. It has 7.4 million subscribers. It uploads 4 or 5 times a day. At least two 3 hours videos, and a couple short ones. All videos are generated with a voice over. All those that I have checked have comments with real people in them. This battle is already lost.
Morbid curiosity: would you be willing to share a link to that channel?
FWIW, I've gone down the rabbit hole of flat-earthers to try and understand what the appeal is (spoiler: it's often some variant of "if the Earth were a ball hurtling through space, that would be too terrifying")... maybe I should just stay away from this one :D
Easy to hate on this, but the existence of these kinds of things is so fascinating. Hundreds (maybe thousands) of people are out there generating AI click bait because people or system somewhere allows / encourages / pays for this market to exist
They are fascinating in the same sense road accidents or fatal diseases are - some people might like to watch them, but overall there is nothing good about this.
That’s not what’s happening here. The get-rich-quick grifters started selling AI slop manuals. It’s like saying brands that spend a lot on ads are successful. Post hoc ergo propter hoc.
And the feedback loop means we’re doing fuzzing on the human brain! If we can just take the human out of the loop and let the AI post and use a genetic algorithm to tweak its content, we’ll all be getting rooted in no time.
Still waiting for the top notch open source video AI. Want to see those kittens squeezed in the hydraulic press. Let’s just hope it doesn’t take as long as the 4K to 8k transition.
I fear your opinion is one that most casual content consumers don't even realize is happening. They just want to consume content...
However, they also share the road with you, help decide what content/messaging gets spread, and vote with their dollar (and political vote), just like you.
While I agree with your opinion, although it may be their fault, it easily can spread to become everyone's problem.
Could this be some sort of meta training activity? Meta allows this kind of content and uses engagement as a measure of value for future training material.
> The article you've just read is one such example.
There is no way to verify that. Until a method exists to discern real from AI-generated content, we should probably assume it's generated (for our safety).
Sadly, that's the world we live in now. It was kind of like that before, but it's even more so now.
How well does that actually work in practice? I was using the demo on Fakespot's site and verbatim output from ChatGPT seems to get detected pretty well, but random HN comments got a lot of (presumably) false positives. "AI detectors" are kind of a crapshoot in my opinion, but I'm curious.
I agree. It doesn't matter. Even if a human wrote it, how does that change anything? It certainly doesn't make it more true or a more honest reflection of somebody's feelings. These people who keep worrying about AI generated content don't seem to realize the rubbish that humans create is not better.
As I read I was thinking this article had been AI-generated. It's too empty, a lot of descriptions of the same images it had just shown, and then some commentary about how they are fake, then repeat and repeat again.
> a lot of descriptions of the same images it had just shown
This is an accessibility thing that seems to be very popular on Mastodon and I guess the author put it in the prose directly instead of as mouseover text.
It helps for people using screen readers, people who are in text browsers, people who don't load images by default, and funny enough, AI scrapers.
Author here. I definitely wrote it myself. As another commentator says, this is an accessibility thing. Individual images have alt tags but the Ghost gallery component didn't have an easy way to add those.
You don't need AI to attach bullshit titles to photos. The problem is with Facebook algorithm that recommends content based on"engagement" - nothing good can come out of this by design. AI just makes bullshit makers more productive like the rest of us.
>The problem is with Facebook algorithm that recommends content based on"engagement"
Ye it is pure toxic and made me stop using Facebook.
At some point in time a was fed "people falling and hurting them self a bit" videos. And I hovered over them and tried to find the blacklist button. So Facebook started to feed me more of those! And a hate those kind of videos.
>Bizarre as it may seem, some people actually send real money to these "creators"
It's also just plain ol' ad revenue as well. Not such thing as bad engagement.
>And then there are opportunities to sell "guest posts" to other spam merchants who want to get their content in front of gullible eyeballs.
Yeah, that too. get ad revenue, then become the ad yourself.
and... the article ends. Well that was a lot more fluff than I anticipated. Yes, not much will change until the people themsevles start rejecting this content en masse. But there will be people falling for it. There are still people falling for old school email spam, so it's sadly not shocking AI works for the non-discerning viewer.
hard to put a finger on it, but i do think you should give humanity a little credit... it's actually somewhat easy to spot the "AI" when it does appear in these situations? or maybe i'm just deluded.
I can only imagine that the "AI" look must be somehow intentional.
I recently discovered how to make very realistic images - you simply ask for your image, then you use the follow-up prompt "make it more realistic" and, most shockingly, it actually works. I never thought of doing it before because it's so stupid.
Critical thinking is taught in schools though. People in the real world don't, by and large, fall for this shit. The very small fraction that do ends up being a rather large number in absolute terms when spread across the whole world though.
Critical thinking doesn't make people less religious nor should less religious people be a goal in society.
I struggle to explain, but I feel like our data-obssessed society has completely thrown out the baby with the bathwater when it comes to these things. No one labels themselves with a religion or political party because there is a flaw in each one and anything with a flaw can't be correct, scientifically, so we just don't believe in any grand purpose to our lives, don't believe in any world leaders, don't believe in any shared tenets, and basically are all lonely and weak (because we abandon every group with a flaw).
The result of critical thinking shouldn't be a worse off society...
We don’t reject religion because of “a flaw.” We reject it because the fundamental basis of it is unsupported. It’s not “a flaw” when a house has no foundation. There is no baby, only bathwater.
When it comes to images like those in the article, “Is this real?” is not really part of my reaction. Neither is “poor thing” or “wow that’s impressive” or whatever else.
My reaction is “why is this useless shit being shoved in my face?”.
I don’t care if it’s real, the only reason I visited the website is to check if the local market’s on this week, and maybe see if anyone I know has posted anything (increasingly unlikely). I think in the modern age it's healthy to have a much wider cynicism - what is this crap, I didn't ask for it, f### off.
I don’t really get why Facebook tries so hard to get me to look at this rubbish. The more of this shit-shovelling there is, the less often I go and the fewer friend posts there are. It’s becoming a dead platform.
The technology of mind-control is advancing at a furious pace. These AI generated images and videos are just the latest evolution.
Who is vulnerable? Who is immune? What will its final form look like?
What do the scifi prophets say?
(This vast irresistible mind-control machinery serves the billionaires of course)
Poverty is probably your best shield. Because then you can't afford a phone. Someday the universal suicide order will drop and the only people left will be monks and beggars.
I am betting on relationships with people, and avoiding public Internet, except for shitposting on here.
This can be modelled as the back half of a whalefall. The Internet used to be a magic place where you could just stumble into community, useful information, etc. Now it's been over-exploited and game is becoming scarce.
The information is saturated with crap and mass media was never a substitute for friendships. Even though it's the hardest thing I've ever done, I'll have to make real friends.
What you wrote sounds a bit "out there", but, like Margaret Atwood's work, it's actually not too far away.
> Who is vulnerable? Who is immune?
I'm reminded of Hiro Protagonist (living in a storage unit), or Ready Player One protagonist (living in some impoverished mobile home stack) -- these are the "vulnerable".
As for "immune", it's the people who "control the supply":
P.S. Like all real-world examples, we have an exception -- Felon Husk, who gets high on his own supply, making him simultaneously victim and perpetrator.
There's a saddening effect that this kind of garbage foists onto society, and onto individual humans, that costs drastically more than the money the content itself scrapes up. On a societal level, you have the net effect of millions of individuals being duped into throwing their empathy at something absurdly false. That is a recipe for wild and random backlashes as people become convinced that the world is something it isn't. On an individual level, liking some fake bullshit, then realizing it's fake, causes people to feel humiliated. Ultimately, being humiliated by giving away your empathy to the wrong subject causes people to become angry and reject their natural empathy towards legitimate other people, having been fooled so many times already.
Maybe a better way of saying this is: Bad currency drives out good. And if the currency is empathy and sympathy itself, then we are racing toward a society that will no longer be able to cast itself into anyone else's shoes.
Good, people should learn that all the stuff they read online is fake. It's been like that for a few decades, the only thing that's changed is that now lying is cheap enough to be ubiquitous, and thus our collective immune system can finally work.
I'm not sure it's good to sharpen the collective immune system to the point where it becomes allergic to everything. This article decries the stupid blind trust that late 20th century humans still put into AI generated facebook posts. One shouldn't expect more discernment once that audience's trust is fully shattered. More likely is that once someone is scammed enough times to give up their sympathy, any genuine plea for sympathy will be met with distrust.
We do have a collective immune system, and it functions based on a collective understanding of reality. The hijacking of individual empathy for a few pennies at a time, using fake stories that trick and humiliate the targets, is like a retrovirus. It attacks the collective immune system that normally allows people to feel empathy for one another, by dissolving the sense that anyone else out there is real or honest.
1 reply →
Decades?
You know there is a story about virgin giving birth, guy coming from death after 3 days.
Not starting about dude throwing thunders or yet another one having a hammer only worthy can pick up.
There were loads of people who would swear by those stories to be true.
5 replies →
> all the stuff they read online is fake
It isn't though? But the stuff that is fake has a corrosive effect on out ability to interact with that which isn't.
For example the fine article.. reading it, it didn't seem off to me, at all. Then I see a lot of people in the comments saying it "feels generated". The author is also in the comments saying it's not generated.
> So braindead and stereotypical are these comments that you might think they are themselves AI generated. But, picking a few at random, I checked out their profiles and they seem genuine.
Maybe I don't read enough LLM slop, but that doesn't read like LLM slop to me. I have a hard time imagining the prompt that would create it, I have an easy time seeing a human write it, so to me "it doesn't feel generated". But that's all just us "feeling" something. Like in that well known Carl Sagan quote: "unable to distinguish between what feels good and what’s true"
Can I or the author prove that it's not generated? No, so that is enough to cast anything into doubt. And I don't think that destroying all trust between humans would make them super-resilient and rational and experts in spotting disinfo. I think it could isolate people completely, not even cut the bonds between them but singe the ability to even make bonds, and that might be to the horrors that the rootlessness of industrialized mass societies gave birth to what blindness is to darkness. Without the village it takes to raise a child, and also to "make" a mind, and a heart, there might be no more hearts and minds as we understand and cherish them.
When I talk to some old folks I know and they insist on "knowing" something from Facebook, and have to step on egg shells to not hurt their feelings, while my alarms are tingling because of what they tell me, and all of that without "AI" crap involved... man, there's so many vulnerable people out there, who are lonely enough as is. They will learn nothing from being assaulted like this, it will just make life more painful and confusing and misleading for them. All because their "engagement" is worth a few cents to someone. As if the brains and lifespans of people are just fruit to squeeze juice and pulp from.
Do I seem down on LLM slop? Because I'm super down on it and I want that to come across :)
> Bad currency drives out good
That's the second time I've read this phrase on HN this week (earlier, it was in reference to the ad-supported Internet).
The race-to-the-bottom of zeroing out people's empathy to grab fractions of a penny (at scale) also reminds me of the Malthusian trap (aka "rats on the island") in slatestarcodex's "Meditations on Moloch". Whatever little nooks exist where some micro-pennies can still be found, will be farmed (at scale).
Empathy farming is what people with a religious or moral framework would call wicked. The reason the writer here observed that most of the suckers writing sympathy notes appear to be religious is not that religious people are stupid, it's that religious people are the last to completely abandon the notion that other people are real and need their support. They are the last to realize when a false messiah or a fake 105 year old woman baking fake cakes needs their help. It hijacks their moral framework, because they have invested a lifetime in believing certain tropes and terms and what (I as an atheist) would call sappy horseshit, which nonetheless imbues meaning into all aspects of their daily life. They don't suspect that the attack on their dignity will come disguised as an appeal to their faith.
2 replies →
I think you're giving people too much credit. Quite often they say empathetic things not because they're empathetic, but because that's the socially expected thing to do. They are so well-versed at this that they do it without even realizing it. Actual empathy is rare. Think about the corporate lingo which will kill you with kindness, but when there's an actual problem, you'll get thrown under the bus instantly. That's how people naturally behave.
To add on the point above: the whole panic is overblown. Majority of people have always been susceptible to blatant lies and manipulation. In my country there was an internet meme "Capital's sewer system as a tool of Judaism and quackery aimed at destroying the farming industry and extermination of local population". The thing is, that's a title of an actual essay printed 100 years ago. Someone genuinely believed this. In my father's village local peasants burnt down one guy's house because he used a scythe instead of a sickle on his own field, believing that this would lead to some curse or whatever.
The point is, people are stupid, have always been, and finding yet another way of generating obvious lies won't change anything in the big scheme of things. Reminds me of a South Park quote that goes along like this:
- I'm worried that you won't like me anymore
- We never liked you in the first place
- So... nothing changes?
- Nothing changes.
> I think you're giving people too much credit. Quite often they say empathetic things not because they're empathetic, but because that's the socially expected thing to do. They are so well-versed at this that they do it without even realizing it. Actual empathy is rare. Think about the corporate lingo which will kill you with kindness, but when there's an actual problem, you'll get thrown under the bus instantly. That's how people naturally behave.
I mostly believe the psychology studies that suggest that humans are generally empathetic by default. But then it seems to be fragile. Like you say, stress and culture can inhibit it. And that's without mentioning inborn cognitive biases like tribalism, which I guess works great if you're part of a desperate tribe fighting to survive and where the goal is winning Darwinism, and not so great when you're in a multicultural society at peace and there are a million and one problems to blame on others.
> The point is, people are stupid, have always been, and finding yet another way of generating obvious lies won't change anything in the big scheme of things.
But as we get more advanced levels of delusion (it can always get worse, when the metacognition kicks in), more powerful ways of influencing others, and bigger problems for society to confront, that means all the less capability to address those problems. Although I guess our lack of capability has not changed in the grand scheme of things...
[dead]
404media have by far the best coverage of the epidemic of AI generated spam images on Facebook:
https://www.404media.co/where-facebooks-ai-slop-comes-from/
It is mainly driven by get rich quick YouTube influencers, many from India and the Philippines.
Unsurprisingly it is not a great way to actually make money.
Some are better than others. About three months ago I started seeing channels that were clearly stealing content from other channels and splicing it over a text-to-speech narration. I knew they were doing this because I had seen the original videos they were stealing from.
But then I've also seen a far larger number of channels which use text-to-speech to make audiobooks from public domain works. I recently listened to a couple of Plato's works on a channel like this. The text-to-speech did a fairly convincing reading, with Socrates and his companions each having their own distinct voices. These videos had tens of thousands of views, which might not be great, but if you can put out a few hundred videos like that in less than a year it might be worth it to the right person.
I also suspect most of the film summary channels have been using AI in similar ways since before it was even popular. Those videos get into the millions of views and in my opinion they are not bad as far as low-effort content goes.
I can't read the whole article, but one important point is that Facebook subsidizes users from these areas. It's cheaper to get 'Facebook data' than regular data.
As a result, life revolves around Facebook. All day, every day. It's how you shop, you contact each other, you 'google things', it's everything. And being rather high unemployment areas, people tend to spend all day there.
So given that you've essentially captured an entire demographic with little to no money that spend all their time on your platform, it's no surprise that's where the scams come from, whether it's outright or through such slop. Who can blame them?
And FB doesn't care, they just report 10 million user growth, without telling the actual truth that they're in a roundabout way, paying those users.
You may be thinking of Facebook’s Free Basics. That was a short lived program that ended years ago.
1 reply →
a very small amount of US dollars paid out to these people could be considered a "significant" amount of income when converted to those lower cost of living places.
A form that is particularly annoying to me is where official-sounding accounts are posting about plausible scientific advances or discoveries. Just yesterday, I saw some "James Webb Space Telescope" account gushing about new photos that dropped. They were beautiful, though obviously a little too nice, and of course NASA/JPL had released no such photos.
I can't put my finger on why this bugs me--and my wife thinks I'm being a pedantic when I comment that they're fake ("it's a nice picture, they like it, let them be")--but I think it has to do with the targeting of folks that absolutely would like that photo to be real.
It bugs you becouse it is a lie and fools people I guess?
It is like these "life hack" cooking recipes that don't work. They are wasting kids enthusiasm in cooking and trust in people.
It bothers me because I know that being lied to frequently can have terrifying outcomes for everyone.
Disinformation is a very real thing to get angry about. At the same time it's a blessing, because it gives away if people fact check the stuff they see
On the other hand, without dis/misinformation, it wouldn’t be so important to know whether people fact check the stuff they see.
In recent India's general election. A deepfake video of a leading politician costed a massive vote shift and they almost lost an election. The damage was done by the time they have clarified it is fake.
I've heard the social media landscape is a lot different in India because so much of it passes through the comparatively-opaque WhatsApp rumor mill.
Compared to?
5 replies →
John Oliver did a piece on this back in 2021.
https://www.nbcnews.com/news/asian-america/john-oliver-tackl...
Larger populations are hurt the most of this. Think of some small town with say 10 people in it and a local election. Someone puts out a deepfake. If it converts 10% of people to believe it that is just one crazy person in the town of ten people. Easily ignored. Now if you have an indian city of 10 million and convert 10% to believe your deepfake, now that is 1 million people on your side and that can’t so easily be ignored.
Propaganda spreads faster and affects more people in denser and larger population sizes. And when propaganda affects more people it starts feeling less like propaganda and starts feeling believable.
In Romania it's full of these pages that I never understood, but apparently they are used for 2 things:
- filter people that can be scammed, you can see that people commenting on these posts and congratulating whatever AI slop you have in the picture are targeted by bots that try to tell them they won something or getting them to buy various useless stuff online
- gather real likes and people in groups and for pages that later turn up to promote certain political candidates (see the recent annulled Romanian elections) or just change their names from "Beautiful Romania" to "Calin Georgescu for president"
> They did also all seem to be active churchgoers but that must be some kind of coincidence...
Oh i love this one !
He forgot to mention the reverse bait effect where loads of people engage due to their need to let us know that this is obviously ai. But in reallity these comments play a role in boosting the content
Yeah this is a bit part of youtube too. Some accounts are masterclasses in putting out just slightly frustrating quirks solely designed to drive a subset of people to the comments and boost engagement. The whole performance is a carefully rehearsed act from some of these content creators.
Damn two of those would have fooled me too if I wasn't looking too carefully.. the one like the one with the old lady and her cake and the kittens one.
The one where the cake is decorated with the phrase "Happy Birthday Bithiday"? Maybe that's her name.
Seriously though, I completely understand. AI slop tends to have a "when you see it, you can't unsee it" issue, but on first glance they're pretty compelling. Only when writing this did I notice how uncanny the 42 year old woman's thumbs were.
And no one is carefully vetting their content feeds, if anything their brains are completely shut off as they scroll.
Yeah the two Birthday's gave it away. If it had some name, I would have believed it . Even the ones with the statue thing, the rose one looks realistic whereas the other two with the wooden lady is too perfect for a wooden statue (though I am no carpenter ) and the one with the sheep in the background, they just seem a little too small Idk why.
1 reply →
Those worthless posts on Facebook are all bot engagement farms. You can recognize them for having nothing to do with anything, and the commenters are all bots as well. You can click on their profile because it has to be public. If you look at their friends, they won't be mostly from someplace like real friends would, they'll be from all over. Each one of them will have a cover photo that is a low quality picture of...some people, doing nothing. Their posts will all be bot engagement, with a few nothing pictures (such as a sunset with the caption "Nice sunset.")
It's all, 100%, crap.
You've basically summed up most social media users habits and profiles.
And they make money. Zuck makes money because he can claim users. Facebook ad metrics (what they report as clicks) were always inflated to begin with and now its just ai slop.
What’s the end game here for Facebook? Surely people will just stop going to these platforms or is the thinking Ai + the algorithm will be even more effective… shutter at the thought
> What’s the end game here for Facebook?
FB is functionally dead for many people, unlikely to meaningfully grow. So they’re trying to reduce costs and make sure new platforms and ones that still have some growth potential get the effort instead. I’d rather work on the Threads or Instagram Reels teams than the FB one. And who knows, maybe the VR revolution Zuck wants really will happen.
VR revolution lacks the physics for it to ever happen. People need both time to plug into the vr and also money to buy the vr device and subscription. Two things that every business in the world wants from people which are in short supply these days due to how many hands want a finger in the finite pie of the consumers available time and money. Not to mention how the pie is generally shrinking due to increases to cost of living and wage depression.
It is kind of interesting how the incentive to profit leads to an outcome of lessened potential for profit. Someone who lives paycheck to paycheck is economically useless in the eyes of the capitalist: they have no money to part with essentially as whatever they are paid flows right out of their hands. What happens to capitalism when the numbers of paycheck to paycheck people continue to rise? A lot of the economic success of this country since WWII relative to other countries has depended on americans having more available disposable income to spend on consumer products than any other population in the world. We see that changing now. Strange times ahead.
Better this AI generated crap than the cute animal stories and videos where the animals are abused and hurt and then the abused pretends to heal them to gain sympathy views and money.
I saw this kind of posts since months ago and I wondered what is the point of "sadcore".
At first I thought they want to raise the engagement of some accounts and gather a following. But looking at the accounts, they doesn't seem to sell or promote anything.
Are they just sad trolls with a mission to amplify the amount of sadness in the world?
Attention is worth money. I assume they'll sell the accounts to scammers or political campaigns at some point.
What social media has microtransaction stars?
Facebook, apparently: https://www.facebook.com/business/help/347766223948126
telegram also has stars, and reddit wanted to let you cash out gold, not sure what happened with that. let's also not forget tiktok's gifts. i believe instagram also has some kind of gifts but i can't find them
Twitch and twitch bits. YouTube has something similar.
Twitter?
Even before AI images became viable, you had folks in low cost countries doing the same thing with local talent. AI might make it easier but humans in those countries can give you art for a small enough amount that scale makes up the expense enough to profit.
This is even present in fandom communities where young artists from low cost countries charge a lot less than their high cost country counterparts, leading to popularity and exposure on top of just revenue.
The arts have so few barriers to competition that it's quite hard to compete unless your art is just that novel. Most artists that make a living do it by outputting boring things like commercial voiceovers, corporate illustrations, or headshot photography.
Maybe this an effect from ai, but this article reads like straight up ai
There is plenty of human in the article. I suspect your ai-detector senses are heightened due to the content, and the writing style is mostly straightforward with a few needless flowery words, which makes it ripe for false detection.
But when I re-read it with "did AI write this?" in mind, there's plenty of stuff in there that I would find it rather difficult to get an LLM to write if I tried.
I was sent a YouTube video that was blatantly fake. Five seconds in I closed it and moved on. But I got curious.
I went back to the video and checked the channel. It has 7.4 million subscribers. It uploads 4 or 5 times a day. At least two 3 hours videos, and a couple short ones. All videos are generated with a voice over. All those that I have checked have comments with real people in them. This battle is already lost.
Morbid curiosity: would you be willing to share a link to that channel?
FWIW, I've gone down the rabbit hole of flat-earthers to try and understand what the appeal is (spoiler: it's often some variant of "if the Earth were a ball hurtling through space, that would be too terrifying")... maybe I should just stay away from this one :D
I didn't really want to link them. But here @sympa_fr on youtube. Note it's in french.
I was trolling as flat earther a little bit in my youth when the internet was more innocent. I thought it was mostly trolls and some strange people?
The villagers had a point about the scythe. Using it led to his house burning down.
Easy to hate on this, but the existence of these kinds of things is so fascinating. Hundreds (maybe thousands) of people are out there generating AI click bait because people or system somewhere allows / encourages / pays for this market to exist
They are fascinating in the same sense road accidents or fatal diseases are - some people might like to watch them, but overall there is nothing good about this.
That’s not what’s happening here. The get-rich-quick grifters started selling AI slop manuals. It’s like saying brands that spend a lot on ads are successful. Post hoc ergo propter hoc.
And the feedback loop means we’re doing fuzzing on the human brain! If we can just take the human out of the loop and let the AI post and use a genetic algorithm to tweak its content, we’ll all be getting rooted in no time.
To enrage or endear at all costs, the engagement justifies the means.
It is ironic that an article railing against people doing all they can for engagement ends with... a call to engage. Sign up for my newsletter.
Still waiting for the top notch open source video AI. Want to see those kittens squeezed in the hydraulic press. Let’s just hope it doesn’t take as long as the 4K to 8k transition.
For real or do you mean you don't want the model to tell you what you can generate?
Letting algorithms bring you slop of any kind is your own fault. wontfix.
My poor grandmother spent her entire weekend baking this AI slop and no one went to her birthday
I fear your opinion is one that most casual content consumers don't even realize is happening. They just want to consume content...
However, they also share the road with you, help decide what content/messaging gets spread, and vote with their dollar (and political vote), just like you.
While I agree with your opinion, although it may be their fault, it easily can spread to become everyone's problem.
but couldfix? noyoucouldntlol
Could this be some sort of meta training activity? Meta allows this kind of content and uses engagement as a measure of value for future training material.
> The article you've just read is one such example.
There is no way to verify that. Until a method exists to discern real from AI-generated content, we should probably assume it's generated (for our safety).
Sadly, that's the world we live in now. It was kind of like that before, but it's even more so now.
I dunno, his cake baking dog looks pretty honest to me.
Deepfake Detector by Mozilla can assist
https://addons.mozilla.org/en-US/firefox/addon/deep-fake-det...
How well does that actually work in practice? I was using the demo on Fakespot's site and verbatim output from ChatGPT seems to get detected pretty well, but random HN comments got a lot of (presumably) false positives. "AI detectors" are kind of a crapshoot in my opinion, but I'm curious.
1 reply →
I don't think it matters whether it was created by an AI or an advertising agency or a dude in his basement. What matters is how powerful it is.
The powerful stuff will play your little mind like a piano. One glance and you're lost.
I agree. It doesn't matter. Even if a human wrote it, how does that change anything? It certainly doesn't make it more true or a more honest reflection of somebody's feelings. These people who keep worrying about AI generated content don't seem to realize the rubbish that humans create is not better.
Hard agree. Kony2012 was human made (or at least human started) and then it went on and spread like wildfire by using useful idiots along the way...
Seriously, the article itself feels like it was at least influenced in its style by AI. No surprises.
As I read I was thinking this article had been AI-generated. It's too empty, a lot of descriptions of the same images it had just shown, and then some commentary about how they are fake, then repeat and repeat again.
Now I'm confused, was it AI-generated or not?
> a lot of descriptions of the same images it had just shown
This is an accessibility thing that seems to be very popular on Mastodon and I guess the author put it in the prose directly instead of as mouseover text.
It helps for people using screen readers, people who are in text browsers, people who don't load images by default, and funny enough, AI scrapers.
Author here. I definitely wrote it myself. As another commentator says, this is an accessibility thing. Individual images have alt tags but the Ghost gallery component didn't have an easy way to add those.
I had the same thought. An ai-generated blog about ai-generated fake content getting upvoted on hn with decent engagement. Here we are.
I would bet it’s ai generated.
I adopted this panda. Could really use some help but he brings me joy
https://civitai.com/posts/12371259
You don't need AI to attach bullshit titles to photos. The problem is with Facebook algorithm that recommends content based on"engagement" - nothing good can come out of this by design. AI just makes bullshit makers more productive like the rest of us.
>The problem is with Facebook algorithm that recommends content based on"engagement"
Ye it is pure toxic and made me stop using Facebook.
At some point in time a was fed "people falling and hurting them self a bit" videos. And I hovered over them and tried to find the blacklist button. So Facebook started to feed me more of those! And a hate those kind of videos.
>Bizarre as it may seem, some people actually send real money to these "creators"
It's also just plain ol' ad revenue as well. Not such thing as bad engagement.
>And then there are opportunities to sell "guest posts" to other spam merchants who want to get their content in front of gullible eyeballs.
Yeah, that too. get ad revenue, then become the ad yourself.
and... the article ends. Well that was a lot more fluff than I anticipated. Yes, not much will change until the people themsevles start rejecting this content en masse. But there will be people falling for it. There are still people falling for old school email spam, so it's sadly not shocking AI works for the non-discerning viewer.
hard to put a finger on it, but i do think you should give humanity a little credit... it's actually somewhat easy to spot the "AI" when it does appear in these situations? or maybe i'm just deluded.
I can only imagine that the "AI" look must be somehow intentional.
I recently discovered how to make very realistic images - you simply ask for your image, then you use the follow-up prompt "make it more realistic" and, most shockingly, it actually works. I never thought of doing it before because it's so stupid.
It's probably like how scam emails often have obvious misspellings in order to intentionally filter out anyone incredulous enough to notice.
Critical thinking needs to be taught in schools. Either that or the society of the future is going to be hugely screwed.
If we teach kids critical thinking, they will not only be able to recognise scam, they will also stop being religious, which is another good thing.
I wonder why we still don't have critical thinking in schools.
Teaching math is a great foundation for learning how to think. Logical fallacies, for example, can be expressed as math.
Math teaches nothing in terms of critical thinking.
1 reply →
Critical thinking is taught in schools though. People in the real world don't, by and large, fall for this shit. The very small fraction that do ends up being a rather large number in absolute terms when spread across the whole world though.
Critical thinking doesn't make people less religious nor should less religious people be a goal in society.
I struggle to explain, but I feel like our data-obssessed society has completely thrown out the baby with the bathwater when it comes to these things. No one labels themselves with a religion or political party because there is a flaw in each one and anything with a flaw can't be correct, scientifically, so we just don't believe in any grand purpose to our lives, don't believe in any world leaders, don't believe in any shared tenets, and basically are all lonely and weak (because we abandon every group with a flaw).
The result of critical thinking shouldn't be a worse off society...
We don’t reject religion because of “a flaw.” We reject it because the fundamental basis of it is unsupported. It’s not “a flaw” when a house has no foundation. There is no baby, only bathwater.
24 replies →
I think before we teach critical thinking we may have to provide food and shelter security for the students.
Critical thinking won't save you. Emotion trumps rationality. This stuff plays on emotions.
...which critical thinking skills help you to notice, so that you are less likely to fall prey to it.
1 reply →
When it comes to images like those in the article, “Is this real?” is not really part of my reaction. Neither is “poor thing” or “wow that’s impressive” or whatever else.
My reaction is “why is this useless shit being shoved in my face?”.
I don’t care if it’s real, the only reason I visited the website is to check if the local market’s on this week, and maybe see if anyone I know has posted anything (increasingly unlikely). I think in the modern age it's healthy to have a much wider cynicism - what is this crap, I didn't ask for it, f### off.
I don’t really get why Facebook tries so hard to get me to look at this rubbish. The more of this shit-shovelling there is, the less often I go and the fewer friend posts there are. It’s becoming a dead platform.
“why is this useless shit being shoved in my face?” Because you allow it to.
That's not really a useful answer. I'm looking for the motive, which (when it's something you're not directly looking for) is usually money.
I fully understand that "stop using the service" is an option.
3 replies →
The technology of mind-control is advancing at a furious pace. These AI generated images and videos are just the latest evolution.
Who is vulnerable? Who is immune? What will its final form look like?
What do the scifi prophets say?
(This vast irresistible mind-control machinery serves the billionaires of course)
Poverty is probably your best shield. Because then you can't afford a phone. Someday the universal suicide order will drop and the only people left will be monks and beggars.
> Poverty is probably your best shield.
I am betting on relationships with people, and avoiding public Internet, except for shitposting on here.
This can be modelled as the back half of a whalefall. The Internet used to be a magic place where you could just stumble into community, useful information, etc. Now it's been over-exploited and game is becoming scarce.
The information is saturated with crap and mass media was never a substitute for friendships. Even though it's the hardest thing I've ever done, I'll have to make real friends.
I upvoted you.
What you wrote sounds a bit "out there", but, like Margaret Atwood's work, it's actually not too far away.
> Who is vulnerable? Who is immune?
I'm reminded of Hiro Protagonist (living in a storage unit), or Ready Player One protagonist (living in some impoverished mobile home stack) -- these are the "vulnerable".
As for "immune", it's the people who "control the supply":
https://www.cnet.com/culture/all-the-ways-people-freaked-out...
P.S. Like all real-world examples, we have an exception -- Felon Husk, who gets high on his own supply, making him simultaneously victim and perpetrator.
Wow amazing article