Comment by carlgreene
17 hours ago
I have largely written Reddit off and no longer visit it after an experiment I did where I had an agent karma farm for me and do some covert advertising. As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer. Many many people (or other bots) had full on conversations with it and it scared me a bit.
I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
For a while there were a lot of posts from people experimenting with ChatGPT to write anger bait posts on Reddit where they would later edit the post to say it was fake, written by ChatGPT.
I assume they thought they'd be teaching people a lesson by making them feel foolish for responding to AI stories, most of which were too fake to be believable.
However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake. In advice subreddits, commenters continue to give advice on the situation. Some comments would say they saw the notice that it was fake but continue arguing about it anyway.
This makes a feature of Reddit very clear: The truthiness of a post doesn't matter. The active commenter base on popular subreddits just wants something to discuss and, usually, be angry about.
In retrospect it's obvious given that misinfo posts were the easiest way to karma farm for years even before AI.
We do precisely the same thing here. Here's a relatively recent post that, to me, seems obviously LLM-written. It just rattles off some management platitudes:
https://news.ycombinator.com/item?id=47913650
It had 639 comments and 866 upvotes. And that's not a one-off.
Even the title is in "x, not y" format.
Sufficiently advanced "AI" is indistinguishable from a linkedin true believer koolaid drinker middle management type.
I wish there was an internet-wide "don't show again" button for such slop pages
3 replies →
>However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake
That's 90% of current Facebook pages and groups.
The decline of Facebook is sad. I liked it early on. I used it primarily to follow family and casual friends from high school. When they posted, it would show up on my feed, I read all the posts, and that was that.
After awhile I had to wade through all sorts of nonsense to get to the posts I actually wanted to see, and even later Facebook stopped putting posts from people I follow in my feed. It was 100% garbage. I can't imagine why anyone uses Facebook for anything other than the marketplace.
6 replies →
Even without AI slop I've noticed this happen on Reddit.
I once made a rather boisterously-argued comment on a political issue I'm passionate about, and I realised that I'd made a serious error of reading comprehension when it came to my opponent's argument. I apologised to them for being an abrasive arse over my own mistake, then edited my comment to say that I was mistaken.
My incorrect comment which literally said at the bottom it was incorrect continued to be upvoted while my opponent who had made the stronger argument continued to be downvoted.
I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust. Or rather turn them into little better than comment sections on news sites; thriving but worthless.
I'm active in a number of online communities that are doing just fine but the difference is those all involve ongoing relationships, built over time and with engagement across multiple platforms. I've no doubt this clock is ticking too but it's still harder to fake a user across a mix of text chat, voice and video calls, playing an online game, etc and when much of the web of relationships extends back into real life activity.
But I agree the golden age of easy anonymous connections online has ended.
Note that "attestation through a web of trust" means something like needing an invite from an existing user. It doesn't have to mean mass surveillance.
Private torrent trackers have been doing this for a while. If some number of your downstreams act like shitheads - you get nipped and so do your other downstreams.
7 replies →
PGP’s web of trust was kinda bad privacy-wise in some regards, as it basically revealed your IRL social network.
If my PGP public key has 6 signatures and they’re all members of the East Manitoba Arch Linux User Group, you can probably work out pretty easily which Michael T I am.
Are there successful newer designs, which avoid this problem?
3 replies →
> Note that "attestation through a web of trust" means something like needing an invite from an existing user.
It's probably better to call this something like vouching and leave "attestation" as the contemptible power grab by megacorps delenda est. The advantage in using the same word for a useful thing as a completely unrelated vile thing only goes to the villain.
Then how can you have a community that is welcoming to people who are not part of the ingroup?
I want to create a community for immigrants. How would I make it welcoming to recent immigrants for whom no one can vouch?
A web of trust is a wonderful tool, but it's exclusive by design. This is a problem for some communities, even though it makes others much better.
10 replies →
Which is, funnily (?) enough, how a lot of IRL organizations used to be. And basically don't be of the wrong ethnicity or religion.
It still happens more informally today, of course, but it used to be a pretty (if un-spoken) part of how a lot of WASPy organizations operated to a greater or lesser degree.
4 replies →
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
This seems self evident to me too.
It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
Tell your TPM who you are and prove it with face and fingerprint ID that get matched to a real old person.
Leave them on the device, authorize the device to validate before age inappropriate content appears.
Website wants to know your age? Your face and fingerprint support your attestation signed by a trusted party.
Can it be tricked potentially? Sure, but then you’re probably a super genius kid and not the reason that these laws were created (as if).
Don’t let anyone tell you anonymity must die for safety to exist.
EU's ZKP implementation provides complete anonymity and untrackability:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
2 replies →
> It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
The problem here is that the premise is the error. "Prove your ID" is the thing to be prevented. It's the privacy invasion. What people actually want are a disjoint set of only marginally related things:
1) They want a way to rate limit something. IDs do this poorly anyway; everyone has one so anyone so criminal organizations with a botnet just compromise the IDs of innocent people -- and then the innocent are the ones who get banned. The best way to do this one would be to have an anonymous way for ordinary people to pay a nominal fee. A $5 one-time fee to create an account is nothing to most ordinary people but a major expense to spammers who have 10,000 of their accounts banned every day. The ugly hack for not having this is proof of work, which kinda sorta works but not as well, and then you're back to botnets being useful because $50,000/day in losses is cash money to the attacker that in turn funds the service's anti-spam team, but burning up some compromised victim's electricity is at best the opportunity cost of not mining cryptocurrency or similar, which isn't nearly as much. It would be great to solve this one (properly anonymous easy to use small payments) but the state of the law is a significant impediment so you either need to get some reform through there or come up with a creative way to do it under the existing rules.
2) You want to know if someone is e.g. over 18. This is the one where people keep pointing back to government IDs, but you only need one piece of information for this. You don't need their name, their picture, you don't even need their exact birthdate. Since people get older over time rather than younger, all you need to know is whether they've ever been over 18, since in that case they always will be. Which means you can just issue an "over 18" digital signature -- the same signature, so it's provably impossible to tie it to a specific person -- and give a copy to anyone who is over 18. Maybe you change the signature e.g. once a day and unconditionally (whether they require it that day or not) email all the adults a new copy, but again they all get the same indistinguishable current signature. Then there are no timing attacks because the new signature comes to everyone as an unconditional push and is waiting for them in their inbox rather than something where the request coincides with the time you want to use it for something, but kids only have it if an adult is giving it to them every day. The latter is true for basically any age verification system -- if an adult with an ID wants to lend it to you then you can get in.
3) You want to know if the person accessing some account is the same person who created it or is otherwise authorized to use it. This is the traditional use of IDs, e.g. you go to the bank and want to withdraw some cash so you need a bank card or government ID to prove you're the account holder. But this is the problem which is already long-solved on the internet. The user has a username and password, TOTP, etc. and then the service can tell if they're authorized to use the account. It's why you don't need government ID on the internet -- user accounts do the thing it used to do only they don't force you to tie all your accounts together against a single name, which is a feature. The only people who want to prevent this are the surveillance apparatchiks who are trying to take that feature away.
I'd be interested in working on a problem like that.
I have a strong preference for remaining anonymous or at least making it a reasonably high bar to tying my online identity to my personal identity
I would love to be involved in helping to design a sort of "human verified" badge that doesn't necessarily make it possible or at least not easy for everyone to find your real identity
I've been thinking about it a bunch and it seems like a really interesting problem. Difficult though.
I suspect there is too much political and corporate will that wants to force everyone online to use their real identity in the open, though
28 replies →
I'm in many public chat communities as well and the issue whether someone is an AI or not is not really coming up, I've not seen any actual AI chatters and the only AI spam that exists is the one that humans regurgitate. The more real impact AI has on chat communities in my opinion is that people are shifting some of their chatting to AI bots via voice or text on other platforms, resulting in fewer chatters.
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
I'm happy to verify my identity as an honest-to-god sack of meat if it's done in a privacy-protecting way.
That probably is where things are gonna go, in the long run. Too hard to stop bots otherwise.
In order to make this viable, wouldn't you have to verify identity repeatedly? What's to stop me from providing a valid identity and then handing my account over to an agent after I'm verified?
6 replies →
I guess it would have to be something like a service which confirms whether a person already has an account on the site but doesn’t have to track which particular account it is.
I’m not sure if that would work for account deletions though.
That is effectively impossible though. There's data centers of stripped down phones, so "it's actually a phone" doesn't do it.
1 reply →
What's stoping bots to verify identity? This will not work, especially with frequent data breaches.
> without either proof of identity or attestation through a web of trust.
Let's put aside the idea whether it will be the end of all privacy as we know it (I'm not sure if I personally think it's a good idea), but isn't Sam Altman's World eye ID thing supposed to do that? (https://world.org).
How does it work (like OpenId)? Do I have an orb on my desk, or some sort of phone app? I still want to use my desktop to login to HN.
Would it stop this sort of "get human id", past it into .env, so agents can use it?
this eye thing will never work. people in general are realizing the last people we should trust with our personal stuff are tech bro billionaires. they’ve broken trust too many times.
even worse many of them are just plain vocal about their disdain for people in general.
at least from what i’m seeing, people are starting to walk away from online at an increasing rate so i definitely don’t see widespread adoption of his creepy eye thing.
1 reply →
Im not sure proof of identity solves anything. People will still have LLMs with their real identity verified.
I’m imagining like, a physical place you would go and get your text spoken out of your personal speaker directly into someone else’s microphones.
Personally I think we need to start utilising the safety features built into AI, to ensure that who we're talking to is a human. We'll start to have to only reply to people who talk in nsfw cursewords (like cocks), or profess their love of capybaras
LLMs can curse without issue
5 replies →
Who doesn't love capybaras?
>I think it's going to effectively kill public chat communities without either proof of identity
How? I have an identity. A state driver's license, birth certificate, social security number. I've even considered getting a federal license before, never bit the bullet. If I wanted to run a bot, what stops me from giving it my identity? How do I prove I'm really me (a "me" exists, that's provable), and not something I'm letting pretend to be me? You can't even demand that I do that, because it's essentially impossible.
Is there even some totalitarian scheme that, if brutal and homicidal enough, could manage to prevent this from happening (even partially)?
I'm limited to a single identity only as a resource constraint. Others more wealthy than I (corporations or ad hoc criminal enterprises) could harvest thousands of real identities and use those. Consensually, through identity theft. The only thing slowing it down at the moment are quickly eroding social norms (and, as you point out, maybe they're not doing that and it's not even slow at the moment).
Digital totalitarianism would prevent it. The moment you were found to be running a bot, your identity would be blacklisted across the entire internet.
2 replies →
It'll come back again once ZKPs become standardized and become baked into devices:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
I personally can't wait for a mechanism to kill 99% of bot traffic.
"I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust."
Those sorts of places were always the only places with reliably good communities.
To the contrary, platforms like Facebook and X demonstrate that even personal verification won't save you from identity politics.
4 replies →
The web could become a way to indicate identity if public institutions publish for example www.university-country/professors/John. And that implies that John is a professor. I designed a 6000 lines protocol, but anyone could construct that web using hmac(salt+ url).
Reddit is more or less dead to me, as the popular subs are botfests and the niche subs are empty. I'm lucky to get a single reply on gaming subs.
The fact that reddit enabled hiding your posts is crazy to me. In a time where knowing who's engaging in a community is more important than ever (am I talking to a bot or a troll?) reddit removes even more options to validate.
I interpreted that as an attempt to mask the number of bots on the site so as to not scare paying advertisers into thinking their ads won't be seen by real humans.
1 reply →
The problem is that it has become very popular to ban people from a sub based on what other subs they post to. It was turning Reddit into a two-party universe.
The better fix would be to make the support for multiple accounts in the reddit app not so incredibly-shitty, where you're basically logging out and logging back in. Instead, just tell it "posts to this sub use this account, posts to that sub use that account", etc.
1 reply →
I enabled hiding my posts because I kept getting harrassed and even doxxed.
1 reply →
There's also a third category where the sub looks organic because the moderator deletes and bans anyone who doesn't post exactly what the moderator wants.
Wait isn't that every sub? /s
Plenty of good subs they are just under the radar. Once something gets more than about 10k users the quality sinks.
I also believe it is used by AI companies to train their models: Post something semi correct (even grammar issues..), wait for humans to correct it in the comments and used upvotes as a confidence indicator, and then retrain models on this free refined data. Meanwhile people think they read a legit post, feel certain emotions and influence their behaviour, just so a bot can be trained.
This is actually my hope for AI-gen content as well. That after it gets so 'good' that people genuinely can't distinguish it from reality anymore that they'll retreat (or return triumphantly rather) to the physical world to gather truthful fulfilling experiences and dopamine.
My hope as well. If AI doesn't kill us all, the real world, with all its dirt and grime and beauty, will become the only thing that can be trusted
I've thought about this a lot as well and could definitely see it happening.
The issue is that the physical world for someone in the hinterlands of Tibet is not the same as the physical world for someone in SF.
People were finding each other online when they couldn’t in person.
This isn’t to say I disagree with you. Just expressing sorrow over the loss of such a grand moment in our shared history.
Serious question: If there are so many LLMs on online forums, who is doing it? Is it just 1000s of research students or something more nefarious? Is it AI businesses building up evidence that their output is as highly scored as humans therefore "buy our software"?
We're in the middle of an active cold war where countries are trying to manipulate the citizens of rival countries to destroy their civilization without having to fire a single bullet. Anonymous, over the internet mass manipulation, all for some minimal electricity cost.
That's definitely the most insidious use, but I think the larger portion is advertisers and karma farmers (who later sell to advertisers).
6 replies →
AI is particularly bad at this, and regimes that employ tactics generally are not short of labour to have humans to do it.
If AI is being used in these areas it is less as an attempt to manipulate as it is to just create noise and engender distrust in what they hear.
Established accounts are worth money, often for scamming/propaganda.
Not too dissimilar to people bot-leveling in MMOs to the sell the accounts.
It's very common for folks to search Reddit to find reviews of products etc. these days. If you can have a bot account post a fake review of how awesome your product us, and have that upvoted, it can pay huge dividends.
I've noticed 4 categories of inauthentic users. Ranked by my perceived prevalence:
Account farmers: these can be people in 3rd world countries automated/not automated. Can be using hundreds of mobile phones to create accounts and do daily activity to make the account look legitimate. While they're building an activity history they are also being paid to like/follow/interact with content.
Advertisers: these are brought accounts that are used to pose inauthentic reviews of their service and inject it into discussion and to do PR
Sloppers: people who build AI pipelines and then just pump the most dogshit content directly into a platform trying to make any amount of money.
Nation State propaganda arms: These accounts build a narrative character and then join discussion pushing a certain narrative, boost real content creators who share their message and bog down discussion.
People like the above poster who are "just running an experiment" or "trying something for fun" who then wonder why online communities are full of AI now.
In the case of Reddit and HN a lot of it is done by businesses either blatantly advertising themselves or building up the karma they need to effectively do so. I recall reading obviously AI generated replies to news articles written by accounts associated with businesses related to the events in the news. This isn't new in the LLM era. Hobby subreddits are well known to be always full of businesses selling hobby gears and items doing self promotion. It's just that now it is a lot more obvious because of the AI text smell.
That, and probably political astroturfing. Before every election my local subreddit sees a surge of crime stories. Go figure.
I think some of it is account farming, but some is just people buying wholesale into the idea that if you're not using AI for everything, you're gonna be left behind. On the Kagi Small Web list, there's plenty of hobby blogs that used to be normal pre-2023 and are now obviously LLM-written and AI-illustrated. There's also plenty of people on LinkedIn who post AI slop because they think it helps them build a "professional brand". I even have some distant friends who are using AI for responding to friend & family posts on Facebook just because it makes you seem... smart? engaged? I don't know.
It's actively encouraged by some of the platforms too. In Gmail and Google Docs, you have incessant AI prompts along the lines of "help me write this". I think LinkedIn does the same.
HN has historically been gamed for visibility. The stakes for doing this can be quite high if you can pull it off.
Lots of marketing. Not even AI business, just regular consumer crap. They realized that blatantly spamming their product looks bad, so they orchestrate multiple accounts to look more organic. And people actually engage with it.
My impression is that they're sometimes unemployed people or students hoping to create a popular open source project, and use it to find a job.
They aren't going to care about any of the advice in the article about not posting slop -- finding a job is (of course?) more important to them.
Can't really say they are doing anything wrong, maybe I too would have? ... Just that large scale, doesn't work
There are many reasons for influence campaigns, that isn't new. Influencing the public is incredible valuable; that's why so many invest so much in it. LLMs automate it like never before.
Plain advertising, governments' propaganda, political propaganda for one group or another to shift public opinion (it's done on TV networks, why would they not do online campaigns?), astroturfing by corporations promoting acceptance or fighting negative news (e.g. rideshare, AI, whatever certain wealthy personalities are doing) ... the list goes on.
HN has always been relatively influential in the tech industry and therefore worth influencing, and now the cost is very cheap - you don't even need to hire many people, so less-resourced operators will find it worthwhile (and they will also attack lower-value forums).
If you farm a fleet of good accounts, you control the discourse. On HN, you could boost whatever you're trying to push, and downvote or flagkill whoever objects.
There are obvious benefits to controlling public discourse, right? Even if it's just to support some project you're working on.
There are certain topics that seem to get instantly flag-killed unusually often. IPv6 is one.
3 replies →
Did you ever introspect about who ruined Reddit?
Isn't the ceo a pdfile and compromised and forced to work at reddit (or go to jail)? Reddit is now just a propaganda machine for the intelligence agencies and their dirty ceo is there to make sure the machine keeps pumping honey...wrecking teenagers brains in the process too, and gathering kompromat on young people which will bear its fruit in the next 20 years. I feel a good chunk of US politicians are being blackmailed because of their past online activities. Same shit on 4chan, how can it possibly be allowed to exist except for being a honeypot, all of these site dodgy sites being guarded by cloudflare no-less, which is the ultimate man-in-middle machine used by "them".
I think the real explanation is simpler - it's just not particularly interesting to the authorities. No need for conspiracy theories.
As to compromising material for bribery, that can be collected in so many different ways, and things like email or messaging or tiktok videos are probably far more interesting, reddit is not particularly useful for that.
1 reply →
It’s a tragedy of the commons, many have done it, but no one user did it.
I'd argue that Reddit leadership, which insulted, hobbled, and wrote off its mods and power users (destroying projects like /r/BotDefense) while doing little to crack down on the proliferation of bot repost content, had a major role in encouraging this. They might even like it better this way -- lots of extra fake engagement boosting traffic stats without messy human drama, which they can then ironically sell back to AI labs as training data.
13 replies →
We’re all trying to find the guy that did this
Reddit itself by virtue of being a venture capital backed startup.
It was a midpoint between Facebook and Geocities, it got people to build communities within its walled garden, but it was always going to betray them for cash.
Yeah, if carlgreene specifically stopped doing that Reddit would be saved. They are the one savior.
They directly contributed to the problem that they say forced them to leave Reddit.
Do you sincerely believe that that's how grey-area's comment was meant to be read?
2 replies →
Directly my fault. Specifically me. No one else is to blame.
[dead]
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
Would be super fascinating to watch play out. I grew up before the internet so, historically, I know how to seek out external communities, but by early high school I was deeply entrenched in online life - so I'm very rusty with finding new IRL clubs, cliques, etc. Fortunately my life is full of many friends and I go out frequently, regardless. For those younger people that never had life without the internet, I wish them luck on their search but at the same time I'm very curious to witness their journey.
Yes when I interact on reddit, I normally do so solely with the intention 'this is for an LLM'. I feel like a majority of the posts/comments I reply to are AI, a majority of the responses to my posts are AI, but have to keep telling myself to keep posting so it becomes training data.
(I'm normally posting in the context of my startup - although I try to keep the self promotion to a minimum and always contribute to the "conversation," if LLMs replying to one another can be called such).
For what it's worth, I created a community for paying users of Phrasing that has been going really well. I think free online communities may be going away, but there may be a future in exclusive/paid communities.
Public* online communities are dying. Discord is thriving
If all you value is sub-IRC level irreverent discussion, maybe.
Discord is far better for discussion than IRC. You can be much more expressive on discord, instantly jump into a call and screenshare, easily link people to other rooms, tag, import bots etc. IRC kinda sucks compared to modern chat and they refuse to implement features that are considered basic.
1 reply →
This. Everything important has moved to discord. Which is sad because of how undiscoverable and unsearchable it is.
I'm more sad about how the UI of it all is just clunky. Even though it resembles ye olde IRC clients like mIRC, nowhere near readable for some reason.
1 reply →
are those attributes now assets?
5 replies →
I got banned the other day from the Stellaris Discord server because someone accused me of hacking Roblox accounts. I’ve never played Roblox in my life. So that’s nice.
This shit will come to Discord too.
2 replies →
Discord is terrible. Full of bots, creeps and ai slopped to the gills.
Some communities are better than others but the sheer volume of stinky trash is immense despite discord and the poor volunteer moderators efforts to prevent it. Most mods are neutral on it too.
There are chat communities that are still somewhat safe with zero user verification. But I will not mention them.
discord is a tool for hosting private chat servers. it's pretty neutral. the UI is not great for building a shared knowledge base, although people do that anyway
but yes the publicly accessible servers are going to face similar problems. the socially competent people tend not to run those servers, and have smaller private servers with people they know as they have no drive to try to create a space for strangers to gather.
I really don't understand the folks fleeing to Discord. A mailing list does 99% of the same thing for most of the communities.
Sure, if you want to chat while gaming, that's the whole point of Discord. Ganbatte.
But, for everything else, Discord is such a horrible misfit that I don't understand why it's the default.
i predominantly use it for real time chatting, its a big group text chat and a place to hop in a voice channel and shoot the shit while doing whatever we want on the computer a la ventrilo/mumble/teamspeak
but yes i also game and it gets a lot of use for that as well
i agree though that for collecting and organizing information longer term like forums do, it is not ideal
You are booming out. I cant believe suggesting a mailing list
1 reply →
> I don't understand why it's the default.
Because it equally well supports real-time communication.
And it looks shiny.
And some people use it to e.g. watch a video together, or other social purposes.
Reddit has had a bot problem for well over a decade now but the sheer volume of it has exploded. It is also much more difficult to tell nowadays as the "quality" if you will is now at the good enough stage.
Alas, Reddit is basically dead to me because of this.
There's this old meme where someone asks what will happen when AI bots posts helpful, curious and thoughtful messages!? That's mission accomplish :D They can't be better then the average human though because of training data, so I don't worry about AI comments getting up-voted by real humans, I am however worried about fake upvotes.
> They can't be better then the average human though because of training data
Is this based on the belief that an LLM can only represent an "average" human being?
If posting good messages is automated then the AI will post a good question and another AI will answer it and the humans will look and see nothing extra to contribute.
It is not a meme, it's an xkcd: https://xkcd.com/810/
Reddit sold it's data to AI companies for training[1]. They could have refused, but companies like OpenAI likely would have harvested that data anyways. As such, it should not be surprising that AI models are pretty good at generating reddit posts. They were specifically trained to do that.
This is sad, because Reddit remained one of the final bastions of human content on the internet. For several years, appending "site:reddit.com" to a google search was a valid way to get something usable out of a google search. Doing that is still an improvement over raw-dogging Google's ranking algorithms with an unfettered search, but AI slop increasingly is the result.
This is one of my great disappointments in the current rise of AI. LLM's can give good search results when dealing with a topic they've been specifically trained on by human experts, but they're not good at separating human-produced signal from AI slop noise. We've done nothing to prevent a sea of AI slop from being dumped on top all the human signal that's out there. When AI companies enter their enshittification phase and stop investing in expert human trainers, the search results LLM's produce are going to fall off a cliff. Search is a bigger problem than ever.
_____
[1]https://9to5mac.com/2024/02/19/reddit-user-content-being-sol...
Doesn't help there is that feature that hides the user's posts and comments
> I do know for a fact that many "users" here are LLMs.
HN autokills comments it detects as LLM. I think maybe you're not giving HN enough credit. :)
HN front page is about 25% LLM written blog posts at any given time.
There’s no rule against submitting LLM-written—ahem, “cleaned up my notes”—articles, just comments.
2 replies →
HN kills lots of posts. I try to be careful about my online footprint (since HN posts are forever), and try to switch to new accounts every so often. It's no use anymore, HN just kills any post I make from a new account, even when I spend 20 minutes researching a response and trying to get useful information.
It doesn't even show you the post is killed, it looks to you like it posted fine, and you have to logout to see it's actually dead. It's an approach that's extremely hostile to the user.
It's specifically against the guidelines to keep registering new accounts, and this is a good reason why. We have to have ways of determining credibility and authenticity, now more than ever, and a track record of good posting is one of the best ways to do that. We are drowning with spam and low-quality posts/projects posted from brand new accounts. If it's a well-researched, high-quality post, of course we want to give it exposure. We just have to be realistic about what we're up against.
1 reply →
I have read enough “you are replying to an LLM” comments that I am pretty sure this is still a hit or miss process.
Why do you think those comments are accurate? Maybe those comments are by LLMs? If you believe crowd wisdom on its face, you will have big problems with LLMs.
It needs help. I often pipe my screed though an LLM and post it. I do request that it use a 10th grade reading level, and no emdashes.
For giggles, here's how it would look for this comment. Rather meta, but in this case it removed the "It needs hellp" so here we are.
I often run my screed through an LLM before posting. I ask it to keep the writing at about a 10th grade reading level and to avoid em dashes.
>HN autokills comments it detects as LLM
No it doesn't. Unless you have proof.... ???
The question is how reliable that detection is.
There was a post today that Google introduced unbreakable capture that required unrooted phone to pass its QR code.
We may end up with things like that…
I find it amusing that this is the top comment. Reddit is so awful you finally wrote it off, but not before you used it to try to “karma farm and do some covert advertising”. It’s on-brand for HN hypocritical bullshit. But, since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard, have an upboat fellow traveler.
> since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard
Same as it ever was.
Reddit was already on its way way before this LLM craze, hopefully the recent tech-related changes will only accelerate that process.
> As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer.
I don't suppose you could show some examples? How convincing is the state of the art now?
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
You can have both IRL and online-free-of-bots. I already wrote about it but one of the very best forum I'm a member of, where real people are posting, requires to be vetted in, web-of-trust (but IRL) style. It's a forum about cars from one fancy brand and you can only ever join the forum by having a member (I think it may be two, don't remember) who's already in confirm that he saw you driving a car of that brand. It's not 100% foolproof (someone could be renting the car for two hours and show up at a cars&coffee or take a friend's car etc.) but this place really feels like a forum of yore.
And people do eventually travel, so it's bound to happen that an owner shall go to another country, meet someone there, vet him in etc.
Now, sure, it may not be the "1 million users acquired in three days thanks to my vibe-coded app" scenario but that is the point.
You can imagine other domains where IRL communities have local groups, but where forums regroup different IRL communities all interested by the same hobby/topic/domain. And when people travel and meet, the vetted members do grow and connect.
Oh and on the forums a lot of the posts are pictures, where "Julian xxx" met "Black yyy Cyril" and you see both cars (and from more than two people): suddenly it becomes much harder to fake a persona... You now need to fake both Julian xxx and Black yyy Cyril and fake the pics. And explain why your car has never been posted by any carspotter on autogespot etc.
You can imagine the same for, say, model trains: "Met Jean at the zzz meetup, where he brought his wonderful 4-8-8-4 'big boy' locomotive, I confirm he's into the hobby, vet him in".
Naysayers and depressive people are going say it cannot work but I'm literally on one such forum and it just works.
P.S: if I'm not mistaken in the past in some nobility circles you had to be vetted by up to sixteen (!) other people from the nobility that'd confirm they knew you, your parents, etc. before you'd even meet the king/emperor/monarch to make sure that someone from far away couldn't come to, say, Versailles or Schonnbrun pretending to be a baroness or count or whatever. Quite the extensive check if you ask me.
Unless their account is <1 year I wouldn't assume they are a bot.
Reddit astroturfing firms and bot farms learned to buy/use “seasoned” accounts over a decade ago. I’d venture there have been countless bots just in a holding pattern harmlessly building up reputation and a human-like history of posts across different subs etc just to eventually be either activated or sold to someone else to “burn”
It used to be super common that when you spotted a bot post and clicked through to the user's history, you'd see very average, human-looking activity from years ago, followed by a long gap of inactivity, and then a flurry of obvious bot comments.
It's very obvious that these accounts were abandoned and then either bought from their original owners, or more likely bought from someone who compromised them, because of their history and karma.
And I would bet money that Reddit is well aware of this phenomenon, because not long after it became so common as to be impossible to ignore, they papered over it by allowing users to hide their history from public view. (AFAIK subreddit moderators can still see it, but typical users now have much less ability to see whether they're interacting with actual humans.)
2 replies →
I recently spotted one unmistakable example of this[0]. It’s been a trick for many years now that duplicating a human post and its comments is a good way to appear human but this was quite the example.
0: https://wiki.roshangeorge.dev/w/Blog/2026-01-06/Is_The_Inter...
1 reply →
So what is the comment frequency of these bots? There must be some signal in the activity even if the comments themselves pass the turing test.
4 replies →
So easy to purchase online accounts nowadays, neither karma nor age of the account matters anything anymore.
IRL communities have to have some guides because a lot of people forgot how to gather. It can be seen among kids - try to give them soccer ball and see what they do with it :)
Yesterday I was watching people on the street and on the tram. Every other person was staring at their phone and scrolling through something.
That might scare me more than the fact that someone is chatting with an LLM bot online.
(I am pro-ai, use it every day for coding that I couldn’t achieve pre-2022 as I am lame coder.)
I don't really see the problem of using your phone while commuting. Doesn't make you an asocial weirdo.
As long as you're not sharing the things you're watching with a loudspeaker. And that's really not a given among commuters.
How do we know now that this comment wasn't written by LLM?
You don't and that's the problem :)
It's easy to botspam Reddit because even the real users always acted like bots. The big subreddits were the worst, but contrary to how the users keep saying "it's good if you find the right subs," no it's not. Wrote that place off like 10 years ago.
I feel you. Especially in the larger subreddita. i participate, and mod, a few small ones, and the community there is pretty strong and folks shut down ai slop pretty quickly.
I'm not saying being a mod means it's bullet proof, but i do notice smaller communities tend to self police better and know what's real.
That said, your experiment scares me as well.
I will say that I believe you probably have absolutely no idea because it's not "slop". It looks like every other reddit comment you see.
My experiment was focused on niche subreddits as well due to the nature of the product I was trying to market.
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs
People using LLMs without being fed their own post history are still pretty easy to detect. There's just something very recognizable about the cadence and tone of LLMs.
What really stuns me is that if you call someone out for it, 9/10 times you get absolutely buried in downvotes. Even here on HN. Its like people are angry that you're lifting the curtain on the slop, that the writing they enjoyed is fake.
Communities in FB, WhatsApp, Telegram etc are actually flourishing. As it appears real time gated communities are doing fine.
It’s an unpopular opinion but I am looking forward to ID and age verified social media. If done right we can have real people around again.
BTW, ironically the harsher communities like 4Chan doesn’t seem to suffer from the dead internet. I guess it’s either because the advertising value is too low to justify AI use there or maybe AI API providers refuse to work with such a content this reducing opportunities to infest with bots.
I wonder, how much of the discussions on the results of agentic coding is just LLM slop.
More of a philosophical question but if you have no idea whether it's a human or robot, does it really matter? Personally I dislike AI slop only when I can tell it is...
Yes, for a number of reasons:
- I am trying to learn about the topic at hand and trust a human's comment more than an LLM's guess - I am trying to connect with other humans to fulfill my social needs - I am maybe spending time to help another human out with a response because I want to help someone else - I am interested in the perspective of other humans
Those are just a few reasons. For each of those if it's actually an AI I feel I'm losing out on something.
Reddit users are definitely smartening up on many subreddits to the same kinds of engagement happening.
This kind of thing made me imagine the creation of "digital towns" the other day.
Imagine an online community where you can only join on the recommendation of two other members, who you must have actually met in person, to participate. Meanwhile, you leave at least some of the activity publicly available to the general public so that interested parties can meet up IRL and join.
This could probably be implemented easily on top of existing online platforms like Discord, Reddit, etc. since it's really just a community building rule, not a community itself.
> I do know for a fact that many "users" here are LLMs
What factual basis do you have for that?
It might come down to shareholder/IPO stuff but you can tell Reddit doesn't actually care to put the effort in to crack down on bots (however you'd do that) because they already don't give communities proper moderation tools/third party tools and the site does censor
Whatever allegiances (with people, or allegiances to ideas) Steve Huffman has, or people like him - it's not enough. It's a site seemingly killed by greed
(Yes, I know moderating this stuff at scale is hard)
- A human. Beep boop.
Do you have an example of comments people engaged with?
On the other hand, I’ve been accused of being AI/bot and if I say things the mod doesn’t like and is not their favorite thing to hear I’m “flamebaiting” or engaging in personal attacks when pointing out specific things.
Frankly, online communities have been doing for many years now, when the censorship, anti-free-speech, tone policing mods and mobs started dominating online and America really did not have the self-respect or confidence anymore to enforce the Constitution online.
> America really did not have the self-respect or confidence anymore to enforce the Constitution online.
“Mods are Unconstitutional” lmao
> where I had an agent karma
Was this a browser using agent? What did you use?
It used the browser agent to grab user cookies after signing in, then made API calls iirc.
Using just a browser is way too token intensive and slow. It would look for 401 errors then run the browser automation to login with the credentials and grab the token.
I'm surprised these platforms don't have advanced heuristics to detect API calls and inauthentic traffic.
Did you clone the Reddit API from browser traffic and then turn it into a 100% API driven thing?
I'd imagine they'd be sniffing browser agents, plugins, cookies, etc. to fingerprint. Using JavaScript scroll position, browsing rate and patterns, etc.
Maybe their protections just aren't that sophisticated.
3 replies →
So you ran an "experiment" where you deliberately made someone else's community worse to see what would happen? Cool project.
> I do know for a fact that many "users" here are LLMs.
Name and shame.
If you look at the bottom of most threads here you’ll see a bunch of green username dead LLM comments. Those are just the obvious ones though.
[dead]
[dead]
[flagged]
Don't do it anywhere. He's a jerk for doing it on reddit.
Can’t make it worse than Huffman already has. After the API nonsense most mods who are still there should’ve left. I don’t know how most continue to do Reddit’s work for free after the way Huffman talked about us all. He made it clear how much he despises the community and the complete lack of respect he has for the people who run all the subs (I helped moderate a medium-sized sub and I miss it but don’t regret leaving at all)
[flagged]
9 replies →
People are definitely trying to make HN bots because I have seen several get flagged. No idea to what end though.
the suits or suit minded people have realised that HN is good for advertising to the kind of demographic that'll give them free labour and is easily swayed by whatever the latest trend is
1 reply →
People still see HN as a valuable place for promoting their businesses. Usually some poorly thought out SaaS.
If you look at what people outside HN talk about HN, it's not uncommon to see wannabe tech entrepreneurs talking about how to promote their products via Show HNs and how to stay HN front page. It's honestly a little sad considering that HN has a tendency to rip these projects apart.
1 reply →
Why would reddit bots exist? (In)organic advertising, same concept here.
The ones you see flagged are the very obvious bots. What about the more sophisticated ones? How do I know skupig isn't a bot?
Possibly to test reactions to a bot they plan to build a startup around.
I've seen some claim they do it to avoid stylometry or being fingerprinted, or because of social anxiety problems.
Some people just have a compulsive need to optimize everything, and HN's guidelines and tone policing are more easily followed by a bot than a human.
4 replies →
I didn’t say that people weren’t doing it. I was asking this person not to do that here since it sort of sounds like they have plans to
He's stating a fact. Turn on showed in your options and scroll to the bottom of the comments on any popular story. There are so many agentic users here.
Having to turn on showdead (which I have turned on by default) demonstrates that’s it’s not much of a problem in practice.
I generally disagree, because the level of discourse here has always been very high, curious and intellectual.
4 replies →
[flagged]
For what it's worth the admins here have let the tone of conversation slip a little when it comes to AI, as in there are many people who now openly mock (and worse) the AI zealots and there's no admin coming in and "saving" the metaphorical day anymore. In the not so distant past that kind of behaviour was almost instantly reprimanded, kindergarten-style.
[flagged]
I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political". This is somewhat related to the Overton window but really a bunch of (mostly conservative) ideas get normalized so aren't deemed "political".
I see the same thing with "AI Slop". Yes, there is AI Slop but (IME) it's pretty easy to spot. But what's more annoying is how often people are willing to throw that accusation whenever someone takes a position they don't like, much like the "political" label. It's lazy and honestly just as bad as the slop itself because it unintentionally launders the slop in a "boy who cried wolf" kind of way.
I also have a theory that some AI slop isn't inherently successful. It's just heavily botted by people who are interested in promoting certain positions. I bet you could make a pro-administration LLM bot and another one promoting a communist revolution and no amount of model tuning would make the second as popular as the first because the first would hit third-party botting as well as platform content biases (eg Twitter).
I've personally been accused of being a bot. This is particularly true in recent time as I've tried to share facts and fact-based analysis of, say, what's going on with crude oil markets, the military operation in the Gulf and the politics and economics around it. I even saw one hilarious comment saying (paraphrased) "the bots are getting clever and posting about unrelated topics". This was funny because it never occurred to this person that no, it was just a real person posting something you disagreed with.
> I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political".
This happens on HN all the time. For a lot of downvoters and flaggers, there are two kinds of opinions: "Things I agree with" and "Too political for HN."
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
This just makes me wonder...so what?
Some of the oldest posters here with the most karma continue to post absolute garbage takes on topics ranging from US healthcare to history of USSR, that are trivially disproven by learning the very basics from a Wiki article (e.g. not a high bar).
To be fair, this opinion slop is also present for new users and LLM bots, but is one kind really worse than the other, if both of them contribute to killing the community?
We already know what kills communities. It's the eternal Septembers. Infighting within leadership also doesn't help, but time and time again it's the influx of too many new users that nosedive and drown out quality contributions.
Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”, and then actually spend time reading them?
No? I’m imagining not at least. Because there would be no point to it.
If you would enjoy it, then I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
> Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”
The reason I'm not simulating the experience with an LLM is because:
1. It costs more time to do so, because I have to prompt it to create a single comment. Multiply that by the typical number of an HN thread.
2. I suppose in a way you need bad takes to form your own view of a topic or an issue. LLMs would also be unable to provide truly unique experiences, such as some of the veterans who sometimes post here who were part of the living computing history as we know it.
> I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
That's something you imagined that I claimed I want. If you read my comment again, you'll see there was no such thing.
An irascible human being with "wrong" opinions is still better than a polite and factually correct bot because there's no fucking point in having a conversation with a bot. We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Do you really not care one way or the other? Would you really rather just be talking to LLMs here? Or would you just script yourself as well and call it a day? Then what?
> We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Maybe you are. I like getting to a reasonably correct model of a topic or issue. Bad human takes can still be useful here. I just get inevitably tired of the people crying about potential LLM comments all the time.
> Would you really rather just be talking to LLMs here?
Obviously we're not there yet, regardless of what I want. But there is a great number of HN threads posted here that touch on topics that have been discussed so many countless times, that an average LLM summary would do better than most comments.
Unless you've discovered the secret sauce, LLM comments are very obvious. Even Altman revealed that they focused on coding at the expense of writing.
The obvious ones are the ones you notice
LLMs are not good at writing. If they were we would have entire libraries of new, amazing literature.
8 replies →
With the current batch of SOTA models, it is not hard to prompt a model to pass the sniff test on social media forums. If you don't believe me, try it.
All you really need to do is give it some guidelines of a style to follow and styles to avoid. There's also a bunch of skills people have already written to accomplish this.
I have worked with LLMs for a couple years at a very non-technical level and it was not that difficult to give it proper prompting and reference material.
If you are reading LLM content just about everywhere and have no idea. Obviously there are easy to spot things, but the stuff you don't spot is the stuff you don't spot
[flagged]
People that like to fancy themselves as good llm content detectors just end up accusing everything they don't like as llm content.
The only thing worst than a slop comment are the people that bitch about it incessantly. I'm convinced it's become a new expression of a mental illness.
The main thing I suspect of being LLM written is the sort of LinkedIn style: very short sentences, overly focused on sort of… making an impact on the user. But that’s also how a certain type of bad human writer writes. So in the end, I’m not sure I know if anything in particular was written by an LLM.
I guess… “that’s not just an AI red flag, it’s generally shit prose” would be how ChatGPT would describe most things nowadays.
1 reply →
A mere opinion is not mental illness.
6 replies →
Dead Internet theory ?