Looking at the list of removed stories makes me really happy with the moderators here. They're all sensationalist, advertising for some company, clickbait, way off topic, or some combination of above. In fact, I don't see a single story that I personally feel should not have been removed.
(Perhaps that last one could be renamed to be less hyperbolic, but the content was still an interesting opinion piece)
I don't think this is being done by the mods, by the way. It's more likely some spam filter with false positives, report brigading, or an anti upvote ring mechanism.
The first two you listed were downranked by the flamewar detector. The last one was downranked by users. Admins didn't touch any of them.
Note for everybody: can you guys please include the HN /item link if you're mentioning specific threads? That would be much more efficient and that way I can answer many more of people's questions.
I don't agree with the GP at all. Most seem normal for the front page or the intellectual curiosity standard (I mean, personally I'd like a much higher standard, but I'm basing it on what HN already has).
The things you see on HN are not purely decided by the community. Mods can and do "freeze" the vote count on comments and posts, and do other non-obvious things too. You will notice the effect after participating for a while.
The last story is so full of outdated and misinformation that I tried to find out whether it was written a few years ago (though it would have still been full of misinformation back then).
I suspect that it has been flagged for that reason by multiple people.
Two out of three currently aren't removed. There's no moderator comment on the third, but a fair number of upvotes and user comments; I think it was flagged by users.
I witnessed a recent front page link silently get changed to point to a parody video, then silently changed back later, with the top comment that remarked on the change silently removed.
That told me all I needed to know about the moderation of this site.
If what you require from an internet forum is that the moderators under no circumstances will ever commit a copy/paste error, HN is definitely below your standards.
Edit: the mods would like to share that they weren't drunk when they made that mistake, just rushed and watching a rather gripping tennis final.
One thing I've learned today, after 11 hours of posting in this thread, is that it's easy to look at an article in isolation and say "Really? That got moderated?" - when in fact if you know the larger context there's nearly always a straightforward explanation.
One can certainly argue that 86 points and 46 comments is too low a threshold to treat the repost as a dupe, but that's a different question, no?
Yeah this list seems to be pretty low quality stuff. There's a couple economic/political links that I think are interesting but I can totally see why they would be removed as off-topic or likely to produce a flamewar.
It's pretty clear to me that any online forum needs good moderation to be healthy long term. HN has been good about this with a strong community providing upvotes/downvotes and a moderation team that seems pretty light handed but not afraid to say no when necessary. Please keep doing what you're doing.
Agree, and to be clear, that's why I upvoted this submission -- it really is an endorsement of the algorithm and moderation we see here. I know the person who wrote the article did it from a place of skepticism, but it functions as a nice gold-star transparency report.
It really is impressive how HN has been such a quality community for so long. I can’t think of any of many other online communities that I have been using for 10+ years. So definitely much gratitude to the mods from me for the work they do.
You're right, but I'd like to add that we do turn off flags sometimes when we think a story (a) has a good (if not high) chance at a substantive discussion and (b) hasn't had much discussion previously. If anyone notices such a case, they're welcome to email us at hn@ycombinator.com and we'll take a look.
dang has been doing a fantastic job for years now. How big is his team? What kind of tools are they using? Would love to read a writeup sometime, but I guess there are good reasons to keep this secret.
> I don't see a single story that I personally feel should not have been removed
Another way to look at this is that the mods have the same biases you do. Depending on how you’re feeling on a given day, you could call that an echo chamber.
We haven't touched those stories except reduce the penalties on them (user flags mostly) and moderate them less than we normally would (per https://news.ycombinator.com/item?id=39172045
If you (or anyone) read those explanations and still have a question that I haven't answered there, I'd like to know what it is. These practices have been in place for many years and haven't changed.
> The assumption is that a Story cannot go from the top 30 to a position higher than 90 in a single minute, without having been explicitly removed.
That's wrong. Both the flamewar detector (a.k.a. the overheated discussion detector) and user flags do that, and there are other software mechanisms that do it too. For example, if a story has been on the front page for more than (IIRC) 18 hours, it gets an automatic downweight unless we manually override it.
Also, keep in mind that user flags affect a submission's rank long before the [flagged] marker appears.
How do you keep user flags from being used as a way to squash articles on a particular topic before they have had the chance to be exposed to the wider HN community?
Meaning if someone were to theoretically get a real time feed of HN submissions, and flagged articles that they didn't want seen as well as messaging a group of friends to do the same thing. Do you have protections for this type of behavior that would prevent this person from having undue influence on what can and cannot have a chance at being seen by others?
I just posted about that here: https://news.ycombinator.com/item?id=39235382. The relevant part is in the middle of that rather, er, wallish text, but if you read it (assuming you find it!) and have a question I didn't answer, I'd be happy to take a crack at it.
It's just the flagging you already have access to (as you have more than 30 karma). It's the flag option. There's nothing special otherwise that I'm aware of (at least at 2500 karma)
I don't have a problem with users building things like this because the principles by which HN works are all easy enough to explain and defend—just remember that anything this complex is inevitably a mess, so you need to have high tolerance for messiness if you want to understand it accurately.
However, it's important to correct inaccuracies like the one mentioned here: https://news.ycombinator.com/item?id=39231537. Robin89, can you please fix the text? I know it was just a mistaken good-faith assumption but it's super wrong.
Also, it would make it easier for me to respond to the questions here if you'd link the HN IDs on your page to the actual HN threads. Currently they link to social-protocols.org. Obviously you can link to whatever you want but I'm having trouble tracing the questions here. Everyone has their own list of "what happened to story X, Y, Z, and what about W and V and J too" and while I'm happy to answer all those in principle, there are physical limits on how many I can work through.
I'm going to be in meetings for most of the next few hours but I'll try to answer questions in this thread later, assuming I don't drown in it.
There are several low-point, long-lived but highly ranked unnatural posts on the top page that appear to be manipulated. Such unnaturalness and opacity make users feel that the ranking is arbitrary and unfair, even if for good reasons. Can you display the manipulation that has been done on the ranking and other lists per post? For example, a reset of the submission time should be easily displayed.
I use the downvote button for two reasons: overwhelmingly for rude, come-uppance, and similar type of comments; very rarely for comments spreading FUD for no particular reason. I just downvoted your comment for the second reason.
Demanding transparency is fine but you’ve got to provide proof with your claims. If there are stories which feel manipulated to you link them and let the audience see, maybe you’re right.
Additionally, just because it’s possible that this could happen doesn’t really give us an idea of how likely it is. Is it one of those theoretically possible, but it never actually happens events? there’s a huge difference between it impacting half of the stories that fall off that quickly, and it impacting 1 in 10,000 stories that fall off that quickly.
Wtf are you talking about? He’s literally telling us and has mentioned in the community many times that flagging quickly crushes a story.
I’ve seen it happen when I’ve flagged stories so either there is a vast conspiracy of moderators that receive pages when I flag things so they can downrank… or maybe dang isn’t lying about something that should be super obvious as a community self policing mechanism.
There was some heavy handed moderation decision that moot made, can't remember what, but he enforced it by saying "One man's shitpost is another man's board culture". I think about that a lot when it comes to moderation because people tend to assume everyone in the community is just like them; and really only moderators have a gauge on how saturated certain can be.
It's also why I don't like the "free speech at all costs" meme that gets thrown around when $corporation bans $person_i_like. Every community needs moderation and it's often a thankless job that feels like nothing is being done at all when it's being done right.
> it's hard to believe that HN users would be tired of LLM-related news
For sure many are. This happens with every Major Ongoing Topic (MOT) and LLMs are way beyond a MOT [1]. The hivemind tires of repetition extremely quickly [2]. The trick is to try to separate wheat from chaff, where 'wheat' means the stories that bring Significant New Information (SNI) [3] and 'chaff' means the follow-up and copycat stories, which are legion [4].
It's important to understand are that there's a wide spectrum of opinion about this stuff. If you imagine a slider with "allow zero posts about $TOPIC" at one end, and "allow all posts about $TOPIC" at the other end, pretty much every user would slide it to a different position. This is true for every $TOPIC and especially for the biggest ones.
Frontpage space is the scarcest resource HN has [5] and every reader has a different 'signature' of preferences that they would like to see (or not see) there. This means not only that it's impossible to satisfy everybody, but that it's impossible to fully satisfy anybody—because nobody's 'signature' is perfectly matched on the front page, and (lest any of you be thinking of this quick riposte) certainly not the mods'!
It took about 10 years for the crypto headline hysteria to taper but it might have been only because ai is now the big annoying thing to shoehorn into everything. Monkeys pawl will curl and you will emerge out of your hibernation to more disgust at whatever the next annoying thing will be.
I'm sick of LLM-related news. I'm fascinated by the technology and the progress, but for every one article about something novel, there are dozens rehashing the same points about social impact, bias, deepfakes, plagiarism, etc. These topics are of some interest to me, but the vast majority of the articles bring nothing new to the table and are reactionary responses to the latest infraction.
I believe that is close to what the median HN reader feels: interested by the significant new developments, fatigued by the endless incremental updates, and grossed out by the hypemeisters.
LLMs are like crypto, where scams and scam-adjacents are everywhere.
I am the biggest local ML advocate you will find. My 3090 is either running Yi 34B queries or other experiments all day, my job is with local LLMS... But I am totally OK with heavy handed AI-related moderation. I dont want the sea of AI grifters to have a single second on the HN front page.
I will occasionally flag things that will result in discussions that are always the same because I'm tired of them. Stories about tipping at restaurants or Trump or Biden, for example -- literally every argument for or against has been made and there's nothing new or interesting to say. But I'm more likely to hide them.
I would also include the periodic Monty Hall re-post (everything that ever comes up in the discussions can be found in the Monty Hall problem wikipedia page).
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I think the answer to this is... go set up your our LLM News web site and build a community. I really love HN but I wanted more retro computing and gaming news so I created by own site (https://twostopbits.com/) using the HN source code. It's not hard. Go build the thing you want and moderate it.
I've been in various online communities for over 35 years and I can tell you that by far the best moderated and longest successfully running community is HN (for a while The Well was amazing).
I hadn't see this before, but that is exactly what I've been wanting more of - thanks for setting that up John! I'm curious, do you feel like lower traffic communities like yours serves up the content mix you were hoping for when you started it?
There are moderation tools. I can do things like kill a story, ban a domain, ban a user, alter the score on a story, mark a story as dead, lock a story so no one else can edit it, see how many sock puppets voted for the story, edit any aspect of a story.
No, I use a lot of Cloudflare products (the domain is registered through Cloudflare, the site is proxied and protected by Cloudflare, I use Cloudflare's free Web Analytics), but I am not using Workers for this.
The HN source base is a monolithic Arc program and Arc is in Racket/Scheme. To use Workers I would have had to get Racket working on Wasm which I simply haven't tried. Also news.arc does a bunch of file system access and I'd have to rewrite that to use Workers KV or something. So, I decided to use lots of Cloudflare and run the Arc code on a VPS I've had for many years. The whole thing is running in a screen session which I can hop into and be in the REPL when I want.
Don’t the vast majority of these get removed via flags from users?
Edit: I’m not asking a rhetorical question. There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page. Can anyone attest to this?
Just to clarify one misunderstanding: most flags on submissions (nearly all actually) come from users, not mods. So if you see [flagged], it's almost always there because of users and in many cases the mods haven't even seen it yet.
But there are other ways besides flags for stories to fall suddenly off the front page: software penalties (e.g. the flamewar detector, a,k.a. the overheated discussion detector, various abuse detection systems, etc.) and moderation downweights. Users don't do either of those.
These points are covered in the FAQ although necessarily tersely. See "How are stories ranked?" and "What does [flagged] mean?" - https://news.ycombinator.com/newsfaq.html
There are stories on this list that deserved to be seen, were popular, were important, and were not in fact dumpster fires in the comments - but a particular crowd with a particular bias decided to flag them.
Does this crowd think it's cool and normal that all discussion of the ICJ's decision - truly momentous - were completely removed, based on the opinion of a dedicated minority?
US tech giants are heavily implicated in this, so no one can seriously argue the topic isn't relevant. A World War could come from these "plausibly genocidal" actions, which are enabled in various ways by US tech giants.
> There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page. Can anyone attest to this?
I don’t get the impression from that article that Daniel and Scott are curating the front page in the way the thanks in this thread suggest. I am still of the impression that the front page composition is decided by upvotes, downvotes, and flags. Contrary to the implication in this repos’ text.
> There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page.
IMO this happens because fundamentally people have "The reddit mental model" about how moderation works here, as if moderation is some privileged, limited position. It's just wrong.
Yes, there is dang, the single admin who posts publicly, and I guess it's possible/probable there are other HN admins who assist him. But 99.9% of the time when I hear people complaining about "the mods" or "power tripping mods" or "censorship", it's basically that other users saw what you had to say, and we just don't want to see it here.
It's also weird that occasionally people think there is some sort of "rule" about what can be flagged. There are obviously guidelines, but as this power is held by any normal user, it's basically whatever they want it to mean. For example, I frequently flag stories where I think the topic and article is totally valid, but where every single time I've seen the topic debated on HN it becomes a useless flamewar or is filled with the lowest quality commentary. At least for me, flagging isn't a value judgment on the "worthiness" of an article, it's simply about stuff I don't want to see on HN.
> IMO this happens because fundamentally people have "The reddit mental model" about how moderation works here, as if moderation is some privileged, limited position. It's just wrong.
Partially, but I think these are all symptoms for a more fundamental root cause: HN is just comprised of too many emotional, passionate users with fundamentally differing beliefs.
The usual song and dance with flagging goes something like the following with cryptocurrency:
1. User posts cryptocurrency article
2. People who passionately hate cryptocurrency start adding in emotional comments about how they hate it.
3. People who want to fight this passionate hate respond in kind.
4. The thread turns into a giant argument where nobody is willing to concede anything and everyone is just shouting at each other.
5. Either the flamewar detector kicks in (as it should) or everyone not in the thread tires of the shouting and flags it.
That's fine but regrettable when limited to some topics like crypto. But it's happening with social media company earnings reports, layoff posts, RTO discussions, posts about Musk, autonomous vehicles, and on and on.
dang (and the mod team?) are doing great work, but this is despite the feeling I have that HN is barely being held together into a cohesive community, and I'm struggling to even use the word "community" here. I feel the temperature of discussions has gotten a lot hotter here than it used to be and some basic work I've done with sentiment classifiers on comments here mirrors my perspective.
I just don't think a single community can handle so many passionate, opposed groups. It bubbles up by proxy in these sorts of flagging wars where so many articles get bumped off the page due to the inability of the community to discuss it well. Maybe the solution is to just discuss software as some people really want, but even then you get massive flamewars over things like Rust async. Even with interesting topics like VR posts, the overall temperature of the comments here is high enough that I've stopped bothering to comment as much as I used to.
> In the case of the first, the Story was among the first on the Front Page, until its title was changed from "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes" to "OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2". This effectively "killed" the Story.
> In the case of the second, the Story was in third place on the Front Page, less than an hour after the submission. In this case it was simply removed from the Front Page.
With repeatedly getting flagged articles like this, at some point you have to begin to wonder if you are not simply spamming the community by trying to promote your links.
I get that people want to promote their stuff but the community has preferences too. The community can get tired of LLM articles reaching the front page everyday! The community can refuse to be spammed and the community can flag articles!
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
Denial? Why is it so hard to believe that HN users would get tired LLM-related news. I get tired of it myself but I don't have flagging privilege. I find it very believable that HN users who have the flagging privilege might want to flag LLM-related news.
Honestly only 3 or 4 out of 13 look like possible moderation to me. And they don't seem bad. Does a story about razor wire in Texas belong on hacker news? I'm in Texas so the story is of interest to me but I'd expect to hear about it elsewhere, not on HN.
Overall it just makes me think HN is doing a good job at moderation.
> Does a story about razor wire in Texas belong on hacker news? I'm in Texas so the story is of interest to me but I'd expect to hear about it elsewhere, not on HN.
Perhaps in TX you don't realize it, but it's a big national story, implicating the Constitution, federal authority, even the Civil War.
When I had more downtime I’d spend a lot of time browsing /new.
There’s a wealth of great blogposts that show up there which don’t always make it to the front page (understandable; we only have so much attention to give).
What I will say is that there is a ton of cruft that spams the board. Thinking of spammed blog posts from one or more accounts, sensationalist news, etc which wouldn’t provide much value here.
Flagging really helps on /new IME. It’s worth spending time there if you haven’t tried HN other than via the front page
>it's hard to believe that HN users would be tired of LLM-related news
I know there are already a handful of comments about this line, but wow! It bears repeating: My eyebrows almost shot off the top of my head when I read this. What kinds of things does the author find easy to believe??
This is a good thing to be able to see, but I'm much more interested in identifying the soft censorship in comment sections.
This is obviously harder, because vote totals aren't publicly available for comment sections, but it is much more important as a tool. What topics are on the front page is much more clearly the legitimate domain of moderation than what commentary is made about them, especially when moderation of those comments contradicts the vote mechanism.
It's more boring than you'd expect. The comments that get most heavily upvoted tend to be either (1) indignantly rhetorical, or (2) generic. Mostly what we do is downweight those when they're at the top of the thread. The "algorithm" is about as simple as:
(1) if the top comment is indignantly rhetorical or generic, downweight it; otherwise go look at another thread;
(2) refresh the page;
(3) goto (1).
If I'm feeling diligent, I might do this for the top few subthreads, but that's about it.
This simple intervention turns out to be the highest-leverage thing we've figured out in recent years:
I wrote a program that tracked the changes in story titles on Hacker News a while ago. Some of them were really quite strange. I can't understand some of the policies, like why the words "How" and "Why" are stripped from the front of titles (eg "How I rewrote my app" would be changed to "I rewrote my app"). Some very small proportion of the title changes could definitely have been construed as politically motivated but overwhelmingly they were benign.
Can we please have this in reverse chronological order - later dates at the top? Also, another request -- can we change this to TABLES instead of the plain Markdown list. I believe this is where a tabular display will be much easier to browse.
> I sent an email to the moderator. @dang, who was very kind and quick in his response, explained to me that the Story had been flagged by users even without being explicitly [flagged], and that he could therefore only hypothesize the causes of the flag.
Maybe this is a consequence of Hacker News not having a way to downvote stories?
I only flag stories that are blatant violation of HN's guidelines: SPAM, politics, racist... Otherwise, if I don't like a story I don't do anything.
Maybe I'll start flagging stories that I don't like?
Yea, I suspect a sizable number of people use "flag" as a mega-downvote for things they passionately don't like, rather than for policy violations and spam.
Yeah, like you I would have never thought to use the flag unless it was violations etc but turns out it's a weird behaviour of many users. Not sure why or when it originated but seems like its been driving the up/downs of a number of topics/posts for years now. Still use with moderation.
Reddit used to have a similar thing (r/RedditMinusMods). Eventually when it got to the point that 48-50 out of the top 50 posts were removed every 12 hours, reddit banned the sub for unspecified reasons.
How can you complain about your submissions getting hundreds of upvotes and a bunch of discussion over the last 4 months only. That's a decent amount of eyeballs.
Other than the blatant offtopic/spam ones, most of them are just ones that have drifted away and are old news, or flagged, or dupes. It's driven by the flow of the site and its users.
i feel like every news source, forum, link aggregator, ... has its own target audience and scope of topics that make for productive discussion, its own biases and predispositions, its own trolls and need for pruning and moderation.
i feel like yes of course there are many things i disagree with on this site. but ultimately i value the information shared and the discussion enough to keep coming back. any relationship where people always agree there is probably only one person doing the actual thinking.
i have learned so much about tech here, i have learned about many best practices and projects that i would have never heard of, i have made no bones about my thoughts on various subjects that could easily be classified as touchy, i have really enjoyed the discourse. for the time being i definitely plan and hope to continue doing so.
(so while this site is an interesting artifact, and maybe it is good that someone is taking a look and keeping a record, i personally won't bother unless/until i see a pressing need. at which point i will maybe just move on instead tbh.)
I did something with a similar idea. Rather than looking at deleted stories, I historicize the stories and display them as a graph. You can see which stories have a second life (they were created several days ago, but the score doesn't increase until several days later). https://y-combinator-news-trends.vercel.app/
You can see your story!
The github isn't open source because the project isn't really finished (the page is ugly, by the way).
I have to say that I'm not relying on the api but on scraping the front page. The reason was to migrate code I had from python to typescript (I'm better at the later...)
> Using the official HN API, the service fetches 90 Top Stories every minute and makes a comparison with the first 30 Top Stories (i.e. the Front Page) fetched the previous minute. It logs all missing Stories here. The assumption is that a Story cannot go from the top 30 to a position higher than 90 in a single minute, without having been explicitly removed.
The OP's hypothesis is that, if rank drops from top-30 to below top-90 (I think "higher than 90" is a typo?), in less than a minute, then it must be due to moderator action.
No, it's not true. I have many times been the person who hits "flag" and then refreshed the page to find the story disappeared as [flagged]. When that happens the story is completely gone, off at least the first 3 pages (which is what OP is measuring).
I think it would be really interesting to get a tracker for the ones that hit the front page, then fall off, and then surge back in 30-60 mins. I wonder what's going on with those ones.
Interesting service. I think it would benefit from further improvement. Many stories are actually dupes, self-promotion, etc… It would be nice to see a much smaller list of stories that were actually censored by the moderators or self-censored by the community.
How to do this? One idea is to write an appropriate prompt for GPT-4. Something along the lines of “if you were HN moderator or HN community, would you censor this story? Please give numerical score.” Then post a much smaller list with top scores. That would be useful I think.
I started collecting the /news feed something like 7 years ago in a script, I think it's still running. It's fascinating watching stories get dropped or auto-killed and then running stats to find out what the algorithm is trimming. I think I started it because there'd be things changed or removed and you couldn't tell unless you had looked at it before it was culled. At some point I'll kill the linode it's running on, maybe move it to a Lambda, push the database to GitHub.
Yesterday I posted a link that was flagged off the front page. I can see that your tool is driving traffic to it and appreciate that.
Not everyone wants to discuss political topics on HN. They say there are other places to discuss such topics. I like to hear the opinions of the HN audience on a wide variety of topics. Maybe this tool will help those of us who value the HN community in this way by facilitating discussions on topics deemed inappropriate for the official front page.
I've always assumed that when a story rapidly drops past the first three pages (e.g. >90) it is because it has been flagged some indicative, disproportionate number of times[1] by users with flag functionality. The submission seems to presume that such drops are only the result of Daniel or whoever manually doing it.
Which is true?
[1] - Notably the majority of the "removed" stories have pretty tiny number of upvotes, so if flags are weighted proportionately it wouldn't take many.
I’ve wanted something like this for years. My top requests would be to offer this as a standalone separate site, with sorting in reverse chron order, and an easy way to click into the comments for each story. Right now I can click into the info for the link, and to the outlink, but there doesn’t appear to be a one-click path to the HN comments themselves.
I'm honestly not sure if that's a bad thing. To an extent, it seems like reddit's "sort by controversial". If you're looking for a flamewar, here you go.
It's interesting to see the comments sometimes, but since part of the reason these things get removed is because of the flamewar detector, I feel like I can't be that surprised or edified when I open the bag labeled "manure" to find it is full of shit.
I think my comment was early before all your responses, thank you for your work!
I didn't expect to get an answer from Daniel Gackle himself - shows the amount of dedication.
Only saw one Gary tan link removed. I thought I'd see more. Maybe it was only removed because it was a dupe? I'm referring to the "Gary tan tupac lyrics" one.
There was a bunch of Gary Tan links -- you can see in my comments I was arguing with a bunch of HNers today about whether he's right or wrong on that "Die Slow" tweet. Probably dang removed it because it's a dupe story.
He could have phrased it a little better but the people calling for his removal from YC are just plain silly.
I am more interested in the mechanics of how something like this works, especially over time.
All kinds of tools related to HN content generate front page interest even for days but then once that passes things that cost money or use unreliable free resources start to disappear at an ever more rapid pace.
When they don't, the UI can become unmanageable... I'm not sure how this content will be organized over time but updating the README won't be tenable for long!
It's great that they offer the source code so you can modify it and run the program on your own hardware to make sure stories aren't being removed from the stories removed from the Hacker News front page list.
While the above is me joking, I appreciate the extreme transparency that showing code and explaining methodology provides. This adds more credibility than any other single thing the author could have done.
I wrote something similar, it's a simple Python tool that estimates what a story's rank should be based on the score, and compares it to the actual rank. It's rough and only tested on Win 10 so far:
Long overdue transparency. Sometimes these are innocuous or warranted removals, but there is also an element of protectionism at play. And that may not even be due to mod actions, but blocks of users who all flag articles to get them pushed to no mans land.
There are companies who if you submit a negative post about, within short order the post is pushed out of view of the top pages.
>blocks of users who all flag articles to get them pushed to no mans land
This is just another way of saying that a critical numeric threshold of users didn't like something. Framing the opinions/actions of groups of people on the internet as conspiring or dog-piling is a fallacy. E.g. if a person Tweets something that a million people read and a hundred of them reply to disagree, you'll often see that person follow up with something like, "wow, now all these people are attacking me", even though everybody acted in complete isolation and did nothing strange or harmful individually. Nobody rang a bell in the town square and handed out pitchforks. The internet breaks human psychology.
Except when it's not. If you don't think groups within organizations all message each other to quickly flag posts that are negative towards them, then you may be looking through this with an idealistic lens that hasn't been shattered yet.
I'm not denying your premise that yes sometimes independent people with no coordination, all flag an article. That is how the system should work. But there are also articles that will quickly get flagged through coordination of interested parties.
Hacker News has a lot more power than many think in terms of tastemaking in the tech industry. So there is a lot of motivation and benefit for people to manipulate its functionality to either boost or protect their business.
I went to check if the story I submitted today (Breathing 101 — https://news.ycombinator.com/item?id=39227295) was present in the list, and it was indeed! I don't know how it reached the front page and why it got removed.
I would have downvoted that for a number of reasons. I guess people flagged it, since there's no downvote. I don't flag unless it's something egregious (racism, spam, etc). But since this thread is all about the topic of why stories might not make the cut here, this is why I would have downvoted it if I could:
1. "Breathing 101" is an uninformative headline. I correctly guessed that it was literally referring to the human act of breathing, but it's still a bad title (I know it's not your title, and that HN encourages using the original source's title; it just sucks).
2. You submitted the link with no comment or context about what the article was or why it might be interesting. If a headline grabs my eye, I always click on the "N comments" link and the article link to open two tabs, and I look for additional descriptive text from the submitter, or a comment from them about what they found interesting. Sometimes I read the actual article first, but if the title is ambiguous or the topic is contentious, I'll usually start with the comments tab and see if I'm going to be wasting my time before I read an article. This alone wouldn't be a reason to downvote, but if I was leaning that way, it would factor in.
3. The word "wellness" in the link's domain is a huge red flag. To me it means "this is going to be a bunch of hippie crap". Not a primary factor, but seeing that would be enough to make me dig farther and find evidence so that I could Angry Downvote something I don't want to see on HN ever, if we could downvote. Yes, this is petty.
4. The very top of the linked article says "Click here to make an appointment". This indicates spam.
5. The article is just bad. There's not much information. It's not scientific. It touches lightly on some potentially interesting things but doesn't dive into them at all, or link to better sources, and it ends with what appears to be advice and encouragement to incorporate breathing exercises, but without much information about how or what the benefit is.
It looks like spam. It's the kind of clickbait that floods my Facebook feed.
I wish we had meta-moderation and voting and flagging reasons like on Slashdot. Flagging as a super-downvote seems user-hostile and passive-aggressive for a site like HN, but I’ve seen it tacitly acknowledged as allowed as okay on HN by dang, which seems counter to the spirit of the guidelines.
That rule does not generally apply to HN meta debates, which this is, so it's a bit of a corner case. If there was a duplicate of this story on the front page tomorrow, I'd expect HN to honor the user flags on it.
This story has close to 500 up votes as of 2:19PM Pacific and is no longer on the front page. Why flag an article that is simply trying to show how HN can potentially be improved?
The problem is that while the major media sites generate a lot of noise, they also publish solid original articles from time to time, and we don't want to miss those. For that reason, they're downweighted, but not banned.
I ran my own client side script to strip out links to such articles. New computer, didn't copy the script over, and now I'm engaged in passive aggressive discussions over some guardian article. It's my own weakness of course, but I am definitely not the only one.
While I understand why "Secret Plan against Germany" - a story about Germany's far right planning "remigration" of even citizens - was removed, I still would've liked to see the part of the discussion that wouldn't have been arguing about the semantics of the term "nazi". This is the article that sparked a never seen before mobilization of demonstrators against these planners and their party in Germany.
I'm echoing others, but the article on rust async/await seems good.
You may or may not agree with the conclusions of the post, but its a technical topic with at least some specific exploration of the (performance/code writing) issues, that links to quite a few further topics for exploration.
I also found this one interesting. I don't agree with the article, but its an interesting viewpoint and I learned a bit about what some people are doing with git. I couldn't tell you why it dropped (unpopular)?
Which is where its possible that this (new) tool falls short, it can't actually tell what was censored, just what wasn't popular.
Unpopular things sometimes are so because they fly in the face of conventional wisdom, but aren't actually wrong or invaluable, which might be the real value of this tool.
It is really not. It is a rant that produced no good discussion anywhere else on the internet. It has no novel insight and is dressed up in a really ugly way. I'm not saying HN should have removed it, but I don't mind that it got flagged.
Hmm most seem to be idiotic, spam or off topic for HN.
I wonder if a useful application for these "AI"s could be to pull interesting - to someone - stories from what the hive minds rejected ;)
Just to be clear, this is stories that got completely removed off the front page and does not include whatever is still available 4 pages behind but got overtaken by other stuff?
Sometimes I see an interesting heading but skip it, and when i reload it's gone. I doubt they were all flagged into oblivion.
I really don't like Musk but I don't flag things Musk related. I frequently upvote them because I'm interested in the discussion.
Unfortunately those stories often turn into flamewars. That's probably why people are flagging them.
I don't think it's wise to draw so many inferences about why people vote the way they do. Frequently I see comments where someone makes a reasonable point, but also drops a bunch of flamebait, and when they're inevitably flagged they edit their comment to claim that the flags prove their point and that the problem who disagree with them are overly sensitive and censorious. But in reality a lot of the people flagging them probably agree with them, but don't want them to start a flamewars. I flag a lot of comments like that, even when I am agree with their overall point. (I actually did that with a comment just now.)
It's a form of self fulfilling prophecy and further entrenches you into your position, which is antithetical to curious discussion.
> certain things (e.g. anything pro-Musk) get mass flagged for emotional reasons.
Lots of Musk stuff, including positive stuff, on the front page. Yesterday there was a story about petabytes of data on the Starlink laser network, based only on Starlink PR afaict.
"While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news."
If you believe that HN is a hive mind and all users must believe in the exact same things, then yes, this is probably hard to believe.
I however, am tired of LLM news, but I just simply ignore them as I'm well aware that many people here are very much interested in them. So at least an anecdotal response of one that some HN users are tired of LLM related news.
You might also be surprised that not all HN users like social media while some do. Some are very privacy conscious while others will freely post all of their everythings to anywhere. You might find it hard to believe that some lean left while others lean right with some even landing straight in the middle. Why you would think anything is hard to believe in this day and age is very strange to me.
That "While I have no reason" line has been quoted in six top-level comments so far, obviously it struck a nerve here.
It would be ultra-cool to have rough topic filters here, so I could just go to settings and hit a checkbox to ignore all the LLM-this and AI-that articles. Easier said than done, I'm sure.
Any time you paint with a broad brush with comments like that, you're going to miss some of the details. Looking at the time stamps of those comments shows they were pretty much at the same time. I use the phrase "group think" a lot, but intentionally do it to in part rabble rouse, but also to get those in the group think to maybe think and take a second to question if it truly is group think behind their current position.
I think this is an unnecessarily uncharitable reading, that the author assumes HN is a hive mind.
Replace "HN users" with "most HN users" (it's common to use general language when one's intention is to point out a trend in a population) and, as another person tired of AI/LLM news, I would also be surprised given how much popularity (upvotes, comments) HN users tended to give to those stories.
The subthread your comment generated here already answers your question. (<-- not a criticism! just an observation.) People are flaming each other about the inverse square law, droplets vs. aerosols, who is refusing to face reality, and sundry other nastinesses in the comments below. It demonstrates what a shitshow a frontpage thread would have been.
It's not that the topic itself is "untouchable". HN had quite a few threads about the lab leak hypothesis for example. But these things are sensitive to initial conditions, and something about the way that headline frames the story feels doomed to me, from an HN point of view. The sweet spot for HN is substantive, thoughtful conversation driven by intellectual curiosity. That's what the site is for. We don't always get there by any means, but I only want to turn off user flags when the odds give us a fighting chance. I remember seeing that story get flagged and thinking: it'll never work.
Another aspect of this: like it or not, curiosity and repetition have an inverse relationship. After the mind has been hammered with the same hammer enough times, curiosity gets sick of it and goes "ugh, not that again". That means that on a topic like all-things-covid, which we all got hammered with, the majority of the audience, who don't care that much, check out at first mention of the topic. Who does that leave? The ones whose motive is more intense than mere curiosity.
From an HN point of view, that's a ticket to hell. Curiosity can only operate within a certain range of nervous system activation. If the needle sinks too low, the topic is 'bleh' and nobody cares; but if the needle goes into the red, people will care—my god will they care—but they'll no longer be functioning out of curiosity. That's a failure mode for HN.
When it comes to divisive, heavily-covered topics like that one, the thing to watch for is some kind of interesting new information that isn't entirely reducible to existing battle lines. The same forces driving the thread into flamewar will still be present—but at least you'll have some current running the other way.
Stories about COVID controversies are almost certainly getting flagged off the front page by users, not touched by mods. People look at the titles of these stories and think that's all flaggers are going by, but lots of people flag stories based on their experience of what the threads are like, and the threads on COVID controversies are fucking dreadful. I didn't flag (or see) that story, but I would have.
But why must they be dreadful? Genuine question, I am not being obtuse. We should be able as a community to discuss conterversial subjects somehow.
I also think this sort of thing invites flag brigades. Or better yet, a small batch of bad actor can easily start brigading and forcefully associate such flamewar expectations with any subject they don't like to drive it off HN.
Maybe worth reconsidering how you flag? You might be getting played. Or not, I really don't know. No obvious answers.
> lots of people flag stories based on their experience of what the threads are like
IMHO story submissions should be judged based upon their own merits. Toxic commenters can be downvoted/banned but the story submitter shouldn't be punished for the misbehavior of others.
> I didn't flag (or see) that story, but I would have.
You mean purely based on the expected awfulness of imagined future comments, instead of the actual comments? If so, with a precrime mindset like that, you're fanning the flames of controversy.
I flagged that article, so I'll clearly explain why:
1. I think for anyone that has been on HN throughout pandemic knows it is extremely unlikely for topics like this to produce any sort of valuable discussion. I almost never see any sort of humility on the topic (to be clear, from many/all sides) that admits that people (individuals, experts, literally everyone) were doing what they thought best with the information they had available at the time. It always devolves into portraying the other side as evil. I'm tired of it, I don't want to see it on HN, there are literally pages and pages and pages of place on the Internet where you can have that debate if you're so inclined.
2. Are you honestly purporting that specific article is well tailored to "an overwhelmingly science-minded audience", as opposed to just having a particular political axe to grind, given the title is "Anthony Fauci Fesses Up"? Honestly, if the article was written with an intent to encourage an actual understanding about where the 6-foot rule came from, and about whether the evidence for it was lacking, I probably wouldn't have flagged it.
> it bothers me that an important topic like that is deemed untouchable.
I think the mistake you are making there is thinking because a particular article is flagged by a lot of users that "an important topic like that is deemed untouchable." I can't speak for others, but for me that is absolutely not what I think, and it's not why I flagged this particular submission.
That isn't clear at all. You seem to be saying that if you anticipate that people might question other people's competence or motives, or in your view a discussion won't lead people to think the right thoughts ("encourage actual understanding") then you flag it to try to ensure nobody can discuss it.
But you also say that making it undiscussable is also not about making the topic untouchable. That's just playing with words, isn't it? It's exactly what you're trying to do and exactly why you're flagging it.
This particular case is really egregious. Fauci has said this draconian policy "just sort of appeared", yet you damn anyone questioning his competence or motives as lacking humility? What would it take for you to allow criticism of this guy?
(Shrug) I don't require scientific proof of the inverse-square law. It's self-evident to the point of being axiomatic. Standing 6 feet away from a virus source will expose you to about 44% fewer virus particles than standing 5 feet away from one, while not imposing any real hardships in most public interaction scenarios. What's controversial about that?
If you demand precise scientific rigor in all aspects of everyday life, public health is probably not the career field for you.
Put a water hose on mist and spray someone with it. Then put a cloth over the nozzle and try to spray them. It's self evident yet people just could not grasp it.
Personally, I thought it was already pretty well established that the six-foot rule was based on poor science. I remember hearing about that years ago.
The thing is, you're not even wrong. The six foot rule was based on what the best understanding of the experts was at the time, and probably saved thousands of lives. Just like forced masking up probably saved tens of thousands of lives. Both were great examples of science, which readily admits to tuning when new evidence comes into play.
However, because there's a right wing cult around Donald Trump, whose fortunes were hurt by the pandemic, the six foot rule and masking and vaccines are set up as straw men and attacked by a gigantic and well funded and organized horde of proxies, including the #1 media network in the US. It goes something like this: because a particular individual got COVID, that's proof that vaccines are not 100% effective and so They Lied To Us For Nefarious Purposes. Or because this particular individual stood 6 feet away and still got COVID, that's evidence that Fauci Is In A Conspiracy With The Chinese. Or because this particular individual survived COVID, it's just a cold. Or because masks are not 100% effective when not worn securely, they are not effective. And on and on.
So it's not unreasonable or unlikely that you heard a thing about bad science and six feet of social distance or whatever. But hearing a thing, and the thing being true from foundational motivations of actual science, are very different right now.
Probably because if they bring that tone to enough topics, eventually they will be right. Though also probably not for the reasons they think (e.g. they say a post will be removed because it discusses a controversial topic when in reality it was removed because it was just plain garbage content).
Looking at the list of removed stories makes me really happy with the moderators here. They're all sensationalist, advertising for some company, clickbait, way off topic, or some combination of above. In fact, I don't see a single story that I personally feel should not have been removed.
Thanks, mods.
At a quick glance, I found several that don't match that criteria you mention, here are a few:
Open Source Doesn't Require Providing Builds
https://codeengineered.com/blog/2024/open-source-not-builds/
Sam Altman Says AI Using Too Much Energy Will Require Breakthrough Energy Source
https://futurism.com/sam-altman-energy-breakthrough
Avoid Async Rust at All Cost
https://blog.hugpoint.tech/avoid_async_rust.html
(Perhaps that last one could be renamed to be less hyperbolic, but the content was still an interesting opinion piece)
I don't think this is being done by the mods, by the way. It's more likely some spam filter with false positives, report brigading, or an anti upvote ring mechanism.
The first two you listed were downranked by the flamewar detector. The last one was downranked by users. Admins didn't touch any of them.
Note for everybody: can you guys please include the HN /item link if you're mentioning specific threads? That would be much more efficient and that way I can answer many more of people's questions.
66 replies →
I don't agree with the GP at all. Most seem normal for the front page or the intellectual curiosity standard (I mean, personally I'd like a much higher standard, but I'm basing it on what HN already has).
All from only one day:
* Ford's new 48-inch digital dashboard is a lot of Android for one car: https://www.theverge.com/2024/1/22/24045932/ford-android-scr...
* Secret Plan Against Germany (a very big story in Germany about a far-right planning meeting): https://correctiv.org/en/top-stories/2024/01/15/secret-plan-...
* Show HN: Vx.dev – GitHub-Powered AI for effortless development: https://vxdev.pages.dev/
* Open Source Doesn't Require Providing Builds: https://codeengineered.com/blog/2024/open-source-not-builds/
22 replies →
The things you see on HN are not purely decided by the community. Mods can and do "freeze" the vote count on comments and posts, and do other non-obvious things too. You will notice the effect after participating for a while.
19 replies →
Sam Altman led invests in a nuclear fusion company, Helion. Guessing the potential conflict of interest is why the 2nd article drew vote controversy.
https://www.helionenergy.com/articles/announcing-500-million...
1 reply →
The last story is so full of outdated and misinformation that I tried to find out whether it was written a few years ago (though it would have still been full of misinformation back then).
I suspect that it has been flagged for that reason by multiple people.
The Altman story was likely a dupe (or triplicate)
7 replies →
Possibly dups too.
YMMV. I don't want to see Altman's fearmongering and hyperbolic statements.
At this point he's indistinguishable from a bitcoin advocate or a tv preacher.
9 replies →
Two out of three currently aren't removed. There's no moderator comment on the third, but a fair number of upvotes and user comments; I think it was flagged by users.
Open Source Doesn't Require Providing Builds
https://news.ycombinator.com/item?id=39102078
4 replies →
I witnessed a recent front page link silently get changed to point to a parody video, then silently changed back later, with the top comment that remarked on the change silently removed.
That told me all I needed to know about the moderation of this site.
Thankfully someone captured a screenshot: https://merveilles.town/@cancel/111834048502040552
> That told me all I needed to know about the moderation of this site.
In that case you drew a general conclusion from a freak accident so rare that I doubt it had happened in the 17 years this site has been around. (Edit: 17 years this month in fact! https://web.archive.org/web/20070221033032/http://news.ycomb...)
If what you require from an internet forum is that the moderators under no circumstances will ever commit a copy/paste error, HN is definitely below your standards.
Edit: the mods would like to share that they weren't drunk when they made that mistake, just rushed and watching a rather gripping tennis final.
2 replies →
Dang explained it was a mod error: https://news.ycombinator.com/item?id=39170137.
9 replies →
https://www.nasa.gov/solar-system/asteroids/nasa-system-pred...
Really? A NASA report, on the official .gov site? Maybe the comments were horrible but that seems right in the middle of what HN is interested in.
You're talking about this submission:
NASA System Predicts Impact of a Small Asteroid over Germany - https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...).
One thing I've learned today, after 11 hours of posting in this thread, is that it's easy to look at an article in isolation and say "Really? That got moderated?" - when in fact if you know the larger context there's nearly always a straightforward explanation.
One can certainly argue that 86 points and 46 comments is too low a threshold to treat the repost as a dupe, but that's a different question, no?
2 replies →
Yeah this list seems to be pretty low quality stuff. There's a couple economic/political links that I think are interesting but I can totally see why they would be removed as off-topic or likely to produce a flamewar.
It's pretty clear to me that any online forum needs good moderation to be healthy long term. HN has been good about this with a strong community providing upvotes/downvotes and a moderation team that seems pretty light handed but not afraid to say no when necessary. Please keep doing what you're doing.
Agree, and to be clear, that's why I upvoted this submission -- it really is an endorsement of the algorithm and moderation we see here. I know the person who wrote the article did it from a place of skepticism, but it functions as a nice gold-star transparency report.
It really is impressive how HN has been such a quality community for so long. I can’t think of any of many other online communities that I have been using for 10+ years. So definitely much gratitude to the mods from me for the work they do.
Props to the mods for keeping the post quality high.
However, I do see a few decent posts in this list that probably warrant a second chance.
Happy to look at specific links if you want to mention any!
The moderators are mainly the users. Flags are what kills a story quickly
You're right, but I'd like to add that we do turn off flags sometimes when we think a story (a) has a good (if not high) chance at a substantive discussion and (b) hasn't had much discussion previously. If anyone notices such a case, they're welcome to email us at hn@ycombinator.com and we'll take a look.
dang has been doing a fantastic job for years now. How big is his team? What kind of tools are they using? Would love to read a writeup sometime, but I guess there are good reasons to keep this secret.
https://www.newyorker.com/news/letter-from-silicon-valley/th...
4 replies →
> I don't see a single story that I personally feel should not have been removed
Another way to look at this is that the mods have the same biases you do. Depending on how you’re feeling on a given day, you could call that an echo chamber.
Garry Tan seems to benefit from this system as well. Nothing sensationalist about tracking his awful behavior.
We haven't touched those stories except reduce the penalties on them (user flags mostly) and moderate them less than we normally would (per https://news.ycombinator.com/item?id=39172045
If you (or anyone) read those explanations and still have a question that I haven't answered there, I'd like to know what it is. These practices have been in place for many years and haven't changed.
5 replies →
[flagged]
> mod*s*
plural ?
[flagged]
> The assumption is that a Story cannot go from the top 30 to a position higher than 90 in a single minute, without having been explicitly removed.
That's wrong. Both the flamewar detector (a.k.a. the overheated discussion detector) and user flags do that, and there are other software mechanisms that do it too. For example, if a story has been on the front page for more than (IIRC) 18 hours, it gets an automatic downweight unless we manually override it.
Also, keep in mind that user flags affect a submission's rank long before the [flagged] marker appears.
How do you keep user flags from being used as a way to squash articles on a particular topic before they have had the chance to be exposed to the wider HN community?
Meaning if someone were to theoretically get a real time feed of HN submissions, and flagged articles that they didn't want seen as well as messaging a group of friends to do the same thing. Do you have protections for this type of behavior that would prevent this person from having undue influence on what can and cannot have a chance at being seen by others?
What you’re asking about is referred to as “voting ring detection” and it’s something social networks keep very secret.
2 replies →
I just posted about that here: https://news.ycombinator.com/item?id=39235382. The relevant part is in the middle of that rather, er, wallish text, but if you read it (assuming you find it!) and have a question I didn't answer, I'd be happy to take a crack at it.
1 reply →
> keep in mind that user flags affect a submission's rank long before the [flagged] marker appears
What kinds of user flags are there and why are they not public? People should know. Shadowbanning belongs in the 2010s.
It's just the flagging you already have access to (as you have more than 30 karma). It's the flag option. There's nothing special otherwise that I'm aware of (at least at 2500 karma)
3 replies →
In your profile settings there is an option to show flagged topics and posts.
2 replies →
I don't have a problem with users building things like this because the principles by which HN works are all easy enough to explain and defend—just remember that anything this complex is inevitably a mess, so you need to have high tolerance for messiness if you want to understand it accurately.
However, it's important to correct inaccuracies like the one mentioned here: https://news.ycombinator.com/item?id=39231537. Robin89, can you please fix the text? I know it was just a mistaken good-faith assumption but it's super wrong.
Also, it would make it easier for me to respond to the questions here if you'd link the HN IDs on your page to the actual HN threads. Currently they link to social-protocols.org. Obviously you can link to whatever you want but I'm having trouble tracing the questions here. Everyone has their own list of "what happened to story X, Y, Z, and what about W and V and J too" and while I'm happy to answer all those in principle, there are physical limits on how many I can work through.
I'm going to be in meetings for most of the next few hours but I'll try to answer questions in this thread later, assuming I don't drown in it.
There are several low-point, long-lived but highly ranked unnatural posts on the top page that appear to be manipulated. Such unnaturalness and opacity make users feel that the ranking is arbitrary and unfair, even if for good reasons. Can you display the manipulation that has been done on the ranking and other lists per post? For example, a reset of the submission time should be easily displayed.
> There are several low-point, long-lived but highly ranked unnatural posts on the top page that appear to be manipulated.
Which are they? It's important to include links so that (a) we can say what's going on, and (b) so readers can make up their own minds.
You might be talking about stories that went in to the second-chance pool (https://news.ycombinator.com/item?id=26998308), which get a random placement on HN's front page.
5 replies →
I use the downvote button for two reasons: overwhelmingly for rude, come-uppance, and similar type of comments; very rarely for comments spreading FUD for no particular reason. I just downvoted your comment for the second reason.
Demanding transparency is fine but you’ve got to provide proof with your claims. If there are stories which feel manipulated to you link them and let the audience see, maybe you’re right.
2 replies →
> Robin89, can you please fix the text? know that was just a mistaken good-faith assumption but it's super wrong.
How can he/we verify it's wrong? The down-weighting you describe is not visible to users. Even OP won't know.
You can say that down-weighting happens, but we're asking to see where down-weighting happens.
Additionally, just because it’s possible that this could happen doesn’t really give us an idea of how likely it is. Is it one of those theoretically possible, but it never actually happens events? there’s a huge difference between it impacting half of the stories that fall off that quickly, and it impacting 1 in 10,000 stories that fall off that quickly.
11 replies →
Wtf are you talking about? He’s literally telling us and has mentioned in the community many times that flagging quickly crushes a story.
I’ve seen it happen when I’ve flagged stories so either there is a vast conspiracy of moderators that receive pages when I flag things so they can downrank… or maybe dang isn’t lying about something that should be super obvious as a community self policing mechanism.
7 replies →
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I am, so you can believe it. But: I don't flag things that I'm tired of.
There was some heavy handed moderation decision that moot made, can't remember what, but he enforced it by saying "One man's shitpost is another man's board culture". I think about that a lot when it comes to moderation because people tend to assume everyone in the community is just like them; and really only moderators have a gauge on how saturated certain can be.
It's also why I don't like the "free speech at all costs" meme that gets thrown around when $corporation bans $person_i_like. Every community needs moderation and it's often a thankless job that feels like nothing is being done at all when it's being done right.
Why does seeing moot quoted suddenly make me feel old.
> it's hard to believe that HN users would be tired of LLM-related news
For sure many are. This happens with every Major Ongoing Topic (MOT) and LLMs are way beyond a MOT [1]. The hivemind tires of repetition extremely quickly [2]. The trick is to try to separate wheat from chaff, where 'wheat' means the stories that bring Significant New Information (SNI) [3] and 'chaff' means the follow-up and copycat stories, which are legion [4].
It's important to understand are that there's a wide spectrum of opinion about this stuff. If you imagine a slider with "allow zero posts about $TOPIC" at one end, and "allow all posts about $TOPIC" at the other end, pretty much every user would slide it to a different position. This is true for every $TOPIC and especially for the biggest ones.
Frontpage space is the scarcest resource HN has [5] and every reader has a different 'signature' of preferences that they would like to see (or not see) there. This means not only that it's impossible to satisfy everybody, but that it's impossible to fully satisfy anybody—because nobody's 'signature' is perfectly matched on the front page, and (lest any of you be thinking of this quick riposte) certainly not the mods'!
[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
[3] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
[4] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
[5] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Have you ever considered writing a book about what you've learned about moderation and community?
You seem to have developed these concepts pretty extensively. Seeing you break down this terminology whets my appetite to hear from you in long form.
1 reply →
If I could, I'd hibernate until such time as I didn't have to hear about generative AI anymore.
I agree, I don't find it very interesting.
It took about 10 years for the crypto headline hysteria to taper but it might have been only because ai is now the big annoying thing to shoehorn into everything. Monkeys pawl will curl and you will emerge out of your hibernation to more disgust at whatever the next annoying thing will be.
6 replies →
I'm sick of LLM-related news. I'm fascinated by the technology and the progress, but for every one article about something novel, there are dozens rehashing the same points about social impact, bias, deepfakes, plagiarism, etc. These topics are of some interest to me, but the vast majority of the articles bring nothing new to the table and are reactionary responses to the latest infraction.
I believe that is close to what the median HN reader feels: interested by the significant new developments, fatigued by the endless incremental updates, and grossed out by the hypemeisters.
I make my money building things with LLMs and even I am tired of reading about them
LLMs are like crypto, where scams and scam-adjacents are everywhere.
I am the biggest local ML advocate you will find. My 3090 is either running Yi 34B queries or other experiments all day, my job is with local LLMS... But I am totally OK with heavy handed AI-related moderation. I dont want the sea of AI grifters to have a single second on the HN front page.
Yeah me too and I also wouldn't flag them. I flag things that are false or misleading or just especially stupid.
I will occasionally flag things that will result in discussions that are always the same because I'm tired of them. Stories about tipping at restaurants or Trump or Biden, for example -- literally every argument for or against has been made and there's nothing new or interesting to say. But I'm more likely to hide them.
I would also include the periodic Monty Hall re-post (everything that ever comes up in the discussions can be found in the Monty Hall problem wikipedia page).
And also pretty much any article about inflation.
Why would one ever flag stories they believe will result in the same useless discussions rather than just hiding them?
I think I've only ever flagged one or two instances of spam personally.
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I think the answer to this is... go set up your our LLM News web site and build a community. I really love HN but I wanted more retro computing and gaming news so I created by own site (https://twostopbits.com/) using the HN source code. It's not hard. Go build the thing you want and moderate it.
I've been in various online communities for over 35 years and I can tell you that by far the best moderated and longest successfully running community is HN (for a while The Well was amazing).
I hadn't see this before, but that is exactly what I've been wanting more of - thanks for setting that up John! I'm curious, do you feel like lower traffic communities like yours serves up the content mix you were hoping for when you started it?
I have no idea. I created it as an experiment. It's fairly active despite being a few months old. We'll see what happens.
2 replies →
> It's not hard. Go build the thing you want and moderate it.
Does the source code include moderating tools, or is it just a bare bones aggregator with a default ranking algo?
There are moderation tools. I can do things like kill a story, ban a domain, ban a user, alter the score on a story, mark a story as dead, lock a story so no one else can edit it, see how many sock puppets voted for the story, edit any aspect of a story.
I modified the default source to have a concept of tags on a story because I wanted people to be able to filter stories by their areas of interest (e.g. everything Commodore 64: https://twostopbits.com/tag?q=c64). All my changes are open and here: https://github.com/jgrahamc/twostopbits
1 reply →
Wow very cool love the retro angle. Assuming you are using Workers for this?
No, I use a lot of Cloudflare products (the domain is registered through Cloudflare, the site is proxied and protected by Cloudflare, I use Cloudflare's free Web Analytics), but I am not using Workers for this.
The HN source base is a monolithic Arc program and Arc is in Racket/Scheme. To use Workers I would have had to get Racket working on Wasm which I simply haven't tried. Also news.arc does a bunch of file system access and I'd have to rewrite that to use Workers KV or something. So, I decided to use lots of Cloudflare and run the Arc code on a VPS I've had for many years. The whole thing is running in a screen session which I can hop into and be in the REPL when I want.
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news
Surprise! Yes, We are!
[flagged]
Believe it or not, it's possible to be both interested in LLMs and also feel that there is too much LLM content on a given aggregator.
I fully expect AI to be huge in the future, and only get bigger.
Doesn't mean I want 50% of HN to be about AI.
1 reply →
The corpse of blockchain would like to have a word with you :)
13 replies →
Eh, what I see popular on HN tends to be the trend 5 years down the line.
Don’t the vast majority of these get removed via flags from users?
Edit: I’m not asking a rhetorical question. There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page. Can anyone attest to this?
This is accurate, per dang's comment on the Gary Tan thread the other day:
> We didn't flag the post; users did. When it comes to submissions, that's nearly always the case - see https://news.ycombinator.com/item?id=39169622
Just to clarify one misunderstanding: most flags on submissions (nearly all actually) come from users, not mods. So if you see [flagged], it's almost always there because of users and in many cases the mods haven't even seen it yet.
But there are other ways besides flags for stories to fall suddenly off the front page: software penalties (e.g. the flamewar detector, a,k.a. the overheated discussion detector, various abuse detection systems, etc.) and moderation downweights. Users don't do either of those.
These points are covered in the FAQ although necessarily tersely. See "How are stories ranked?" and "What does [flagged] mean?" - https://news.ycombinator.com/newsfaq.html
There are stories on this list that deserved to be seen, were popular, were important, and were not in fact dumpster fires in the comments - but a particular crowd with a particular bias decided to flag them.
Example 1: https://news.social-protocols.org/stats?id=39142094
Example 2: https://news.social-protocols.org/stats?id=39130652
Example 3: https://news.social-protocols.org/stats?id=39214844
Does this crowd think it's cool and normal that all discussion of the ICJ's decision - truly momentous - were completely removed, based on the opinion of a dedicated minority?
US tech giants are heavily implicated in this, so no one can seriously argue the topic isn't relevant. A World War could come from these "plausibly genocidal" actions, which are enabled in various ways by US tech giants.
24 replies →
> There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page. Can anyone attest to this?
https://www.newyorker.com/news/letter-from-silicon-valley/th...
I don’t get the impression from that article that Daniel and Scott are curating the front page in the way the thanks in this thread suggest. I am still of the impression that the front page composition is decided by upvotes, downvotes, and flags. Contrary to the implication in this repos’ text.
4 replies →
> There are a lot of comments in this thread thanking “the mods” and I didn’t realize there was a mod team cultivating the front page.
IMO this happens because fundamentally people have "The reddit mental model" about how moderation works here, as if moderation is some privileged, limited position. It's just wrong.
Yes, there is dang, the single admin who posts publicly, and I guess it's possible/probable there are other HN admins who assist him. But 99.9% of the time when I hear people complaining about "the mods" or "power tripping mods" or "censorship", it's basically that other users saw what you had to say, and we just don't want to see it here.
It's also weird that occasionally people think there is some sort of "rule" about what can be flagged. There are obviously guidelines, but as this power is held by any normal user, it's basically whatever they want it to mean. For example, I frequently flag stories where I think the topic and article is totally valid, but where every single time I've seen the topic debated on HN it becomes a useless flamewar or is filled with the lowest quality commentary. At least for me, flagging isn't a value judgment on the "worthiness" of an article, it's simply about stuff I don't want to see on HN.
> IMO this happens because fundamentally people have "The reddit mental model" about how moderation works here, as if moderation is some privileged, limited position. It's just wrong.
Partially, but I think these are all symptoms for a more fundamental root cause: HN is just comprised of too many emotional, passionate users with fundamentally differing beliefs.
The usual song and dance with flagging goes something like the following with cryptocurrency:
1. User posts cryptocurrency article
2. People who passionately hate cryptocurrency start adding in emotional comments about how they hate it.
3. People who want to fight this passionate hate respond in kind.
4. The thread turns into a giant argument where nobody is willing to concede anything and everyone is just shouting at each other.
5. Either the flamewar detector kicks in (as it should) or everyone not in the thread tires of the shouting and flags it.
That's fine but regrettable when limited to some topics like crypto. But it's happening with social media company earnings reports, layoff posts, RTO discussions, posts about Musk, autonomous vehicles, and on and on.
dang (and the mod team?) are doing great work, but this is despite the feeling I have that HN is barely being held together into a cohesive community, and I'm struggling to even use the word "community" here. I feel the temperature of discussions has gotten a lot hotter here than it used to be and some basic work I've done with sentiment classifiers on comments here mirrors my perspective.
I just don't think a single community can handle so many passionate, opposed groups. It bubbles up by proxy in these sorts of flagging wars where so many articles get bumped off the page due to the inability of the community to discuss it well. Maybe the solution is to just discuss software as some people really want, but even then you get massive flamewars over things like Rust async. Even with interesting topics like VR posts, the overall temperature of the comments here is high enough that I've stopped bothering to comment as much as I used to.
3 replies →
That's not what the application is measuring:
https://news.ycombinator.com/item?id=39231055
> In the case of the first, the Story was among the first on the Front Page, until its title was changed from "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes" to "OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2". This effectively "killed" the Story.
> In the case of the second, the Story was in third place on the Front Page, less than an hour after the submission. In this case it was simply removed from the Front Page.
With repeatedly getting flagged articles like this, at some point you have to begin to wonder if you are not simply spamming the community by trying to promote your links.
I get that people want to promote their stuff but the community has preferences too. The community can get tired of LLM articles reaching the front page everyday! The community can refuse to be spammed and the community can flag articles!
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
Denial? Why is it so hard to believe that HN users would get tired LLM-related news. I get tired of it myself but I don't have flagging privilege. I find it very believable that HN users who have the flagging privilege might want to flag LLM-related news.
> I don't have flagging privilege
Looks like you should? Is https://github.com/minimaxir/hacker-news-undocumented?tab=re... wrong? You need to be on a comment's page; click its timestamp to get there.
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
If you can't fathom people being tired of LLM-related news, have I got an NFT for you!
> The assumption is that a Story cannot go from the top 30 to a position higher than 90 in a single minute, without having been explicitly removed.
I'm not sure this is a valid assumption. https://news.social-protocols.org/stats?id=39089599, seems to be incorrectly detected.
Honestly only 3 or 4 out of 13 look like possible moderation to me. And they don't seem bad. Does a story about razor wire in Texas belong on hacker news? I'm in Texas so the story is of interest to me but I'd expect to hear about it elsewhere, not on HN.
Overall it just makes me think HN is doing a good job at moderation.
I dont understand what "explicit" means here.
If it gets algorithmically deranked for user flags, but but not hidden, is that explicit?
I assume "explicit" means manual moderator intervention, but I don't really see anything that suggests that.
> I assume "explicit" means manual moderator intervention, but I don't really see anything that suggests that
Because there is no visible indication when moderator intervention happens. It doesn't mean it doesn't happen.
It's not a valid assumption. I'd better post about this at the top level: https://news.ycombinator.com/item?id=39231537.
> Does a story about razor wire in Texas belong on hacker news? I'm in Texas so the story is of interest to me but I'd expect to hear about it elsewhere, not on HN.
Perhaps in TX you don't realize it, but it's a big national story, implicating the Constitution, federal authority, even the Civil War.
It's political, for sure; but it's not local.
HN is present in more than just the US
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I consider myself very optimistic and often naive, but even I would not be surprised by this kind of HN user reaction :D
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I am. Completely sick of it! Thanks dang for your diligent moderation.
When I had more downtime I’d spend a lot of time browsing /new.
There’s a wealth of great blogposts that show up there which don’t always make it to the front page (understandable; we only have so much attention to give).
What I will say is that there is a ton of cruft that spams the board. Thinking of spammed blog posts from one or more accounts, sensationalist news, etc which wouldn’t provide much value here.
Flagging really helps on /new IME. It’s worth spending time there if you haven’t tried HN other than via the front page
>it's hard to believe that HN users would be tired of LLM-related news
I know there are already a handful of comments about this line, but wow! It bears repeating: My eyebrows almost shot off the top of my head when I read this. What kinds of things does the author find easy to believe??
This is a good thing to be able to see, but I'm much more interested in identifying the soft censorship in comment sections.
This is obviously harder, because vote totals aren't publicly available for comment sections, but it is much more important as a tool. What topics are on the front page is much more clearly the legitimate domain of moderation than what commentary is made about them, especially when moderation of those comments contradicts the vote mechanism.
It's more boring than you'd expect. The comments that get most heavily upvoted tend to be either (1) indignantly rhetorical, or (2) generic. Mostly what we do is downweight those when they're at the top of the thread. The "algorithm" is about as simple as:
(1) if the top comment is indignantly rhetorical or generic, downweight it; otherwise go look at another thread;
(2) refresh the page;
(3) goto (1).
If I'm feeling diligent, I might do this for the top few subthreads, but that's about it.
This simple intervention turns out to be the highest-leverage thing we've figured out in recent years:
ttps://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20highest%20leverage&sort=byDate&type=comment
If it's truly a service that's being provided by your superdownvotes, I wonder what would happen if you let people opt out of it.
I would rather see the highest voted comments, and I'm pretty happy to just scroll down myself if I find the top comment to be useless.
I wrote a program that tracked the changes in story titles on Hacker News a while ago. Some of them were really quite strange. I can't understand some of the policies, like why the words "How" and "Why" are stripped from the front of titles (eg "How I rewrote my app" would be changed to "I rewrote my app"). Some very small proportion of the title changes could definitely have been construed as politically motivated but overwhelmingly they were benign.
did you write up your findings anywhere? I'd love to read about them!
I didn't, mainly because the findings were so mundane! I still have the data, so perhaps I will some day.
> While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news.
I don't know about everyone else, but I sure am, and I work on them for my PhD.
Good moderation is exactly why I check HN every day and not so much other places. Thanks mods!
Can we please have this in reverse chronological order - later dates at the top? Also, another request -- can we change this to TABLES instead of the plain Markdown list. I believe this is where a tabular display will be much easier to browse.
Thanks.
> I sent an email to the moderator. @dang, who was very kind and quick in his response, explained to me that the Story had been flagged by users even without being explicitly [flagged], and that he could therefore only hypothesize the causes of the flag.
Maybe this is a consequence of Hacker News not having a way to downvote stories?
I only flag stories that are blatant violation of HN's guidelines: SPAM, politics, racist... Otherwise, if I don't like a story I don't do anything.
Maybe I'll start flagging stories that I don't like?
Yea, I suspect a sizable number of people use "flag" as a mega-downvote for things they passionately don't like, rather than for policy violations and spam.
For anyone that is concerned about over flagging, please consider turning on showdead and vouching responsibly!
If as many people thoughtfully vouched as maliciously flagged it may be less of an issue.
4 replies →
[flagged]
Yeah, like you I would have never thought to use the flag unless it was violations etc but turns out it's a weird behaviour of many users. Not sure why or when it originated but seems like its been driving the up/downs of a number of topics/posts for years now. Still use with moderation.
Where do you see that quote in the OP? I searched for it and didn't find it.
"WHY? Feel free to skip this part or click to expand"
looks like you didn't feel free.
Reddit used to have a similar thing (r/RedditMinusMods). Eventually when it got to the point that 48-50 out of the top 50 posts were removed every 12 hours, reddit banned the sub for unspecified reasons.
http://web.archive.org/web/20221110043732/https://old.reddit...
Spammer wonders why stories are being "removed".
How can you complain about your submissions getting hundreds of upvotes and a bunch of discussion over the last 4 months only. That's a decent amount of eyeballs.
Other than the blatant offtopic/spam ones, most of them are just ones that have drifted away and are old news, or flagged, or dupes. It's driven by the flow of the site and its users.
i feel like every news source, forum, link aggregator, ... has its own target audience and scope of topics that make for productive discussion, its own biases and predispositions, its own trolls and need for pruning and moderation.
i feel like yes of course there are many things i disagree with on this site. but ultimately i value the information shared and the discussion enough to keep coming back. any relationship where people always agree there is probably only one person doing the actual thinking.
i have learned so much about tech here, i have learned about many best practices and projects that i would have never heard of, i have made no bones about my thoughts on various subjects that could easily be classified as touchy, i have really enjoyed the discourse. for the time being i definitely plan and hope to continue doing so.
(so while this site is an interesting artifact, and maybe it is good that someone is taking a look and keeping a record, i personally won't bother unless/until i see a pressing need. at which point i will maybe just move on instead tbh.)
I think you're going to see people start leaving Reddit as the IPO approaches and many will be coming to HN or other reddit-clones.
Half the comments on Reddit really do seem to be made by bots, you can easily tell when you look at their post history.
I did something with a similar idea. Rather than looking at deleted stories, I historicize the stories and display them as a graph. You can see which stories have a second life (they were created several days ago, but the score doesn't increase until several days later). https://y-combinator-news-trends.vercel.app/
You can see your story!
The github isn't open source because the project isn't really finished (the page is ugly, by the way).
I have to say that I'm not relying on the api but on scraping the front page. The reason was to migrate code I had from python to typescript (I'm better at the later...)
> Using the official HN API, the service fetches 90 Top Stories every minute and makes a comparison with the first 30 Top Stories (i.e. the Front Page) fetched the previous minute. It logs all missing Stories here. The assumption is that a Story cannot go from the top 30 to a position higher than 90 in a single minute, without having been explicitly removed.
The OP's hypothesis is that, if rank drops from top-30 to below top-90 (I think "higher than 90" is a typo?), in less than a minute, then it must be due to moderator action.
Is that true?
No, it's not true. I have many times been the person who hits "flag" and then refreshed the page to find the story disappeared as [flagged]. When that happens the story is completely gone, off at least the first 3 pages (which is what OP is measuring).
It's definitely not true, as lolinder has pointed out. See https://news.ycombinator.com/item?id=39231537 for more.
I think it would be really interesting to get a tracker for the ones that hit the front page, then fall off, and then surge back in 30-60 mins. I wonder what's going on with those ones.
Most probably that's us turning off the flamewar detector.
If you see examples you're welcome to ask us at hn@ycombinator.com.
Interesting service. I think it would benefit from further improvement. Many stories are actually dupes, self-promotion, etc… It would be nice to see a much smaller list of stories that were actually censored by the moderators or self-censored by the community.
How to do this? One idea is to write an appropriate prompt for GPT-4. Something along the lines of “if you were HN moderator or HN community, would you censor this story? Please give numerical score.” Then post a much smaller list with top scores. That would be useful I think.
I started collecting the /news feed something like 7 years ago in a script, I think it's still running. It's fascinating watching stories get dropped or auto-killed and then running stats to find out what the algorithm is trimming. I think I started it because there'd be things changed or removed and you couldn't tell unless you had looked at it before it was culled. At some point I'll kill the linode it's running on, maybe move it to a Lambda, push the database to GitHub.
> It's fascinating watching stories get dropped or auto-killed and then running stats to find out what the algorithm is trimming.
So what did you learn ??
Yesterday I posted a link that was flagged off the front page. I can see that your tool is driving traffic to it and appreciate that.
Not everyone wants to discuss political topics on HN. They say there are other places to discuss such topics. I like to hear the opinions of the HN audience on a wide variety of topics. Maybe this tool will help those of us who value the HN community in this way by facilitating discussions on topics deemed inappropriate for the official front page.
I've always assumed that when a story rapidly drops past the first three pages (e.g. >90) it is because it has been flagged some indicative, disproportionate number of times[1] by users with flag functionality. The submission seems to presume that such drops are only the result of Daniel or whoever manually doing it.
Which is true?
[1] - Notably the majority of the "removed" stories have pretty tiny number of upvotes, so if flags are weighted proportionately it wouldn't take many.
Mostly flags, yes, but also software penalties and moderation downweights. More at https://news.ycombinator.com/item?id=39237446.
It looks like some of these are cases of duplicate threads being migrated, which isn't completely obvious when looking at this Github page.
For example https://news.ycombinator.com/item?id=39219568 was just a dupe. Maybe that's the case for some of the more technical stories that are removed.
I’ve wanted something like this for years. My top requests would be to offer this as a standalone separate site, with sorting in reverse chron order, and an easy way to click into the comments for each story. Right now I can click into the info for the link, and to the outlink, but there doesn’t appear to be a one-click path to the HN comments themselves.
I'm honestly not sure if that's a bad thing. To an extent, it seems like reddit's "sort by controversial". If you're looking for a flamewar, here you go.
It's interesting to see the comments sometimes, but since part of the reason these things get removed is because of the flamewar detector, I feel like I can't be that surprised or edified when I open the bag labeled "manure" to find it is full of shit.
Are there HN metrics available that include voting behavior and viewing behavior?
For example, is it more commonplace lately to comment without upvoting, than it used to be? (Upvoting the post and/or the parent comment.)
And how has the comments-to-pageviews ratio changed over time?
Fine with me! I keep coming back here because the site is relatively un-cluttered. Thanks mods!
I love HN and I think the moderators are doing a great job. But could one of the mods explain the logic with some examples from the Github repo?
just trying to see what makes the moderation good :)
I spent all day posting 70 or so comments in this thread trying to explain exactly that.
If there are specific examples I haven't addressed yet, post links and I'll take a look!
I think my comment was early before all your responses, thank you for your work! I didn't expect to get an answer from Daniel Gackle himself - shows the amount of dedication.
Only saw one Gary tan link removed. I thought I'd see more. Maybe it was only removed because it was a dupe? I'm referring to the "Gary tan tupac lyrics" one.
There was a bunch of Gary Tan links -- you can see in my comments I was arguing with a bunch of HNers today about whether he's right or wrong on that "Die Slow" tweet. Probably dang removed it because it's a dupe story.
He could have phrased it a little better but the people calling for his removal from YC are just plain silly.
Thank you. Already saw one I would like to read.
What is criteria to remove some of these.
I've read the 'terms' for submitting. sometimes the removed ones don't appear to violate anything.
The key word there is "appear". For example: sometimes the story is fine except that it has already had a big discussion recently.
I am more interested in the mechanics of how something like this works, especially over time.
All kinds of tools related to HN content generate front page interest even for days but then once that passes things that cost money or use unreliable free resources start to disappear at an ever more rapid pace.
When they don't, the UI can become unmanageable... I'm not sure how this content will be organized over time but updating the README won't be tenable for long!
this is an excellent project, which highlights what a stellar job the moderators are doing
It's great that they offer the source code so you can modify it and run the program on your own hardware to make sure stories aren't being removed from the stories removed from the Hacker News front page list.
While the above is me joking, I appreciate the extreme transparency that showing code and explaining methodology provides. This adds more credibility than any other single thing the author could have done.
I wrote something similar, it's a simple Python tool that estimates what a story's rank should be based on the score, and compares it to the actual rank. It's rough and only tested on Win 10 so far:
https://github.com/calibas/hacker-newd/blob/main/hacker-newd...
Long overdue transparency. Sometimes these are innocuous or warranted removals, but there is also an element of protectionism at play. And that may not even be due to mod actions, but blocks of users who all flag articles to get them pushed to no mans land.
There are companies who if you submit a negative post about, within short order the post is pushed out of view of the top pages.
>blocks of users who all flag articles to get them pushed to no mans land
This is just another way of saying that a critical numeric threshold of users didn't like something. Framing the opinions/actions of groups of people on the internet as conspiring or dog-piling is a fallacy. E.g. if a person Tweets something that a million people read and a hundred of them reply to disagree, you'll often see that person follow up with something like, "wow, now all these people are attacking me", even though everybody acted in complete isolation and did nothing strange or harmful individually. Nobody rang a bell in the town square and handed out pitchforks. The internet breaks human psychology.
Except when it's not. If you don't think groups within organizations all message each other to quickly flag posts that are negative towards them, then you may be looking through this with an idealistic lens that hasn't been shattered yet.
I'm not denying your premise that yes sometimes independent people with no coordination, all flag an article. That is how the system should work. But there are also articles that will quickly get flagged through coordination of interested parties.
Hacker News has a lot more power than many think in terms of tastemaking in the tech industry. So there is a lot of motivation and benefit for people to manipulate its functionality to either boost or protect their business.
Some feedback by the way: might want to sort the dates in reverse chronological order so the newest removed stories show up first :D
I went to check if the story I submitted today (Breathing 101 — https://news.ycombinator.com/item?id=39227295) was present in the list, and it was indeed! I don't know how it reached the front page and why it got removed.
I would have downvoted that for a number of reasons. I guess people flagged it, since there's no downvote. I don't flag unless it's something egregious (racism, spam, etc). But since this thread is all about the topic of why stories might not make the cut here, this is why I would have downvoted it if I could:
1. "Breathing 101" is an uninformative headline. I correctly guessed that it was literally referring to the human act of breathing, but it's still a bad title (I know it's not your title, and that HN encourages using the original source's title; it just sucks).
2. You submitted the link with no comment or context about what the article was or why it might be interesting. If a headline grabs my eye, I always click on the "N comments" link and the article link to open two tabs, and I look for additional descriptive text from the submitter, or a comment from them about what they found interesting. Sometimes I read the actual article first, but if the title is ambiguous or the topic is contentious, I'll usually start with the comments tab and see if I'm going to be wasting my time before I read an article. This alone wouldn't be a reason to downvote, but if I was leaning that way, it would factor in.
3. The word "wellness" in the link's domain is a huge red flag. To me it means "this is going to be a bunch of hippie crap". Not a primary factor, but seeing that would be enough to make me dig farther and find evidence so that I could Angry Downvote something I don't want to see on HN ever, if we could downvote. Yes, this is petty.
4. The very top of the linked article says "Click here to make an appointment". This indicates spam.
5. The article is just bad. There's not much information. It's not scientific. It touches lightly on some potentially interesting things but doesn't dive into them at all, or link to better sources, and it ends with what appears to be advice and encouragement to incorporate breathing exercises, but without much information about how or what the benefit is.
It looks like spam. It's the kind of clickbait that floods my Facebook feed.
Saved one more time. The whole internet will judge.
https://web.archive.org/web/20240203104237/https://news.ycom...
I wish we had meta-moderation and voting and flagging reasons like on Slashdot. Flagging as a super-downvote seems user-hostile and passive-aggressive for a site like HN, but I’ve seen it tacitly acknowledged as allowed as okay on HN by dang, which seems counter to the spirit of the guidelines.
Imagine this itself getting removed... :)
> "literally the first rule of HN moderation is that we moderate less, not more, when YC or a YC startup is involved"
source: https://news.ycombinator.com/item?id=39172045
That rule does not generally apply to HN meta debates, which this is, so it's a bit of a corner case. If there was a duplicate of this story on the front page tomorrow, I'd expect HN to honor the user flags on it.
This story has close to 500 up votes as of 2:19PM Pacific and is no longer on the front page. Why flag an article that is simply trying to show how HN can potentially be improved?
It got flagged by users. We turned that off per https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... The story has been on the front page for 9 hours now and is still there.
What happened with this one?
https://hnrankings.info/39073285/
It just disappeared from HN.
I thought of creating something similar for flagged comments, way too many people are flagging things because it doesn't suit their narrative
HN would be a much better place if they banned domains like the guardian, nytimes etc entirely. There's way too much journospam.
The problem is that while the major media sites generate a lot of noise, they also publish solid original articles from time to time, and we don't want to miss those. For that reason, they're downweighted, but not banned.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
I ran my own client side script to strip out links to such articles. New computer, didn't copy the script over, and now I'm engaged in passive aggressive discussions over some guardian article. It's my own weakness of course, but I am definitely not the only one.
2 replies →
While I understand why "Secret Plan against Germany" - a story about Germany's far right planning "remigration" of even citizens - was removed, I still would've liked to see the part of the discussion that wouldn't have been arguing about the semantics of the term "nazi". This is the article that sparked a never seen before mobilization of demonstrators against these planners and their party in Germany.
This is an interesting dataset. I suspect the main things that get removed are A) politics B) duplicates.
I'm echoing others, but the article on rust async/await seems good.
You may or may not agree with the conclusions of the post, but its a technical topic with at least some specific exploration of the (performance/code writing) issues, that links to quite a few further topics for exploration.
https://blog.hugpoint.tech/avoid_async_rust.html
That said, I noted more than a few typos in the article, so I wonder if there is generally a spell check filter for article quality.
https://trunk.io/blog/git-commit-messages-are-useless
I also found this one interesting. I don't agree with the article, but its an interesting viewpoint and I learned a bit about what some people are doing with git. I couldn't tell you why it dropped (unpopular)?
Which is where its possible that this (new) tool falls short, it can't actually tell what was censored, just what wasn't popular.
Unpopular things sometimes are so because they fly in the face of conventional wisdom, but aren't actually wrong or invaluable, which might be the real value of this tool.
> but the article on rust async/await seems good.
It is really not. It is a rant that produced no good discussion anywhere else on the internet. It has no novel insight and is dressed up in a really ugly way. I'm not saying HN should have removed it, but I don't mind that it got flagged.
I wrote a whole thing about that one in case it's useful:
https://news.ycombinator.com/item?id=39235382
Hmm most seem to be idiotic, spam or off topic for HN.
I wonder if a useful application for these "AI"s could be to pull interesting - to someone - stories from what the hive minds rejected ;)
Just to be clear, this is stories that got completely removed off the front page and does not include whatever is still available 4 pages behind but got overtaken by other stuff?
Sometimes I see an interesting heading but skip it, and when i reload it's gone. I doubt they were all flagged into oblivion.
Who's going to make this into a training data set for dang-bot?
Can we have it sorted in descending order by date?
It'd be interesting to see removed vs flagged, if you can scrape flag kills.
The flagging system is a great utility, but certain things (e.g. anything pro-Musk) get mass flagged for emotional reasons.
I really don't like Musk but I don't flag things Musk related. I frequently upvote them because I'm interested in the discussion.
Unfortunately those stories often turn into flamewars. That's probably why people are flagging them.
I don't think it's wise to draw so many inferences about why people vote the way they do. Frequently I see comments where someone makes a reasonable point, but also drops a bunch of flamebait, and when they're inevitably flagged they edit their comment to claim that the flags prove their point and that the problem who disagree with them are overly sensitive and censorious. But in reality a lot of the people flagging them probably agree with them, but don't want them to start a flamewars. I flag a lot of comments like that, even when I am agree with their overall point. (I actually did that with a comment just now.)
It's a form of self fulfilling prophecy and further entrenches you into your position, which is antithetical to curious discussion.
> certain things (e.g. anything pro-Musk) get mass flagged for emotional reasons.
Lots of Musk stuff, including positive stuff, on the front page. Yesterday there was a story about petabytes of data on the Starlink laser network, based only on Starlink PR afaict.
Looks like good moderators sitting and managing
"While I have no reason to doubt Daniel's good faith, it's hard to believe that HN users would be tired of LLM-related news."
If you believe that HN is a hive mind and all users must believe in the exact same things, then yes, this is probably hard to believe.
I however, am tired of LLM news, but I just simply ignore them as I'm well aware that many people here are very much interested in them. So at least an anecdotal response of one that some HN users are tired of LLM related news.
You might also be surprised that not all HN users like social media while some do. Some are very privacy conscious while others will freely post all of their everythings to anywhere. You might find it hard to believe that some lean left while others lean right with some even landing straight in the middle. Why you would think anything is hard to believe in this day and age is very strange to me.
That "While I have no reason" line has been quoted in six top-level comments so far, obviously it struck a nerve here.
It would be ultra-cool to have rough topic filters here, so I could just go to settings and hit a checkbox to ignore all the LLM-this and AI-that articles. Easier said than done, I'm sure.
Any time you paint with a broad brush with comments like that, you're going to miss some of the details. Looking at the time stamps of those comments shows they were pretty much at the same time. I use the phrase "group think" a lot, but intentionally do it to in part rabble rouse, but also to get those in the group think to maybe think and take a second to question if it truly is group think behind their current position.
> You might find it hard to believe that some lean left while others lean right
And then you get those of us who are simultaneously left-of-left and right-of-right...
I think this is an unnecessarily uncharitable reading, that the author assumes HN is a hive mind.
Replace "HN users" with "most HN users" (it's common to use general language when one's intention is to point out a trend in a population) and, as another person tired of AI/LLM news, I would also be surprised given how much popularity (upvotes, comments) HN users tended to give to those stories.
* 96% of U.S. temperature stations fail to meet what NOAA considers "acceptable" [pdf] ? why remove
[dead]
The first two links I clicked on were week old stories. I think you should filter the results by date posted.
i think it's just sorted in the wrong order
> it's hard to believe that HN users would be tired of LLM-related news.
You find it hard to believe that users are fatigued by a subject that is posted ad nauseam? People have been tired for over a year.
https://news.ycombinator.com/item?id=34722220
man I would love some sort functionality here
RSS feed?
the front page has definitely become less general interest and more niche tech in the last couple months.
> In fact, I don't see a single story that I personally feel should not have been removed.
I don't understand why this story was removed: "It turns out the six-feet social-distancing rule had no scientific basis", https://news.ycombinator.com/item?id=39200511
On a forum with an overwhelmingly science-minded audience, it bothers me that an important topic like that is deemed untouchable.
(I detached this from https://news.ycombinator.com/item?id=39230689 for reasons explained below.)
The subthread your comment generated here already answers your question. (<-- not a criticism! just an observation.) People are flaming each other about the inverse square law, droplets vs. aerosols, who is refusing to face reality, and sundry other nastinesses in the comments below. It demonstrates what a shitshow a frontpage thread would have been.
It's not that the topic itself is "untouchable". HN had quite a few threads about the lab leak hypothesis for example. But these things are sensitive to initial conditions, and something about the way that headline frames the story feels doomed to me, from an HN point of view. The sweet spot for HN is substantive, thoughtful conversation driven by intellectual curiosity. That's what the site is for. We don't always get there by any means, but I only want to turn off user flags when the odds give us a fighting chance. I remember seeing that story get flagged and thinking: it'll never work.
Another aspect of this: like it or not, curiosity and repetition have an inverse relationship. After the mind has been hammered with the same hammer enough times, curiosity gets sick of it and goes "ugh, not that again". That means that on a topic like all-things-covid, which we all got hammered with, the majority of the audience, who don't care that much, check out at first mention of the topic. Who does that leave? The ones whose motive is more intense than mere curiosity.
From an HN point of view, that's a ticket to hell. Curiosity can only operate within a certain range of nervous system activation. If the needle sinks too low, the topic is 'bleh' and nobody cares; but if the needle goes into the red, people will care—my god will they care—but they'll no longer be functioning out of curiosity. That's a failure mode for HN.
When it comes to divisive, heavily-covered topics like that one, the thing to watch for is some kind of interesting new information that isn't entirely reducible to existing battle lines. The same forces driving the thread into flamewar will still be present—but at least you'll have some current running the other way.
Stories about COVID controversies are almost certainly getting flagged off the front page by users, not touched by mods. People look at the titles of these stories and think that's all flaggers are going by, but lots of people flag stories based on their experience of what the threads are like, and the threads on COVID controversies are fucking dreadful. I didn't flag (or see) that story, but I would have.
But why must they be dreadful? Genuine question, I am not being obtuse. We should be able as a community to discuss conterversial subjects somehow.
I also think this sort of thing invites flag brigades. Or better yet, a small batch of bad actor can easily start brigading and forcefully associate such flamewar expectations with any subject they don't like to drive it off HN.
Maybe worth reconsidering how you flag? You might be getting played. Or not, I really don't know. No obvious answers.
32 replies →
> lots of people flag stories based on their experience of what the threads are like
IMHO story submissions should be judged based upon their own merits. Toxic commenters can be downvoted/banned but the story submitter shouldn't be punished for the misbehavior of others.
> I didn't flag (or see) that story, but I would have.
You mean purely based on the expected awfulness of imagined future comments, instead of the actual comments? If so, with a precrime mindset like that, you're fanning the flames of controversy.
4 replies →
I flagged that article, so I'll clearly explain why:
1. I think for anyone that has been on HN throughout pandemic knows it is extremely unlikely for topics like this to produce any sort of valuable discussion. I almost never see any sort of humility on the topic (to be clear, from many/all sides) that admits that people (individuals, experts, literally everyone) were doing what they thought best with the information they had available at the time. It always devolves into portraying the other side as evil. I'm tired of it, I don't want to see it on HN, there are literally pages and pages and pages of place on the Internet where you can have that debate if you're so inclined.
2. Are you honestly purporting that specific article is well tailored to "an overwhelmingly science-minded audience", as opposed to just having a particular political axe to grind, given the title is "Anthony Fauci Fesses Up"? Honestly, if the article was written with an intent to encourage an actual understanding about where the 6-foot rule came from, and about whether the evidence for it was lacking, I probably wouldn't have flagged it.
> it bothers me that an important topic like that is deemed untouchable.
I think the mistake you are making there is thinking because a particular article is flagged by a lot of users that "an important topic like that is deemed untouchable." I can't speak for others, but for me that is absolutely not what I think, and it's not why I flagged this particular submission.
That isn't clear at all. You seem to be saying that if you anticipate that people might question other people's competence or motives, or in your view a discussion won't lead people to think the right thoughts ("encourage actual understanding") then you flag it to try to ensure nobody can discuss it.
But you also say that making it undiscussable is also not about making the topic untouchable. That's just playing with words, isn't it? It's exactly what you're trying to do and exactly why you're flagging it.
This particular case is really egregious. Fauci has said this draconian policy "just sort of appeared", yet you damn anyone questioning his competence or motives as lacking humility? What would it take for you to allow criticism of this guy?
1 reply →
(Shrug) I don't require scientific proof of the inverse-square law. It's self-evident to the point of being axiomatic. Standing 6 feet away from a virus source will expose you to about 44% fewer virus particles than standing 5 feet away from one, while not imposing any real hardships in most public interaction scenarios. What's controversial about that?
If you demand precise scientific rigor in all aspects of everyday life, public health is probably not the career field for you.
The same with masks:
Put a water hose on mist and spray someone with it. Then put a cloth over the nozzle and try to spray them. It's self evident yet people just could not grasp it.
9 replies →
[flagged]
7 replies →
I think reading the top comment on that post provides plenty of explanation why users would flag that post. Perhaps you're trying not to understand.
Please don't cross into personal attack. It just makes everything worse, and you can make your substantive points without that.
https://news.ycombinator.com/newsguidelines.html
2 replies →
> I think reading the top comment on that post provides plenty of explanation why users would flag that post.
That top comment complains that the HN title is WSJ's informative subheading instead of its clickbaity headline.
2 replies →
Are you asserting that, if the title had remained "Anthony Fauchi [sic] Fesses Up" it would have remained, unflagged?
Maybe you should Submit it again with the original title, and see what happens.
2 replies →
Personally, I thought it was already pretty well established that the six-foot rule was based on poor science. I remember hearing about that years ago.
The thing is, you're not even wrong. The six foot rule was based on what the best understanding of the experts was at the time, and probably saved thousands of lives. Just like forced masking up probably saved tens of thousands of lives. Both were great examples of science, which readily admits to tuning when new evidence comes into play.
However, because there's a right wing cult around Donald Trump, whose fortunes were hurt by the pandemic, the six foot rule and masking and vaccines are set up as straw men and attacked by a gigantic and well funded and organized horde of proxies, including the #1 media network in the US. It goes something like this: because a particular individual got COVID, that's proof that vaccines are not 100% effective and so They Lied To Us For Nefarious Purposes. Or because this particular individual stood 6 feet away and still got COVID, that's evidence that Fauci Is In A Conspiracy With The Chinese. Or because this particular individual survived COVID, it's just a cold. Or because masks are not 100% effective when not worn securely, they are not effective. And on and on.
So it's not unreasonable or unlikely that you heard a thing about bad science and six feet of social distance or whatever. But hearing a thing, and the thing being true from foundational motivations of actual science, are very different right now.
5 replies →
[dead]
[dead]
I demand to know why "Men are going to brutal boot camps to reclaim their masculinity" was removed from the front page!</s>
[flagged]
this post in now at the top, I wonder wether it will be removed ;)
Is there any reason why you would assume this in such a snarky conspiracy-esque tone?
Like someone above pointed out, a rule of moderation on HN literally is, that stories about HN or ycombinator companies itself are moderated less [0].
[0] - https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
Probably because if they bring that tone to enough topics, eventually they will be right. Though also probably not for the reasons they think (e.g. they say a post will be removed because it discusses a controversial topic when in reality it was removed because it was just plain garbage content).
Essentially because users are tired of some topic that threatens them (LLMs or AI) we censor it.
I think this is the beginning of HN becoming irrelevant in its old age. It starts to ignore realities it doesn’t like.
> It starts to ignore realities it doesn’t like.
https://en.wikipedia.org/wiki/The_pot_calling_the_kettle_bla...