Comment by johnfn

Oh, hey, Shog, good to see you doing well. It was a heck of a ride, hmm?

Lots of moderation issues are also UI issues.

I suspect it’s the same issue for whatever is the “meta” in a competitive video game.

Optimization based on the available affordances ?

Best answer so far, too bad way down here.

> the core assumption remained that there was a never-ending spring of new people feeding the thing.

Hi Shog, hope you're doing well! Just thought this bit was insightful; I can fully believe this was the idea and the motivating factor for a lot of the decisions made seemingly in a vacuum (from the outside).

How much do you think Area51 and the push for the SE network rather than sticking with the Big Three affected things? I always got the impression that they tried to scale into places that ultimately attracted too much noise and overestimated the willingness of (community) moderators to effectively work for free for them to take on the wave of less technical/principled users.

I swear that about 3 of your replies look like LLM content or at best "LLM-massaged" messages :-(

Shog9, excellent comment and very apt. I have to point out that you were also part of the toxicity and bad tone. You very much were part of the problem. Moderation and staff were very much the downfall.

>it just felt kinda shitty to those of us who spent more time answering, when we actually needed to ask one on a topic we were lacking in. (Speaking as someone who never moderated).

Great observation. Just like friendship, open communities psychologically feel as though there should be some balance. Spending free time contributing to something (even if you don't directly expect anything in return with ulterior motives) to benefit others, then getting an anvil dropped on your head when you dare to ask for a morsel in return, was an awful feeling which occurred too often there. The site and moderation, especially since the late 2010s (and especially in 2020 and beyond), became malignantly predatory.

[dead]

Very similar to my experience. I never managed to either ask or answer a question on there. Everything I did was "bad" for reasons that were never explained to me in a constructive way that made me feel empowered to get a better outcome.

I used it as a reference when someone had a similar question to mine, but over time the bad taste in my mouth caused me to avoid it in google search results.

I fell into using an early — and I would say, far superior — form of ChatGPT, which consisted of carefully and clearly laying out my question, point-by-point, in a blank text file, and then usually having an insight as to what my particular stumbling block actually was and thereby being able to move forward.

Questions are never really deleted , post a link so people with enough reputation may have a look and maybe resurrect it if the question is really good.

>I'd long heard of abusive moderation but... experiencing it first hand is something else. Anecdote of one, but I know I'm never going to ask there again.

And it was a real gut punch when this would happen (or getting suspended/banned) to long-time users, as well. They largely precipitated their own demise, so I say good riddance.

its not just you, I saw this happen to others' posts many times and it happened to me several times

I gave up on Stack Overflow when my jobs started requiring me to use Terraform and suddenly every time I posted a well researched and well formed question about Terraform, it would immediately get flagged and closed with responses that "Terraform is not programming and thus questions about Terraform should not be posted on Stack Overflow", which was insane to me because Stack Overflow has a "terraform" tag and category. If you visit it, you will see tons of users trying to post valid questions only to have the mods shut them down angrily.

I have been highly active in online technical communities since the usenet days in the 1980s and I have never once found myself in a situation where my opinions on sex or gender were solicited, interrogated, or judged.

This really sounds like people were letting their personal flags fly (in avatars or sigs or whatever) and you could not stand to see that because they were not like you. All you have to do is ignore it and look at the content.

This reminds me of someone I worked with, who asked me "why does [Colleague 2] have to shove his gay lifestyle in everyone's face?" after that Colleague 2 put a framed holiday photo with his husband on his desk.

The person who asked this had a photo with his wife on his desk. He was unable to understand (A) how that is "shoving" his sexual orientation in other people's faces to the exact same degree as Colleague 2's photo was; and (B) that the photo was for Colleague 2's own comfort and solace, and for positive engagement with anyone who wanted to engage in same, and that nobody else was required to dwell on it or give it a second glance.

Quite frankly you have missed the point of my comment.

The late 2010s moderator drama I was talking about was beyond the strict question curation. When StackOverflow expanded into StackExchange and started trying to be another Quora the moderation grew beyond curating technical questions. For years there was needless moderator drama and arguments over how the moderator team should run that were spilling over into social media everywhere.

> outdated answers that are locking out more modern and correct answers. There are so many things for which there is no permanently right answer over time.... I’ve also constantly wished that bad answers would get hidden or cleaned out, and that accepted answers that weren’t very good would get more actively changed to better ones that showed up, it’s pretty common to see newer+better answers than the accepted one.

Okay, but who's going to arbitrate that? It's not like anyone was going to delete answers with hundreds of upvotes because someone thought it was wrong or outdated. And there are literally about a million questions per moderator, and moderators are not expected to be subject matter experts on anything in particular. Re-asking the question doesn't actually help, either, except sometimes when the question is bad. (It takes serious community effort to make projects like https://stackoverflow.com/questions/45621722 work.)

The Trending sort was added to try to ameliorate this, though.

They introduced recent-votes-count-more, perhaps five years ago.

yes I noticed this as well, over the past few years, its happened again and again that the "Top Answer" ends up being useless and I found myself constantly sorting the answers by "Recent" to find the ones that are actually useful and relevant

Having gotten used to SO, I was shocked when I found I could mark multiple answers correct on AskMetafilter. It felt like an innovation.

> There are so many things for which there is no permanently right answer over time.

Yeah it's doubly stupid because the likelihood of becoming outdated is one of the reasons they don't allow "recommendation" questions. So they know that it's an issue but just ignore it for programming questions.

> Having duplicates of the question is precisely why people use LLMs instead of StackOverflow. The majority of all users lack the vocabulary to properly articulate their problems using the jargon of mathematicians and programmers.

Yes; so the idea is they fail to find the existing question, and ask it again, and get marked as a duplicate; and then everyone else with the same problem can search, possibly find the new duplicate version, and get automatically redirected to the main version with high quality answers.

> to properly articulate their problems using the jargon of mathematicians and programmers

I once distilled a real-life problem into mathematical language exactly like how the Introduction to Algorithms book would pose them only to have the quesiton immediately closed with the explanation "don't post your CS homework".

(My employer at the time was very sensitive about their IP and being able to access the Internet from the work computer was already a miracle. I once sat through a whole day of InfoSec and diciplinary meetings for posting completely dummy bug repoduction code on Github.

> The problem is moderation marking unrelated questions as duplicates.

This problem isn't directly solvable (what counts as a "duplicate" is inherently subjective, and therefore mistakes/differences of opinion are inevitable).

I think a deeper problem is that once a question becomes closed (for any reason), it's unlikely that it'll ever be reopened. The factors behind this are social (askers interpret close votes as signals that they should give up), cultural (there's not much training/feedback/guidelines about what "duplicate" means for those with voting privileges), and technical (there's no first-class feature for askers to contest closure, and it takes just as many votes to reopen a question as it does to close it (with the same voter reputation requirement)).

> The problem is moderation marking unrelated questions as duplicates.

The idea was, if there's an answer on the other question that solves your question, your question remains in existence as a signpost pointing to the other one without having to pollute and confuse by having a mixture of similar answers across both with different amounts of votes.

What's sad about it is that SO was yet another place for humans to interact that is now dead.

I was part of various forums 15 years ago where I could talk shop about many technical things, and they're all gone without any real substitute.

> People don't realize what a massive advantage Google has over everyone else in that regard. Site owners go out of their way to try to block OpenAI's crawlers, while simultaneously trying to attract Google's.

Not really. Website operators can only block live searches from LLM providers like requests made when someone asks a question on chatgpt.com, only because of the quirk that OpenAI makes the request from their server as a quick hack.

We're quickly moving past that as LLMs just make the request from your device with your browser if it has to (to click "I am not a robot").

As for scraping the internet for training data, those requests are basically impossible to block and don't have anything in common with live answer requests made to answer a prompt.

I think many would agree that this policy was the single biggest moderation failure of the site. And it would Have been so easy to fix. But management believed fewer high quality answers were better. Management was wrong.

This is because the real goal was SEO.

1.5 years ago Gemini (the same brand!) refused to provide C++ help to minors because C++ is dangerous: https://news.ycombinator.com/item?id=39632959

The problem that you worked out is only really useful if it can be recreated and validated, which in many cases it can be by using an LLM to build the same system and write tests that confirm the failure and the fix. Your response telling the model that its answer worked is more helpful for measuring your level of engagement, not so much for evaluating the solution.

You can also turn off the feature to allow ChatGPT to learn from your interactions. Not many people do but those that do would also starve OpenAI for information assume they respect that setting

Asking questions on SO was an exercise in frustration, not "interacting with peers". I've never once had a productive interaction there, everything I've ever asked was either closed for dumb reasons or not answered at all. The library of past answers was more useful, but fell off hard for more recent tech, I assume because people all were having the same frustrations as I was and just stopped going there to ask anything.

I have plenty of real peers I interact with, I do not need that noise when I just need a quick answer to a technical question. LLMs are fantastic for this use case.

Y'know how "users" of modern tech are the product? And how the developers were completely fine with creating such systems?

Well, turns out developers are now the product too. Good job everyone.

Replying to my own comment surprised that everyone is latching on to just poor moderation on a single site and ignoring the wealth of other options for communication and problem solving like slack communities, Reddit, blog posts, running a site like SO but with a better/different moderation policy, the list goes on and on.

I’ve seen this trend a number of times on HN that feels strawman-y. Taking the worst possible example of the status quo but also yada-yadaing or outright ignoring the massive risks of the tech du jour.

The comment I’m replying to hand waves over “legal issues” and totally ignores the fact that this hypothetical (and idealized) version of AI fundamentally destroys core aspects of community problem solving and centralizes the existing knowledge into a black box subscription all for the benefit of a clunky UX and underlying product that has yet to be proven effective enough to justify all the negative externalities.

I actively hated interacting with the power users on SO, and I feel nothing about an LLM, so it's a definite improvement in QoL for me.

The "human touch" on StackOverflow?! I'll take the "robot touch," thanks very much.

The UX sounds better than Stack Overflow.

How is it much different than trading say a bar for livestream? For any org if you can remove the human meatware you should otherwise you are just making a bunch of busywork to exlude people from using your service.

Just through the act of existing meatware prevents other humans from joining. The reasons may be shallow or well thought out. 95+% of answers on stack overflow are written by men so for most women stack overflow is already a hellscape.

If companies did more work on bias (or at least not be so offensive to various identities) that benefit, of distributing knowledge/advice/RTFM, could be even greater.

Indeed, how documentation is written is key. But funny enough, I have been a strong advocate that documentation should always be written in Reference Docs style, and optionally with additional Scenario Docs.

The former is to be consumed by engineers (and now LLMs), while the later is to be consumed by humans.

Scenario Docs, or use case docs, are what millions of blog articles were made of in the early days, then we turned to Stack Overflow questions/answers, then companies started writing documentation in this format too. Lots of Quick Starts for X, Y, and Z scenarios using technology K. Some companies gave away completely on writing reference documentation, which would allow engineers to understand the fundamentals of technology K and then be able to apply to X, Y, and Z.

But now with LLMs, we can certainly go back to writing Reference docs only, and let LLMs do the extra work on Scenario based docs. Can they hallucinate still? Sure. But they will likely get most beyond-basic-maybe-not-too-advanced scenarios right in the first shot.

As for using LLMs to write docs: engineers should be reviewing that as much as they should be reviewing the code generated by AI.

    world where everything is perfect and made to be consumed by LLMs

I believe the parent poster was clearly and specifically talking about software documentation that was strong and LLM consumption-friendly, not "everything"

Yeah, old news? It's how it is today with humans.

You SHOULD be making things in a human/LLM-readable format nowadays anyway if you're in tech, it'll do you well with AIs resorting to citing what you write, and content aggregators - like search engines - giving it more preferential scores.

Then you're doing it wrong?

I'd need to see a few examples, but this is easily solved by giving the llm more context, any really. Give it the version number, give it a url to a doc. Better yet git clone the repo and tell it to reference the source.

Apologies for using you as an example, but this is a common theme on people who slam LLMs. They ask it a specific/complex question with little context and then complain when the answer is wrong.

Usually that's resolved by saying "I want you to use v2" or whatever it is, which you can't really do with a Stack Overflow answer as easily.

Have you tried using context7 or a similar MCP to have the agent automatically fetch up to date documentation?

> When a question gets closed before an answer comes in, the OP has nine days to fix it before it gets deleted automatically by the system.

One of the bigger problems with the site's moderation systems was that 1) this system was incredibly opaque and unintuitive to new users, 2) the reopen queue was almost useless, leading to a very small percentage of closed questions ever getting reopened, and 3) even if a question did get reopened, it would be buried thousands of posts down the front page and answerers would likely never see it.

There were many plans and proposals to overhaul this system -- better "on hold" UI that would walk users through the process of revising their question, and a revamp of the review queues aimed at making them effective at pushing content towards reopening. These efforts got as far as the "triage" queue, which did little to help new users without the several other review queues that were planned to be downstream of it but scrapped as SE abruptly stopped working on improvements to the site.

Management should have been aggressively chasing metrics like "percentage of closed questions that get reopened" and "number of new users whose first question is well-received and answered". But it wasn't a priority for them, and the outcome is unsurprising.

And that was the core problem with Stack Overflow - they wanted to build a system of core Q&As to be a reference, but everyone treated it as a "fix my very specific problem now".

99% of all the junk that got closed was just dumps of code and 'it doesn't work'. Not useful to anyone.

Look at the data - it had already been on the downslide for years before LLMs became a meaningful alternative. AI was the killing blow, but there was undoubtedly other factors.

You overvalue the impact of LLMs in regards to SO. They did have an impact, but it's the moderation that ultimately bent and broke the camel's back. An LLM may give seemingly good answers, but it always lacks in nuance and, most importantly, in being vetted by another person. It's the quality assurance that matters, and anyone with even a bit of technical skill quickly brushes up against that illusion of knowledge an LLM gives and will either try to figure it out on their own or seek out other sources to solve it if it matters. Reddit, for all its many problems, was often still easier to ask on and easier to get answers on without needing an intellectual charade and without some genius not reading the post, closing it and linking to a similar sounding title despite the content being very different. Which is the crux of the issue; you can't ask questions on SO. Or rather, you can't ask questions. No, no, that's not enough. You'll have to engage with the community, answer many other questions first, ensure that your account has enough "clout" to overturn stupid closures of questions, and when you have wasted enough time doing that, then you can finally ask your own question. Or you can just go somewhere else that isn't an intellectual charade and circle jerking and figure it out without wasting tons of time chasing clout and hoping a moderator won't just close the question as duplicate. SO was never the best platform, exactly because of its horrendous moderation. It was good, yes. It had the quality assurance, to a degree, yes. But when just asking a question becomes such a monumental task, people will go elsewhere, to better platforms. Which includes other forums, and, LLMs. So no, what you're attributing to LLMs is merely a symptom of the deeper issue.

It was bad enough that many people resorted to asking their questions in Discord instead which is a massive boomerang back to trying to get help in IRC and just praying that someone is online and willing to help you on the spot. Having to possibly ask your question multiple times before you get some spotty help in a real time chat where it's next to impossible to find again seems unimaginably worse than using an online forum but the fact of it remains and tells us there was something driving people away from sites like SO.

> Question can be marked as duplicate without an answer.

No, they literally cannot. The only valid targets for closure are existing questions that have an upvoted or accepted answer. The system will not permit the closure (or vote to close) otherwise.

If you mean "without writing a direct answer to the new question first", that is the exact point of the system. Literally all you have to do is click the link and read the existing answers.

> it feels insulting because the other is asked in such a weird way, that no-one will find the old when they search for the new

Sure. But someone else knew about the old question, found it for you, and directly pointed you at it so that you could get an answer immediately. And did all of this for free.

And, by doing this, now everyone else who thinks of your phrasing for the question, will be immediately able to find the old question, without even having to wait for someone to recognize the duplicate.

The tone of this answer explains everything why people fled SO as soon as they possibly could.

Thank you for being the voice of reason in this comment section!

Can you explain what you're saying in greater depth?

Are you saying that the reason there is no human expertise on the internet anymore is that everyone with knowledge is now under contract to train AIs?

Your first example is a public announcement of an llm assisted ask question form. A detailed request for feedback on an experiment isn't "sneaking" and the replies are a tire fire of stupidity. One of your top complaints about users in this thread is they ask the wrong sort of questions so AI review seems like it should be useful.

The top voted answer asks why SO is even trying to improve anything when there's a moderator strike on. What is this, the 1930s? It's a voluntary role, if you don't like it just don't do it.

The second top voted answer says "I was able to do a prompt injection and make it write me sql with an injection bug". So? It also complains that the llm might fix people's bad English, meaning they ask the wrong question, lol.

It seems clear these people started from a belief that ai is always bad, and worked backwards to invent reasons why this specific feature is bad.

It's crazy that you are defending this group all over this HN thread, telling people that toxicity isn't a problem. I've not seen such a bitchy passive aggressive thread in years. Those replies are embarrassing for the SO community, not AI.

I believe that this tension about what type of questions was baked into the very foundation of StackOverflow.

https://www.joelonsoftware.com/2008/09/15/stack-overflow-lau...

> What kind of questions are appropriate? Well, thanks to the tagging system, we can be rather broad with that. As long as questions are appropriately tagged, I think it’s okay to be off topic as long as what you’re asking about is of interest to people who make software. But it does have to be a question. Stack Overflow isn’t a good place for imponderables, or public service announcements, or vague complaints, or storytelling.

vs

https://blog.codinghorror.com/introducing-stackoverflow-com/

> Stackoverflow is sort of like the anti-experts-exchange (minus the nausea-inducing sleaze and quasi-legal search engine gaming) meets wikipedia meets programming reddit. It is by programmers, for programmers, with the ultimate intent of collectively increasing the sum total of good programming knowledge in the world. No matter what programming language you use, or what operating system you call home. Better programming is our goal.

(the emphasis on "good" is in the original)

And this can be seen in the revision history of https://stackoverflow.com/posts/1003841/revisions (take note of revision 1 and the moderation actions 2011)

---

Questions that are fun and slightly outside of the intended domain of the site are manageable ... if there is sufficient moderation to keep those types of questions from sucking up all available resources.

That was the first failing of NotProgrammingRelated.StackExchange ... later Programming.StackExchange ... later SoftwareEngineering.StackExchange.

The fun things, while they were fun took way more moderation resources than was available. People would ask a fun question, get a good bit of rep - but then not help in curating those questions. "What is your favorite book" would get countless answers... and then people would keep posting the same answers rather than reading all of them themselves and voting to cause the "good" content to bubble up to the top.

That's why TeX can have https://tex.stackexchange.com/questions/tagged/fun and MathOverflow can have https://mathoverflow.net/questions/tagged/soft-question and https://mathoverflow.net/questions/tagged/big-list -- there is a very high ratio for the active in moderation to active users.

Stack Overflow kind of had this at its start... but over time the "what is acceptable moderation" was curtailed more and more - especially in the face of more and more questions that should be closed.

While fun questions are fun... the "I have 30 minutes free before my next meeting want to help someone and see a good question" is something that became increasingly difficult. The "Keep all the questions" ideal made that harder and so fewer and fewer of the - lets call them "atwoodians" remained. From where I sit, that change in corporate policy was completely solidified when Jeff left.

As moderation and curation restricted (changing the close reasons to more and more specific things - "it's not on that list, so you can't close it") meant that the content that was not as well thought out but did match the rules became more and more prevalent and overwhelmed the ability for the "spolskyites" to close since so many of the atwoodians have left.

What remained where shells of rules that were the "truce" in the tension between the atwoodians and spolskyites and a few people trying to fight the oncoming tide of poorly asked questions with insufficient and neglected tooling.

As the tide of questions went out and corporate realized that there was necessary moderation that wasn't happening because of the higher standards from the earlier days they tried to make it easier. The golden hammer of duplication was a powerful one - though misused in many cases. The "this question closes now because its poorly asked and similar to that other canonical one that works through the issue" was far easier than "close as {something}" that requires another four people to take note of it before the question gets an answer from the Fastest Gun in the West. Later the number of people needed was changed from needing five people to three, but by then there was tide was in retreat.

Corporate, seeing things there were fewer questions being asked measured this as engagement - and has tried things to increase engagement rather than good questions. However, those "let's increase engagement" efforts were also done with even more of a moderation burden upon the community without the tooling to fix the problems or help the diminishing number of people who were participating in moderating and curating the content of the site.