Comment by JimDabell
1 year ago
You don’t really need a state-of-the-art anything here. People get too distracted with building the perfect system when it comes to use cases like this because they are paralysed thinking about the avoidance of false positives and make a bunch of sub-optimal decisions on that basis. False positives are much less of a problem with a human in the loop, and putting a human in the loop doesn’t require moderator effort.
You can probably put a big dent in the number of low-quality comments by just showing a “hey, are you really sure you want to post this?” confirmation prompt and display the site guidelines when you detect a low-quality comment. That way you can have a much more relaxed threshold and stop worrying about false positives. Sure, some people will ignore the gentle reminder, but then you can be more decisive with flags and followup behaviour because anything low quality that has been posted will by definition already have had one warning.
You're right about one thing: I didn't need to say "state of the art". A system that works at all would be great!
I don't think a confirmation prompt will help because people tune such things out after they've seen them a few times.
Even a bad implementation isn’t going to be showing this warning to people often enough to desensitise them. And if they make a habit of ignoring the warning to post flamewar stuff… that’s solved with moderation by a human. The intent is to add friction for knee-jerk low-quality comments, not solve for people who persistently, intentionally post low-quality comments after a warning.
I hate myself for saying it, because of all of the buzz/hype, but LLMs can assist here.
You get better intent assessment than with NLP/ regex/whatever.
Plus HN is entirely in English, so you never have to worry about lexical resource gaps.
There is no off the shelf solution - afaik. In addition I have no idea how expensive running costs will be.
But something serviceable can be built.
Source: mod /t&s person dealing with these things