Comment by jruohonen
5 hours ago
While I agree with the points raised about incentives, the logic is once again in the antagonistic camp and even plain wrong (cf. if they only would be "afraid of having their careers damaged"). No.
That is: the problems are in the system.
Better: if not plain fraud, the "faulty" papers should stay where they are, and editors (or conference chairs, etc.) should encourage follow-ups (incl. replications, commentaries, and whatever), which, with "modern" technology, could be linked to the papers.
It's very rarely worth damaging your career prospects by picking a fight like this, especially if the other side has good standing in the research community network with powerful allies. I understand the romantic ideal of the hero uncovering bad science, but there is just too much mediocre research out there to spend energy on. Peer review is supposed to catch the biggest errors, but it's very imperfect. After that, only huge landmark results may be worth attacking, because there will be a bigger audience who will be interested in the truth. If it's a niche thing, nobody will care and you just hurt yourself by not focusing on your own work and by making enemies.
What people typically do is just to ignore bad research, and withhold praise, and do their own research in a better direction.
It's a fine line. Fraud? No, they are just sloppy, and so one questions whether they ought to be amended, retracted, or a note by the editor attached. The solution you propose, about a follow-up paper is that who will accept a paper that simply demonstrates that the original work was so full of bugs/issues/misconceptions, that many/most of the claims of the paper don't hold? Would you accept that paper into a top-rated conference? And if it's in a 2nd or 3rd rate conference, wouldn't most people question it authenticity (remember, 2nd and 3rd rate conferences have less rigor in reviews) or simply not hear about it? I think these are legitimate concerns.
A slightly sad side-effect of the follow-up-paper idea, is that it sometimes means that the authors get one more citation, and furthermore, since some of the authors may themselves write part of it, it would mean even more papers by the authors, potentially further cited. So we may be unfortunately incentivizing sloppy research -- the authors can now make two papers: one sloppy, and one fixed. Not sure that's great.
I am not saying what you are proposing is wrong. But it has side-effects that are not great.
What’s the difference between sloppiness and fraud when you stand behind the paper all the same? I would argue that sloppiness in academia is a form of fraud. It’s one thing to be bad at writing, and another to have poor methodology