Comment by bluefirebrand
2 days ago
> Overly pessimistic, and doesn't acknowledge that heads of steam only build behind promising findings, while the deficient (or 'fraudulent') work die on the vine, published or not.
This might be true in hard sciences where a "head of steam" can only build based on real, replicatable results
But it's very common that public policy is proposed and adopted based on findings from soft sciences like psychology and sociology
If policy is adopted based on a research paper, I would count that as a "head of steam" being built.
And if that paper is fraudulent, then we are adopting well-intentioned policy on false pretenses
I did just mean AI and Computer Science per OP. By "head of steam" I mean to say that much research is built on it, think the likes of "Attention is All you Need". There isn't quite an equivalent of this in public policy in my experience.
Conversely, computer science/AI doesn't have an equivalent of the rigor that public policy research tends to go through. CS has e.g., benchmark datasets, typical evaluation metrics, but these are more like norms rather than requirements, whereas in public policy, instruments for validations are far more rigorously tested and enforced. Depending on the area.
I agree that outright fraud would be detrimental, but I think OP overblows this issue completely and should apologise to his co-authors.