Comment by doc_ick
20 hours ago
“Requesting feedback is useless, as the article points out”, I would agree post llm availability. Before that (~pre 2023 I suppose), this wouldn’t have been as large of a concern.
“You need to get the author to do an oral defense to verify that they wrote the paper.” While I agree with this point, it can’t scale. As with the indirect response, vishing and likely video mimicry are only going to be easier with the next llm release. This leaves requiring each author who submitted a paper to fly to a place just to defend a paper in person (assuming all volunteer reviewers are in the same place).
> “You need to get the author to do an oral defense to verify that they wrote the paper.” While I agree with this point, it can’t scale.
Are we prepared to sacrifice everything on the altar of scalability?
At any rate, why can't this scale? Go to the nearest accredited university, sit at a terminal they give you and do you interview.
Universities already have the space to do this, for potentially hundreds of defenders per day.
“Are we prepared to sacrifice everything on the altar of scalability?” If the performance of verification isn’t great, it won’t get adopted. Just general acceptance timelines, look at how long it took for Python 2->3, https 1.2, us real-ids. You can’t adopt a significant change immediately and expect things to work immediately.
“Universities already have the space to do this, for potentially hundreds of defenders per day.” Universities have space for their own students while they’re still funded for the time, but conferences aren’t tied to universities. Even so if they were, from the author “The fraction of desk rejected papers at TMLR used to be about 6% in 2023 but is now at about 53%.” universities would have to raise funds for this massive increase of slop review. Us conferences uses unpaid volunteers (most part) from those areas for reviews.
If an independent conference were to hold in-person interviews (and maybe fly people out) for 1000s of submissions, where would they hold it and how would they pay for it?