Comment by mlmonkey
19 hours ago
Another data point: look at the number of submissions to Neurips, ICLR, etc. Now, I agree that a submission does not by itself imply groundbreaking resesarch, but the fact that people are able to try out/vet ideas much faster now makes a difference.
In my own work, I've found that it's much faster to brainstorm with a frontier model like Opus. I'll have an idea and be able to get a working implementation and try it out on toy problems within O(minutes), instead of O(weeks) earlier.
Trying out problems and making tangible progress towards AGI are entirely different categories. Likewise what’s changed in terms of paper submissions is also the amount of slop. Most reviewers say that they haven’t seen a lot more papers that are good but rather they’ve seen a lot more papers and almost all of them are terrible slop that’s desk rejected. This is happening in OSS too where a lot of projects are closing the gate at the onslaught of terrible contributions.
> Trying out problems and making tangible progress towards AGI are entirely different categories.
They seem very strongly related to me.
I agree in general that AI accelerates AI research, but good lord neurips iclr papers (and reviews might I add...) are the worst example of that. It's all complete slop that is totally meaningless.
A lot of great papers by great people had a lot of AI involved and accelerating it of course, but the majority is meaningless (sometimes autoresearch) slop.
Specifically talking about recent neurips/iclr.
In October/September my LinkedIn wall was probably… 55% people complaining that they were getting garbage AI submissions to the conferences they review for, and 35% people complaining that they got garbage AI reviews back from their conference submission… Of course, I have no idea if that means anything given how aggressively that site bubbles your feed.