it's like if you vibe coded something and the onus is now on the reviewer and the reviewer tells you your work contains bugs and is messy - that is not acceptable from the reviewers pov - why should the reviewer spend all his human effort, a scarce resource, reviewing your code while you've spent barely a fraction of his effort generating this. OpenAI is a trillion dollar company, surely they can verify stuff before pushing it out? The problem is not that they are solving the problems, they don't care at all about the actual process of doing mathematics. Using compute to mine problems and throwing results in github and letting human reviewers spend effort to correct these is not going to win them any favor. If OpenAI really cared about math, they would have someone on their side who actually understood the results they produced and was able to verify their correctness and educate others.
No, it's not how math works at all. Math is not experimental science. Results take years to formulate and are generally known to be true by colleagues via correspondence and seminars before a paper is even written. Retraction is very rare.
I don't know why people are talking about something like this that they are clearly not familiar with.
Withdrawing it isn't irresponsible. I don't even think releasing it is, because that is often the best way to find problems with it; expose it to the wider research community. Doing PR victory laps based on unverified, unfinished work, on the other hand, is. It is well known, even scientifically demonstrated, that retractions or refutations get much less attention than the initial exuberant PR announcement. And thus this practice contributes to misinformation.
And just to preempt the kneejerk whataboutism: Yes, all of academia does this to varying extents, and the mainstream media are also complicit. And that is also irresponsible. And no, that is not an excuse for OpenAI. Especially when you consider that OpenAI actively portray themselves as some kind of moral arbiter on AI and "doing good for humanity". They should be held to the extraordinarily high moral standards they purport to hold themselves to.
it's like if you vibe coded something and the onus is now on the reviewer and the reviewer tells you your work contains bugs and is messy - that is not acceptable from the reviewers pov - why should the reviewer spend all his human effort, a scarce resource, reviewing your code while you've spent barely a fraction of his effort generating this. OpenAI is a trillion dollar company, surely they can verify stuff before pushing it out? The problem is not that they are solving the problems, they don't care at all about the actual process of doing mathematics. Using compute to mine problems and throwing results in github and letting human reviewers spend effort to correct these is not going to win them any favor. If OpenAI really cared about math, they would have someone on their side who actually understood the results they produced and was able to verify their correctness and educate others.
They only withdrew it after someone pointed out that it was a flawed paper.
In one case by asking Astra to review it.
Not exactly encouraging that they did their homework before publishing results.
This is how science is meant to work.
No, it's not how math works at all. Math is not experimental science. Results take years to formulate and are generally known to be true by colleagues via correspondence and seminars before a paper is even written. Retraction is very rare.
I don't know why people are talking about something like this that they are clearly not familiar with.
Most good scientists work hard to prove results to themselves before going public.
Peer review is then done _in private_ before publication as a check on quality and significance.
Retracting a paper is pretty embarrassing. And not considered science as usual.
3 replies →
Yes. A researcher who achieves a reputation of making mistakes carries that reputation.
No, scientists check their work before releasing it. Slop is still slop if you wrap it up as an academic result.
Withdrawing it, in itself, might not be, but it kind of skips over the part where they have a paper that warrants it in the first place.
Withdrawing it isn't irresponsible. I don't even think releasing it is, because that is often the best way to find problems with it; expose it to the wider research community. Doing PR victory laps based on unverified, unfinished work, on the other hand, is. It is well known, even scientifically demonstrated, that retractions or refutations get much less attention than the initial exuberant PR announcement. And thus this practice contributes to misinformation.
And just to preempt the kneejerk whataboutism: Yes, all of academia does this to varying extents, and the mainstream media are also complicit. And that is also irresponsible. And no, that is not an excuse for OpenAI. Especially when you consider that OpenAI actively portray themselves as some kind of moral arbiter on AI and "doing good for humanity". They should be held to the extraordinarily high moral standards they purport to hold themselves to.