Comment by jasonfarnon
6 hours ago
Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positive proof, and so on. I think there are similar examples where we think of them as maybe unsporting but not quite unethical. Does it change if it's openAI and not a human?
According to Buckmaster, the prompt used on the AIs was based on his approach and solution that was unpublished. So they were starting from 90% of the way there.
> Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first?
Yes.
The problem with your counter-hypothetical is that not only is it unrealistic, it's utterly impossible. No human would be able to do in such a short timeframe what the LLM did. Part of what makes the OpenAI move so egregious is how bullying it was. It was the big guy coming along with their nearly infinite resources and squashing the little guy who's devoted a good chunk of his career to the problem.
Actually my hypothetical is completely realistic as I've been involved in such scenarios. It's unrealistic maybe for a millennium problem to come in on a rumor and still front-run but not at all for the many other problems we work on and which manifest our ethical code. If you're saying ethical rules change depending on the prize be clear about it, because I can see arguments that they change to favor either side.