Comment by Semkas
5 hours ago
So, leaving aside the idea that OA might've used data from the researchers Codex sessions: Do I understand correctly that the internal OpenAI work on the problems was probably started after they heard Alpoge and Buckmaster had made process by using their models? And they used the publicly available info about the researchers past work to prompt their models?
If compute is cheap, and the difficult thing with scientific discovery is now mostly in steering agents into promising areas, there's an obvious incentive for OA mathematicians to simply monitor closely which researchers are close to releasing exciting results, make some assumptions about their prompts based on their past work, and quickly prompt their own (stronger) model to look into the same areas.
> leaving aside the idea that OA might've used data from the researchers Codex sessions
Why leave that aside? That is _the_ story.
If a Chinese research lab did this we'd call it espionage.
But there's a bunch of people already in this thread calling that stuff unfounded speculation (which I disagree with), and my point is that even if that specific thing isn't true, OA's behavior here is obviously awful.
If they're going to try to beat researchers to discoveries like this it disincentives researchers to talk about their progress publicly, and basically breaks the ecosystem of scientific cooperation / discovery. It's also immoral.
Because they didn't do that. Tristan doesn't specifically claim that they did, and Anthropic employees don't think they did either. https://x.com/_sholtodouglas/status/2097218240397410733
Well, of course Anthropic employees would say that, since they likely do the same. Claiming that your primary competitor doesn't engage in a certain malicious practice is supposed to make it look as if there's no way you would too. If somebody even says that about their competitor, then surely there must be truth to that, otherwise you would never give credit to someone you're opposed to.
By default OA trains their models on codex-sessions. If I understand him correctly this is something Tristan explicitly mentions in his post as a possible reason for the fast results obtained by the internal OA team. Anthropic obviously doesn't want to challenge the idea that training is transformative, even if it means agreeing with their competitor.
Why is that the story? Is there anything to back it up beyond a single accusation?
As a prior I would say that a math professor has about infinite times more integrity than OpenAI.
I’ve thought a lot about publishing research and wanting to do more of it, but right as I finally had the time and energy to start writing articles LLMs start to take off. Now all of a sudden, I’m acutely aware that everything I publish will be used for AI training.
For math, a field that is built on incremental research it feels like AI labs will do nothing but discourage publishing research at all for fear that they will be able to spend the money for compute that publicly funded academia simply cannot afford.
It feels like publishing anything at this point just means that your work will be fed to a machine that will make sure your work will never been seen by anyone else because it will always be the ones making the “true advancements”.
Perhaps I’d feel better about this if AI labs really existed for humanity’s benefit, but for some reason I don’t think that comes up in their investor slide decks.
Similar to https://news.ycombinator.com/item?id=47566442
This is an excellent point...