Comment by whimsicalism
19 hours ago
I'm confused, your employer very directly stated that they are unable to confirm that the model was not trained on the conversations.
19 hours ago
I'm confused, your employer very directly stated that they are unable to confirm that the model was not trained on the conversations.
The models are trained on the conversations of hundreds of millions of people. ChatGPT has several billion conversations every day. I estimate that the model that solved Navier–Stokes was trained on data from nearly a trillion conversations.
It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes.
Do you not log the training data? Seems like you should be able to just check what was in the training data. To not keep track is just sloppy work and certainly unprofessional science.
That training data does not preserve provenance seems a "smoking gun" in terms of intent to plagiarize.
The burden of proof is on Buckmaster and Alpöge to reveal if they had the "Improve the model for everyone" setting enabled or disabled. OpenAI shouldn't be expected to reveal private user configuration data. You're asking them to perform a user privacy violation.
5 replies →
Does Anthropic, Grok, etc. log their training data? I had the impression it was rather a mess.
> ChatGPT has several billion conversations every day. I estimate that the model that solved Navier–Stokes was trained on data from nearly a trillion conversations.
How many of those trillion conversations were about Navier-Stokes you reckon?
I agree that it is not possible to prove if any one specific conversation (or derived RL tasks) was key to solving Navier-Stokes (at least without massive resource expenditure).
I don't really understand how the quantity of training data/rollouts used in training is relevant to the question of whether or not it was trained on these conversations.
I also don't really believe that whether or not this model was trained on these conversations is unknowable information.
[dead]
>It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes.
If the conversation was in the training set, there's a high likelihood that the small set of conversations related to solving Navier-Stokes was used by the model. I get Astra to still quote some of my friends' books or blogposts nearly verbatim on certain niche issues.
Much more importantly, we _can_ determine whether a conversation was used in the training data. And if it was, it gives us a great idea whether that logic was captured in reasoning for a novel problem never yet solved.
Given that you don't see any of this as below the belt according to your other comments, maybe your contribution here is more for yourself than a fair conversation about attribution.
The flaw with this line of reasoning is that Buckmaster and Alpöge only had a partially completed proof of a weaker version of the Navier-Stokes problem. OpenAI's internal model solved the full, harder problem. This means the key information needed to bridge the gap was not present in Buckmaster and Alpöge's chat history.
You might retort that ChatGPT used the training data to copy their approach, but the approach Buckmaster and Alpöge chose was already published by Luis and Diego in 2023 and in every frontier model's training set.
8 replies →
Sorry but this is a misconception: these models are both capable of complete novelty and of plagiarism. For a concrete example, image diffusion models have been shown to reproduce many existing images nearly 100% exactly, yet clearly, they can also create new ones.
A model being trained on lots of irrelevant information does not mean relevant information was not used.
The IP laundering machine strikes again.