Comment by hexomancer

> (1) We'd have to identify their chats. How would we do this? We'd need them to share their chats with us so we could look for matches.

According to the statement by Tristan Buckmaster, he was in communication by email and calls several times over the past week with you (OpenAI that is, not you personally), asked about whether his chats were trained on, and was declined an answer (https://cims.nyu.edu/~tristanb/statement.pdf).

However, it seems like there was great pressure to hurry the release to compete with Anthropic's recent release, so he was unable to get an answer in time.

The mealy mouthed statement in the release "We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ." is realy not much. If OpenAI had wanted to be transparent about this, you could have worked with him to identify if his data was used in the training of your new model, and actually made a somewhat more certain statement on that basis. But you have chosen not to; it was more important to scoop Anthropic on this than it was to be transparent about your training data.

> (2) We'd have to prove those chats changed model behavior. How would we do this? We'd need to retrain many models with those specific chats removed, and ask those models to solve the Navier-Stokes problem many times, and keep doing this until reaching the desired level of statistical significance.

Just the information from step (1) would improve transparency. Yes, you still can't prove one way or another how much the effect of the training is. But if it's included in the training data, it provided some effect.

> we'd have to prove that firing the gun caused the murder. how would we do this? we'd need to redo the murder many times, with and without my client firing his pistol. that's extremely expensive and not really feasible. therefore, we must acquit.

#2 (prove those chats changed model behavior) is pretty straightforward if the anonymized data from chats can be actively searched by a model. In fact, it could be very clear if the provenance of context is traced. If anonymized data from chats leak into the context of an actively running model it would clearly influence the answer.

Sure. But it's possible to say: if the document isn't in the training data, it isn't the cause of the output. If it is in the training data, the question gets more complicated.