Comment by ToValueFunfetti
11 hours ago
I don't think the timelines make sense for it to have been included in training data, LLM training does not seem to be nearly efficient enough to learn these things from lone examples, and it wouldn't make sense for this earlier work to have made its way into both OpenAI and Gemini's data. Navier-Stokes is not strong evidence that this kind of thing can happen, as OpenAI denies that Buckmaster's work was in the training data for the model they used, Buckmaster didn't have a blowup that they could steal, and Scientific American reports[1] that the similarities between their approaches were superficial. It's also well-established that models can find novel vulns at this point.
[1]https://www.scientificamerican.com/article/openais-blockbust...
My understanding at the time I wrote that was “someone else using chat-logged OAI model: September???; OAI using unspecified model: October???”, and when I say “Navier-Stokes-style-scoop” I am thinking along the same lines as the comment I made during that controversy, here: https://news.ycombinator.com/item?id=49639648. I don’t buy that their public statements are not “technically correct but misleading without internal context” on these matters.
Naturally, with the added info from the PHP foundation link (specifically that OAI reported it the same day as Calif, and that Calif was operating under the auspices of an OAI program) the timeline collapses for my initial impression!
This is the new, "they stole the internet" ostrich belief