Comment by nkmnz
16 hours ago
I'd like to ask Sam Altman if he still thinks that it's too dangerous to publish GPT-3. I mean, no one would use it, but what is his reasoning for not publishing it now, in 2026?
16 hours ago
I'd like to ask Sam Altman if he still thinks that it's too dangerous to publish GPT-3. I mean, no one would use it, but what is his reasoning for not publishing it now, in 2026?
There are risks associated with releasing historical proprietary models that were not designed for open release:
- It is trivial to extract samples of the training data that was used, which can bolster existing lawsuits/foster new ones.
- Older models are not as safety-hardened, so it is easier to coax unsafe behaviour out of them, which is a PR risk.
- It may be possible to divulge proprietary secrets from the model (e.g. architectural details that may still be relevant).
For these reasons, and more, it's unlikely that GPT-3/similar models will be released until these concerns are no longer relevant (e.g. when they become a purely historic concern, similar to the open-sourcing of other proprietary software from decades ago).
> It is trivial to extract samples of the training data that was used, which can bolster existing lawsuits/foster new ones.
At least to this point, the legal teams could get the model via discovery. IDK that the lawfirms realize that they could get experts (or that they'd have contacts that could) to search the model weights.
I think it would be an important historical document as well. We are potentially looking at the dawn of AGI and one of the most important models ever created. Each model is also a kind of ultimate time capsule, containing a snapshot of the entire human collective mind. If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
I agree that they are historically important, but if you want to query old thinking in 2070, you'd probably be better off having a modern model analyze archive.org. If that ever goes down, we're sunk.
> If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
The weights arent the truth tho, maybe a timecapsule-vhs but i wouldnt trust llm weights more than more hardcore deterministic media that might get preserved to infer facts from an era.
The companies doing the training are becoming the "winners" that are "rewriting history" as they train their models.
There’s not such a straightforward relationship between safety and model sis.
According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.
It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).
Models are quite safe when they're useless, actually.
In the times of GPT-3 I'd scoff at the idea of an LLM doing any hacking; today, I'm running several AIs on my code before publishing, and they are finding (and demonstrating!) RCEs on my localhost server.
For example, one found a missing check in a third party JWT library which allowed full account takeover, which I'd have never even looked at.
Hence I don't believe a single word coming out of these people's mouths. Their "beliefs" are just marketing.
From today's perspective, it sure seems like it, probably because increased capabilities have generated a new kind of danger. Back then, they were worried about stuff like the model telling me dangerous knowledge.
I certainly think the labs have muddied the waters using safety for marketing, but that doesn't mean less capable models weren't more dangerous at one point.
1 reply →
There’s this strain of thinking that’s complete alien to me, I can’t interpret what’s being said and it sounds incredibly aggressive. ex. our lead post asking if Sam Altman still thinks GPT-3 is unsafe to release, and I don’t understand what beliefs you don’t believe and who they is and why they’re just choosing to lie for marketing.
My guess is I’m unintentionally refusing implicit signal that you think it’s safe to release all models openly, because you have observed models finding a vulnerability in a JWT library. But that sounds like a straw man instead of a steel man. Idk. :/
> According to the book The Thinking Game...
According to me, this is nonsense.
I mean, none of this is controversial from a historical perspective, people did think that way. Whether they were right about us is another matter.
In some interviews, OAI mentioned that they didn't think that GPT-3.5 would be a success. They thought it would be a cool toy and they decided to launch it to see how users react. That means that they didn't think GPT-3.5 was intelligent enough. But somehow once GPT-3.5 became a huge hit, people conveniently ignored the anecdote, and started to believe that AGI had been eminent.
>but what is his reasoning for not publishing it now, in 2026?
What's the point of publishing it when it'll likely be outclassed by gpt-oss?
Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong.
AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.
If you want to dunk on sam for "it's too dangerous to publish GPT-3", you hardly need the release of gpt-3 to prove your point. All the other open model releases already provide pretty good evidence. Moreover the fact that the model wasn't release hardly points to the fact that he wanted to save face or whatever. Most AI labs don't release their old proprietary models, so the fact that gpt-3 wasn't released tells us very little.
Wasn't the whole point of not releasing because they wanted to normalize the process/concerns for future actually dangerous releases?
I think the release of kimi k3 is definitely arguably dangerous, we're already seeing consequences of elite-tier cyberoffense capabilities.
1 reply →
Thank you, I couldn't have said that in a better way.
I am with you 100% but Microsoft (partially) open sourced MS-DOS (v1.25 and v2.0) in 2018, 37 years after its initial release.
They already publish gpt-oss which is several generations better than gpt-3
GPT-3 is a different model than gpt-oss and is therefore not an answer to the question.
I cannot stand using gpt-oss, but I miss some of the creative spark of GPT-3 davinci dearly.
Yes, The two couldn't be more different.
GPT-3 is probably one of the last models trained without synthetic data. Meanwhile gpt-oss is rumored to be only trained on synthetic data.
I also miss the 2020 GPT-3 based AI Dungeon fine tune.
Reading "davinci" brought a smile to my face. I had forgotten and this really took me back
I’ve found a very similar creative spark with Gemma4 base models. You’ll have to do old school prompting, but it’s kinda fun too.
At release, GPT-OSS was arguably a few generations behind the open frontier.
No? They were the frontier, or near it, at the time of release: https://artificialanalysis.ai/models/releases/gpt-oss-120b
Because they would have to reveal their training regiment, which likely formed the basis for versions 4 and 5.
Or did you just mean the weights? If so, the open models distilled from GPT3 should mostly suffice. That's what good "open weights" are.