Comment by cortesoft

Probably not. "This model is too powerful for the public" can also be interpreted another way, which they've also strongly hinted at - the cost/benefit ratio of the upgrade is negative for the vast majority of all users. Finding vulnerabilities is one of the few cases where it makes sense to use it.

Their writing about the model so far does say this is an issue where, for instance, you can't really use Mythos for interactive coding because it's so slow. You have to give it some work, go home, sleep, come in the next day and then maybe it'll have something for you.

All the AI labs and startups are still losing money hand over fist. Launching Mythos would require it to be priced well above current models, for a much slower product. Would the majority of customers notice the difference in intelligence given the tasks they're setting? If the answer is no, it's not economic to launch.

Really, I'm surprised they've done Mythos. Maybe they just wanted to exploit access to larger contiguous training datacenters than OpenAI, but what these labs need isn't smarter models, it's smaller and cheaper models that users will accept as good enough substitutes (or more advanced model routing, dynamic thinking, etc).

This happened before with GPT-2 being touted as "too dangerous to release"[0] at the time by OpenAI. I don't think that means every model will be safe to release in the future, but nothing I've read about Mythos seems like it's going to be different this time.

[0]: https://openai.com/index/better-language-models/

Their main motivation of the model being too dangerous is predicated on their discoveries in its ability to find exploits in commonly used software. The idea is that if this were served on a public API, it would massively increase the scale and scope of what malicious actors could do.

I think it's a reasonable choice to make given that Mythos actually does have cyber capabilities on that level. We already have evidence that large-scale scams are being perpetuated using AI models (such as AI video being passed as real, people deepfaking themselves in job interviews).

If you've noticed your new model can be trivially pointed at some open-source codebase with a prompt and harness that amounts to "find as many exploits as possible" and your results are non-trivially substantial and beyond what existing models can do given the same initial parameters, then a gated rollout seems the most reasonable option.

I feel like "this model is too powerful for the general public" was really just the equivalent of responsible disclosure, with the "too powerful" bit just a positive marketing spin like you say.

That is, Mythos will make it much easier to find lurking zero days, so just like responsible disclosure requires a security researcher to notify the software author first and give them some time to patch, giving critical infrastructure folks at least some time to analyze and patch systems seems reasonable to me.

Yup, this whole thing is quite typical for my generations attempts at activism: they always end up as marketing pawns for the very thing they set out to stop.

This whole "this model is too dangerous" ploy originated from (in my opinion severely misguided) activists who wanted to stop or slow AI development down as much as possible, spreading outlandish Doomsday scenarios wherever they could.

These online-first activists have always been a key driver of the success of the very thing they fight. They share the offending thing among themselves, making it go viral in process, and soon baiting these groups is the best marketing imaginable.

There were some rather interesting studies made on the subject around 2011, I particularly remember one made by Swedish jeans brand cheap Monday, but i can't find it now.

> This feels really premature. The announcement was a week ago. The “this model is too powerful for the general public” sounds like marketing to me.

Anthropic was born out of the idea that they feel paternity over humanity. They believe by limiting access they are performing a necessary pillar of security in multiple facets.

I think it's up to the public, and articles like this are part of the public's voice, whether this belief is serious or not and secondarily whether it's okay to even posture this kind of belief since it inherently results in marginalizing the many and rewards an already very successful few.

For one I fully agree with your statement.

For me, the seeming majority optimism and acceptance of “mythos’” as yet untold capabilities is betrayed as not real by the fact that one can’t react to it with the same reverence while framing it as a downside without being told “it’s not even out yet”.

“It’s not even out yet” should apply to both situations or neither.

It’s not premature. The consequences of the decision making framework used are clear, and the second and third order consequences can be extrapolated

> The “this model is too powerful for the general public” sounds like marketing to me

I tend to agree here. Anthropic has built a reputation and now they are in a position where they can claim to have a model way more powerful than it might actually be, and by limiting its access, there won't be an independent way to test it. I'm not denying that it's not smarter than Opus, but probably it's somewhat exaggerated.

Security researchers always having a model one generation newer than the general public would still achieve the stated goals.

There’s a drain clog clearer sold in a jug like all the rest. But they wrap the jug in a thick clear bag. The implication is clear - this stuff is so powerful it’s extra dangerous.

It’s the same stuff inside as all the others.

Anthropic marketing is working very well. They are strongly incentivized to say their model is too powerful to release even if it’s not. It’s almost standard practice these days.

>The “this model is too powerful for the general public” sounds like marketing to me.

Is no one else suspicious that they literally called it mythos?

They’ve been shilling the same statement since GPT-2

[flagged]