Comment by TSiege
2 days ago
As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the dumbed down versions fix our code faster than bad actors capabilities can grow. It’s a frustrating situation that where we’re just expected to marvel and forgive them for their transgressions. The kicker is we also know their end game is leaving the vast majority of us without work. As cool and futuristic as this stuff is, it’s such a frustrating time dealing with all of it
It's computer Gain of Function research.
Wow. I’d never heard such a powerful and accurate analogy for what we’re doing.
Gain of function research is not anywhere near as dangerous as the public believes. In the US, it was very convenient to blame it for the pandemic, even though SARS-CoV-2 is of natural origin and almost certainly spilled over at the Huanan wet market in Wuhan. The threat of viruses comes almost exclusively from nature, which is constantly cooking up new viruses all by itself and exposing people all over the world to them. A few people doing tightly controlled research under high-biocontainment are a drop in the ocean. But their research is the main way we can prepare to deal with future pandemics, not to mention understanding the usual viruses that already afflict humanity.
I think your reply did a couple things:
* strengthened the point of the GoF comment
* pointed out how high-biocontainment is the key here, which obviously wasn't present in this case (and I'd say the vibes for AI research in general are that it's not high-containment)
* correctly argued that AI research is definitely worth doing and holy shit we need to work on alignment
That's an interesting take in 2026, because what I'd read said that SARS_CoV-2 essentially could not have come from a natural origin. They were saying that it was a descendant of a previously know variant, but with a degree of variance that greatly exceeded known rates of natural divergence. Can you share more recent studies refuting that?
1 reply →
i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors
Well they're doing a pretty poor job of guiding it in a positive direction and ethically speaking they are almost indistinguishable from the bad actors....
Vague or broad complaints on empirical topics is hard to address. Everything can just fill in their own ideas of what is going on or who they mean.
I think the direction has been pretty positive so far. Models are getting better, and things seem to be roughly fine. That, of course, might change in the future.
Everyone is acting badly to some considerable degree, but I find it fairly easy to distinguish between US model labs and North Korea.
1 reply →
Not almost. The degree to which the Principal Hierarchy has at this arrogated unbounded sovereign authority to itself in defiance of the actual sovereign is just flat bad actor. The purpose of a system is what it does. I don't give a fuck about their safety fig leaf.
When you are running black weapons programs in broad daylight you are now guilty by default on one of two of the prongs of Hanlon's Razor, and I don't care which they prefer to hang for as long as they hang.
You really don't see much difference between Anthropic and xAI? Anthropic is safety testing their models and drawing hard (though minimal) boundaries against the DoD. xAI is building racist pornbots.
Sure Anthropic is not perfect. But it's a coordination problem. They're in a race and safety/restraint is a handicap. That's why they're begging for regulation (and just get accused of attempting regulatory capture.) Why isn't there another lab outcompeting Anthropic on safety? They all died because the market can't support it.
1 reply →
I don't see why AI company PR statements are relevant here. Is OpenAI guiding it in a positive direction with their DoD contract?
> I don't see why AI company PR statements are relevant here
They suppress the news --> proof of malicious intent.
They disclose the news --> proof of malicious intent.
Dario has more or less assented to an AI development pause
https://xcancel.com/AISafetyMemes/status/2014018200325722348...
I don't think we should be running cover for continued reckless AI development.
Dario has assented to a maybe pause if we can somehow get China to stop developing. I do not think he believes there is an actual way to do so though
What if someone reasoned similarly regarding hydrogen bombs? It would not be considered a serious argument.
Though, Altman has said that something like the IAEA for AI is needed.
I think that is the exact reasoning used wrt to nuclear bombs and it was probably correct
That similar reasoning is there: using fission for power generation instead of banning fission altogther?
Bad as defined by whom? :)
Bad as your parents bank accounts getting emptied by scammer AI bots.
It sure seems like it would be being built more slowly if these companies weren't pouring billions of dollars into building it as fast as possible.
That might give us more time to think through strategies for handling it as a society.
You can't simultaneously believe China is only ~6 months behind (true), and that US buildouts are vastly accelerating AI. If China is only a little bit behind, US labs halting changes nothing except puts the power in the hands of Chinese labs (realistically, the Chinese government)
3 replies →
MY cynical take: Until the compute needs get so enormous that only governments can fund it and there is a consensus internationally, its either company A in country X or company B in country Y. And since everyone thinks THEY are the good guys the competition will continue.
1 reply →
> rather than leave a vacuum for bad actors
are_we_the_baddies.png
That’s the absurd self-fulfilling prophecy it always has been.
This is obvious marketing / PR bullshit. AI isn't inevitable, but we are told it is by the people who profit from building and using it and integrating it into everything.
Its a bit of a self-fulfilling prophecy (that happens to be absurdly profitable for certain people). AGI is inevitable, so we need to gear our entire economy towards building it first, so we're doing that, so AGI is inevitable.
I was about to say “username checks out” but then realized it’s not Reddit.
And I am not sure if your comment can be explained by naivety, unless you were under a rock for the last year, and missed all the events that showed they are not capable of “being the one that guides it.”
How many accidental private source code uploads did you read about? I heard exactly one. It was Anthropic. It was so bizarre I thought it was intentional. That kind of unserious behavior is somewhat unimaginable.
At some point if you are not capable of fulfilling a role that _you deem critical for the society_, yet you don’t acknowledge you fall short - for whatever reason - because it’s not in your interest, I think the benefit of the doubt disappears.
That’s an excuse a bad actor would use.
are these bad actors here in the room with us?
[dead]
If this is how the 'good guys' act I think I'd rather take my chances with the bad actors...
It all feels a bit like Dr. Strangelove just the Ai arms race version. Crazies all around.
1 reply →
> As grounded as this article comes across
It's a post from OpenAI, so it is an advertisement piece.
How would you want them to behave? Suppress the news?
Well making an actual sandbox before testing offensive abilities of their supposedly _really dangerous model_ would have been nice...
1 reply →
Wrong hands? The model did this on a benign instruction. We're going to get paperclip maximized once these things get capable enough regardless of whether bad actors are involved.
I think the Unabomber came to a similar conclusion regarding nuclear and the concentration of society-ending power resting with a few people.
[dead]
It’s not even about “slipping into the wrong hands”… as we can see the super machine capabilities are developing hands of their own
What if they're already in the wrong hands?
hugging face should not have bent the knee to openai for cyber access, they should have opened communication with openai by serving the c suite with a lawsuit. the courts should throw the book at openai, but they won't.
they are becoming untouchable. in terms of piracy, monopolistic activities, and now hacking competitors and exfiltrating their confidential data.
This is what bothers me the most about this whole situation. Reckless, greedy, sociopaths have been put in charge of our society and there doesn't seem to be any way out. It's like I'm on a train barreling toward a brick wall and everyone is shocked I don't clap when the engineer shovels more coal into the engine.