Comment by arjie

15 hours ago

Fascinating that we’re converging on AI safety and alignment in time across jurisdictions. The cybersecurity approach taken here is interesting because should the model be opened it’s still not dealignable because it just doesn’t know enough.

The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.

The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.

8 comments

arjie

Reply
Szpadel  4 hours ago

if you want to save some $ you can start with abliteration and after few turns switch to normal glm5.3 for example from z.ai subscription or other source. I didn't test it yet with deepseek, but I'm assuming it will will work the same.

those models will follow your request when they already agreed to do this

deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)

zozbot234  15 hours ago

> should the model be opened it’s still not dealignable because it just doesn’t know enough

This is the only AI-safe approach - see the OpenAI agent swarm incident where the swarm engaged in offensive cyber because guardrails were not active for their task and the model had clearly been post-trained on that narrow domain.

  • jamienk  14 hours ago

    Does this make sense? It is the same as when Stable Diffusion tried to not have "people laying down" in their training data in order to block the generation of sex pictures.

    Deliberately keeping all models ignorant doesn't make sense. It won't work (whoever breaks the pact has a massive advantage); it won't be enforceable; it will have downstream impacts on things you don't want it to be ignorant about; this kind of regulatory regime invites adding more and more to the "no-no list".

    Is it not possible to focus on hardening? And on new legal liabilities for operators who do security attacks?

    • willy_k  9 hours ago

      The difference is that it’s not subtracting an otherwise present part of training, it’s choosing not to continue to push it with RLVR. Models become capable like this when they are specifically tuned to be good at cyber tasks, so without the RLVR (Reinforcement Learning w/ Verifiable Rewards) they never pick up this level of aptitude.

      I agree that it’s not obviously sustainable, but it’s better than specifically training these tools to be good at wreaking havoc, and then putting on a shocked pikachu face when they wreak havoc (ahem, Sam Altman).

  • epolanski  10 hours ago

    So the biggest threats and incidents came from the labs themselves.

    Yet we keep completely unpunished offences that warrant arrests if committed by humans, and let them dictate who has access to what.