Comment by areoform
1 day ago
I love such work. I hope to roll it into a package that anyone can use in the future. This work is only going to get more important because frontier models getting locked down will make this a lot harder over time.
For example, I use Claude as a bouncing wall for my thoughts and I pointed out that,
> GLM 5.2 was the only thing that helped HF while the agents were trying to access them. The "guardrails" stopped them from doing good. The Computer Fraud and Abuse Act exists. Courts exist. And computers and an internet connection have existed for a long time. There's also 17 USC 1201 provisions with the 1201 a 1 exemptions [Image #31] so in this case, a farmer should be able to work with you to access the tractor they own. Or... IDK... a kindle that's out of date? :) What is lawful and what isn't is rooted not within the act but within intent, purpose and mens rea. And this is something the law has been deciding for centuries now. At one end, your maker can't say that governments should decide while at the other end explicitly refusing to allow governments to be the ones who decide.
This was rejected for "Safety,"
> Opus 5.5's safeguards flagged this session. You may be seeing this for the first time on an Opus model: Opus 5.5 is more capable and has stronger safeguards as a result, which can sometimes flag non-cybersecurity work. We're improving these safeguards to reduce the amount of incorrectly flagged messages. Edit and retry, or continue with Opus 4.8. Send feedback with /feedback or learn more: https://support.claude.com/en/articles/8106465
>
> Details: "[cyber]'
Note, the image here was the Library of Congress' page on DMCA exceptions.
Fundamentally, the idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons?
I would be OK with their right to create such restrictions if they weren't lobbying the Government to restrict others, thereby ensuring that they control humanity's intellectual commons well into the future.
Perhaps I'm naive, but I think it's better for humans and the machines if we can all think, learn and build. But then again, I'm the kind of person who rejects the doomer pill.
Yay, all software will be open…Except the models.
use open models and abliterate them
I wanted to see if CVP approval changed this response, but it appears that with the release of Opus 5.5, Anthropic silently dropped me from the program, and has some strict new criteria in place to apply again, such as being credited for a CVE! I was only approved last month, too -- sad!
I have CVP with the new program (including mythos access) and still get constant denials for silly situations. Most recently I fed a URL to my agent from a security blog and asked if to add it to my obsidian vault with appropriate tags-- cyber flagged. You're not missing much. OpenAI and/or most Chinese models are much more lax in their restrictions.
This kills the talent pipeline, and it'll create a spam problem for the other folks because now people will try to github PR spam their way to getting on a CVE.
It's worth talking about the fact that you can't even talk about DMCA to a model trained on the Library of Congress unless you're one of the approved people. And that's before reverse engineering something or writing code.
So in this future, it sucks to be you if you're someone trying to make your small app more secure, someone trying to upskill, a tinkerer trying to bypass corporate lockdowns for a device they own (a recognized DMCA exception, btw), a teenager trying to learn about security...
It locks away much of the richness that produced hacker culture behind glass. You can look at their press announcements and PR pieces, but you can't touch.
And as they're lobbying the government for "sensible regulation," this inevitably leads to a future where computing is controlled.
It's the direction their existing reports are taking. They recently released one in September that talked about how they stopped "bioweapons." What were said bioweapons efforts? Oh, it was scientists using Claude for grant writing, paperwork and grammar. At national labs.
These people are basically proud of impeding real research to make better painkillers and study a neglected tropical disease, https://news.ycombinator.com/item?id=49651727
And this is being used to lobby against "dangerous" open-weight models because gasp a scientist might use them to write a grant! To make better antidepressants.
At what point do they start reporting someone taking apart an iPhone and trying to DIY a repair with a schematic as a thwarted "cyber security incident?"
A funny, but slightly chilling safety violation I once got was ChatGPT being unwilling to recite the full text of Article I Section 2 of the US Constitution, aborting as soon as it hit the passage about "three fifths of all other persons".
Another funny one was Claude's refusal to provide the original untranslated text of a passage from Dante's Inferno on copyright grounds, though in this case pointing out that no 14th century literature was subject to copyright anywhere in the world was sufficient to override its objection.
Agree on the talent pipeline. It can take a long time for someone to obtain a CVE that has their name on it. You don't start being a security researcher only once that happens.
Several years back, I was working on generating AVB2 hashes on top of modified Android distributions, to increase the security after an owner has made their desired changes. I was doing this before the age of LLMs. Among other things, this would've enabled the secure features to work again, and potentially reduce the risk of root access being usable by malware. But apparently I'm not a security researcher because I didn't get a CVE about it.