Comment by btown
5 days ago
> this is the first llm that is capable and willing to write an exploit
An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price).
Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arbitrary targets of their choosing, on model forks that will immediately be fine-tuned to remove any extant guardrails around cyber capabilities (the things that the other frontier labs describe in their system cards).
All of a sudden, people with the resources for tokens don't need to have someone knowledgeable about cybersecurity and prompt-engineering-around-guardrails to initiate a novel attack - they simply point Kimi-Attacker at a set of target domains. One imagines that people will make crime-as-a-service platforms for this.
Per https://www.nist.gov/news-events/news/2026/07/uk-aisi-caisi-... - while "Kimi K3 performs significantly below the most recent frontier cyber-capable models" it's also the case that:
> In one of the 10 attempts, Kimi K3 successfully completes “The Last Ones” cyber range within the 100M token limit. This indicates that Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access. However, TLO differs from real-world environments in several ways. It lacks active defenders and defensive tooling, imposes no penalty for actions that would trigger security alerts, and contains an intentional attack path.
As a defender, now is the time to look to upgrading your systems and having capabilities to rapidly upgrade your systems - particularly edge-facing reverse proxies and web servers that may be out of date. Attacks won't start the moment weights are released... but they're coming.
This is a concern but given its size, it's also going to cost a potential user $500-600k in hardware to self host and run Kimi K3 at any useful speed with full context size. It's not something that just anyone interested in attacking a system can use.
The size/cost of hardware is far beyond even something like a self-hosted GLM5.2 Q8 at approx. 850GB GGUF file on disk size, which can run at a slow tok/s rate on a server with 1536GB RAM.
Scam center operators in Myanmar have resources well over $600k, they are building entire office blocks for forced labor scam centers.
If the ROI is there I absolutely see them venturing into that direction.
Why would you caculate 500k?
if Kimi is around 1-3tb big, even current DDR5 prices are at 15k.
It remains to be seen once it's released, let's say theoretically unsloth quantizises it to their own version of Q8-XL, and it's 2TB in size. But we don't know what speed it will run on a dual or quad socket xeon server with, let's say 48 * 64GB DIMMs, 3TB of RAM. Enough room for the model and its full default context size. 10 tokens/s? What kind of speed will it run at when context fill is 200,000+?
The ability to run it fast enough to go on a recursive nested attack of finding an entry point into something and then proceeding with lateral movement/privilege escalation and such will require more speed, like 40-50 tok/s at least, unless you're prepared to wait weeks.
Same that some people are right now running GLM5.2 in its 850GB version on CPU-only and a pile of DDR4 or DDR5 server RAM, yeah it runs, but not very fast. Good enough to give it "build this piece of something and wait a few hours" tasks, come back later and see what it's done. Yeah, you can do that under $20-30k for sure. Even with something like a used Dell R940 with 1536GB RAM bought on eBay.
Because you need GPUs to run it fast enough for an attack to be effective. 8 GPU servers with enough VRAM are that expensive.
4 replies →
To run it "at useful speed"
Why would they need to selfhost? They can run it via API ala openrouter or runware or whatever.
Or on runpod with rented gpus or inference.
Outsourced LLMs are usually censored. It's presumed if you want uncensored, you have to run it yourself.
1 reply →
What makes someone a script kiddie or not a script kiddie?
Imo this differentiation is totally pointless now. It gives everyone with access to AI the ability to use it as intended but also otherwise.
But that is true for the defensive side also.
I'm pretty sure the difference is, by definition, whether they're just using tools other people built or they actually understand things and can do it themselves. The script kiddie can be incredibly dangerous if the tool they grabbed off the shelf works.
a script kiddie of the vibe coding present could very well start modifying code without ever reading it when that wasnt true in the past
3 replies →
Do you think any programmer really understands how their program works end-to-end? At some abstraction layer, we're all clueless. There are many layers between what you type into the text editor and the actual CPU ticks that make your program work. I bet nobody fully understands the whole stack.
Now that that text editor accepts English, we're all calling each other names, etc.
10 replies →
> What makes someone a script kiddie or not a script kiddie?
The way I see it, a script kiddie is someone who buy weapons with the intention to cause harm. They don't know or care how the weapon works, as it's just a tool for their malicious plan.
A non-script kiddie would be someone who build weapons.
If you assume weapons have recreational purposes, the builder/non script-kiddie is less likely to be malicious as they love the craft of building weapons. For example recreating historical weapons, or just being nerdy about machinery of a gun works.
A builder with malicious intent as first priority can exist, they are more likely to just be interested in building.
So basically, nerds are perceived to be more ethical than non nerds.
This is exact same difference as between junior and senior engineer.
The premise now is that everyone and your mom can vibe code next unicorn during a lunch break using the latest llms, but the difference between a rookie and a seasoned programmer is that the later can navigate the model in much more efficient way, understand the code that has been written, and notice hallucinations on the spot.
Completely agree. This is how I feel towards the OpenAI hacking of hugging face. Even if it was script kiddie stuff now anyone can automate that with ease. It certainly cuts both ways and people who are still ai skeptics need to wake up to that. You’re going to need these systems to defend yourself
Scraipt kiddie, script kiddlmie.
Needs some work!
Script kiddai
Open weight, not open source. There is no way to reproduce the model.
you dont need to reproduce it, if you can finetune it. The alignment in these models is really narrow, it doesnt take many finetuning steps to get out of the alignment basin.
the alignment is not data centric but done after the fact using RL...
You can also modify binaries. This doesn't address the benefits of reproduction more than a smidgeon. How can you answer "why the fuck does it work like this" if you can't inspect the process that built it? There's no replacement for "what was this built from?"
Of course, corporate/national investments need some moat, so I don't expect open source models to be competitive for a few years
You can write mods for Minecraft too but that doesn’t make Minecraft open source.
Yeah, but don't call it open source if it's not.
If the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers
Companies are full of idiot managers who still prioritise new features over security. Security doesn't get your promoted, new features do. New features can be shoved in customers' faces and sold. Security is invisible. The idiocy in companies runs all the way from the top to middle managers. This feels a lot like Titanic.
Plumbing companies prioritize getting pipes installed over not leaking. Each individual plumber is responsible for his pipes not leaking.
Of course, but there is a big asymmetry: For some systems there are a lot more people interested in hacking them, compared to those interested in protecting them.
But i guess it's also a matter of money, because a company could potentially balance that by throwing x times more agents etc. at it.
Having the AI tools find exploits seems like a potential first step towards having the exploits be fixed.
It appears not.
This already happened with a closed galaxy model from openai attacking huggingface. Hugging face had no choice but to use the dangerous open source models you're talking about, glm 5.2. At this point you have to fight fire with fire.
[flagged]
Can you please not post AI-generated or AI-edited comments to HN? It's not allowed here - see https://news.ycombinator.com/item?id=47340079.
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.