If we assume that there will be an AI that is perfect in terms of ability to find vulnerabilities, cheap to run and widely available to everyone, then anyone can run it on any piece of software before deploying it. All vulnerabilities get found before they can be exploited.
One of the big challenges with cybersecurity is that attackers only need to find one exploit, while defenders need to stop everything. When you have a large surface area and limited resources, it's much easier to be the side that only has to succeed once. AI eliminates the limited resources problem.
Right now we are at a point in time when AI can find bugs for attackers and defenders, but defenders did not fix/find those bugs yet.
In time most of the bugs AI can find will be fixed, and things will calm down. Some bugs will be left, but will be too complex to find and weaponise (or rarely).
Alin short, attackers have advantage for a brief time now, but ultimately defenders will win. I guess this "fight" might be over before the end of the year.
1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliability/privacy/etc)
Right now all that stuff is optional, so most companies don't do it, which makes more security holes and it takes longer to patch.
We could get somewhere where clouds can provide a framework of secure primitives that act as a framework.
E.g. you build an app, it stores data via api etc. etc. You can test in sandbox. The cloud deploys for customer who paid you via that cloud and you work at arms length. You may not even know their name. You just get the pro subscription fees.
The idea bubbling in my head would be an app store for cloud products. But with competition i.e. you use Railway or Heroku or AWS for the best deal.
Be gentle this is an idea in my head I am sure it can be torn down by a retort at this stage. But this exists in forms and I think it will emerge. It is inversion of control at the entire app level.
This is similar to buying a hammer. If you make hammers you sell them to a store, the store knows the customer and only the customer can see the nails.
I am looking at the results of a mass vulnerability scan as I type this. Half of the bugs in one case are in fact (binary) parser errors for hand-written parsers. These really should not exist in any language - but in C it's particularly bad. Kaitai Struct or something similar would broadly have prevented these. Rust would help here, but less than a parser generator (because it could automate error checking insertion for things that aren't just out of bound access).
However, half of the vulnerabilities are logic errors in terms of what I would call RBAC enforcement, incorrect access permissions, and so on. Rust won't help at all with any of these.
I was just working on a system best thought of as a “dinosaur”: written almost entirely in C (and a bit of PERL) and running on an appliance with BSD as the kernel.
It’s full of bugs and has had a string of RCE vulnerabilities published recently, probably because of Mythos.
Working with it day to day I get this feeling that the tech stack used results in a system that’s… clumsy and constrained.
Little things give me that impression, and I can’t quite put it in words, but it’s thirty years of experience working with dozens of languages and platforms speaking here.
Using C makes you clumsy.
It makes you trip over things other languages don’t.
It makes it obscenely difficult to do even simple things. It’s like trying to put a delicate ship into a bottle while wearing oven mitts.
Switching to a better language isn’t just about the specific capabilities of its compiler, it’s also about what it enables in the humans using it.
If we assume that there will be an AI that is perfect in terms of ability to find vulnerabilities, cheap to run and widely available to everyone, then anyone can run it on any piece of software before deploying it. All vulnerabilities get found before they can be exploited.
One of the big challenges with cybersecurity is that attackers only need to find one exploit, while defenders need to stop everything. When you have a large surface area and limited resources, it's much easier to be the side that only has to succeed once. AI eliminates the limited resources problem.
> If we assume that there will be an AI that is perfect in terms of ability to find vulnerabilities
...so if we assume a halting oracle?
I'd speculate that at this point Linux etc are probably having vulnerabilities discovered and patched faster than created.
It's not only Linux though and many projects don't have the funding to perpetually use something like Mythos.
Right now we are at a point in time when AI can find bugs for attackers and defenders, but defenders did not fix/find those bugs yet.
In time most of the bugs AI can find will be fixed, and things will calm down. Some bugs will be left, but will be too complex to find and weaponise (or rarely).
Alin short, attackers have advantage for a brief time now, but ultimately defenders will win. I guess this "fight" might be over before the end of the year.
1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliability/privacy/etc)
Right now all that stuff is optional, so most companies don't do it, which makes more security holes and it takes longer to patch.
Basically make software development so legally risky that only multi-billion dollar corporations will ever engage in it.
We could get somewhere where clouds can provide a framework of secure primitives that act as a framework.
E.g. you build an app, it stores data via api etc. etc. You can test in sandbox. The cloud deploys for customer who paid you via that cloud and you work at arms length. You may not even know their name. You just get the pro subscription fees.
The idea bubbling in my head would be an app store for cloud products. But with competition i.e. you use Railway or Heroku or AWS for the best deal.
Be gentle this is an idea in my head I am sure it can be torn down by a retort at this stage. But this exists in forms and I think it will emerge. It is inversion of control at the entire app level.
This is similar to buying a hammer. If you make hammers you sell them to a store, the store knows the customer and only the customer can see the nails.
2 replies →
Legal risk is what insurance is for. You get ensured for a small fee and you go about your job. That's how the non-software world operates anyway
1 reply →
Downplaying security has now real coencequences for everyone.
Bulk rewrites of everything into Rust with AI assistance?
I am looking at the results of a mass vulnerability scan as I type this. Half of the bugs in one case are in fact (binary) parser errors for hand-written parsers. These really should not exist in any language - but in C it's particularly bad. Kaitai Struct or something similar would broadly have prevented these. Rust would help here, but less than a parser generator (because it could automate error checking insertion for things that aren't just out of bound access).
However, half of the vulnerabilities are logic errors in terms of what I would call RBAC enforcement, incorrect access permissions, and so on. Rust won't help at all with any of these.
I was just working on a system best thought of as a “dinosaur”: written almost entirely in C (and a bit of PERL) and running on an appliance with BSD as the kernel.
It’s full of bugs and has had a string of RCE vulnerabilities published recently, probably because of Mythos.
Working with it day to day I get this feeling that the tech stack used results in a system that’s… clumsy and constrained.
Little things give me that impression, and I can’t quite put it in words, but it’s thirty years of experience working with dozens of languages and platforms speaking here.
Using C makes you clumsy.
It makes you trip over things other languages don’t.
It makes it obscenely difficult to do even simple things. It’s like trying to put a delicate ship into a bottle while wearing oven mitts.
Switching to a better language isn’t just about the specific capabilities of its compiler, it’s also about what it enables in the humans using it.
1 reply →
Rust is overly complex and difficult, Go is simpler and easier and has the memory protection people are obsessed with