Comment by josephg
5 hours ago
> It is unthinkable to me that anyone believes there is such a thing as computer security after so many years of nonstop hacks and leaks.
Of course there is. For example, SeL4’s security and reliability proofs still hold in the world of LLMs. The problem is that most software isn’t written on that firm foundation. Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”. You don’t get secure software by working like that, because security vulnerabilities aren’t visible.
We - humans - know how to write secure software. Just like we know how to make safe aeroplanes. The problem isn’t that we lack the capability to make secure computers. The problem is we don’t have a culture of security. Secure software is - somehow - niche. And as such, it’s much more expensive. And nobody wants to pay.
> Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”.
I work in secure systems and it’s shocking how many people believe this - the incentives from management are all about it too.
I like to think behind every Dev anxious to ship half baked software sits an omniscient middle manager with a vague idea of what the product was supposed to do, maybe
I believe the technical term is “Move fast, and break things.” MVP is a huge disaster. I can see it working for applications that don’t process PID, but only an idiot ships data handling software before it’s been dragged through a lot of testing. I tested my app for two years, before finalizing, and an LLM still found a couple of holes (minor ones, but ones I missed).
After the DOGE debacle, I suspect that all the previously really secure stuff, is now out there, too. In fact, I wouldn’t be surprised if some of these leaks, came from that.
FBI employee data is very bad.
The issue is that in consumer and enterprise software, move fast-and-break-things outcompetes secure-by-default every time. Critical infrastructure needs to have a different set of priorities, but it’s very hard because the expertise is so thin on the ground. Why would anyone with the expertise to make these calls bang their head against the wall trying to educate bureaucrats about these things for $150k a year when they can easily make multiples of that in big software companies that don’t own that level of risk.
NASA was supposedly a CMM level 5 organization and they still managed to crash a mars probe because of bad unit conversions.
Did any martians hack the probe?
<< And as such, it’s much more expensive. And nobody wants to pay.
Eh. If only it was that simple. I mean, yes, money is always a factor, but not nearly as big of a factor as 'my convenience outweighs pretty much everything ( until it causes sufficient amount of havoc.. and even then.. )'. You can see it in just about everything. It is not just the money. It is the convenience that drives most of the unsecure behavior.
The bank has the best doors, the best locks, and the best cameras, and it is patrolled by a guard who props the doors open to so he doesn't have to keep fooling with the locks and points the cameras the other way to extend his smoke break. SeL4 would be another system used by humans.
It's always possible to break a perfect system by moving an additional layer of abstraction outward, and attacking one of the assumptions upon which it's built. Some of our era's highest security systems - game consoles - have been broken by undervolting them until the logic failed.
> It's always possible
It's often possible. But not all systems are vulnerable to undervoltage attacks. For example, I don't think the iphone secure enclave is vulnerable to this.
And good security uses "defence in depth". Multiple layers which each individually need to be compromised to break the whole thing. To hack chrome, you need a vulnerability in the renderer or VM. Then you also need a sandbox escape, and a way to use that to attack the browser's parent process. This is much harder to do.
6 replies →
I once worked on AUD 450M banking project, the root password was kept in a kickstart file and unchanged, root SSH was allowed. The bank didn't care until I told the external security auditor who included it as part of their report.
Snitch
2 replies →
There's absolutely no way to account for humans, who can be tricked, or pressured, or just make human sized mistakes.
Then there's no such thing as security.
By the way, there are countless ways to account for humans. There are entire branches of engineering devoted to this. If you don't want someone to leave the bank with a pen customers use for signing checks, you just chain it to the desk. If you don't want the installer to forget to put the pen-chain in, make a photo of the chain part of the checklist required to get paid. If you want to... etc.
The idea is that you determine an acceptable level of risk, then secure to that level. Maybe the acceptable level of risk chosen by companies is wrong. Maybe we need to increase that risk exposure via heavier fines and regulations. Maybe the cost of reducing that risk is too high already. Maybe we need to fund that. Maybe it's too confusing and we need to research better standard practices. I dunno. But this is not some unsolvable problem.
Again, of course there is.
Decades ago, I worked in a bank in an old building. The door had a card reader for access. You boop your card and the door opened. People would hold the door open for each other all the time out of politeness, even when they didn't know each other. Security told us not to do that, but it's hard to convince people to stop being polite.
I had a laptop stolen from my desk in a place like that once. (Not a bank - but similar door-card reader system). This guy came in in the middle of the day, wearing overalls. He confidently walked through the door after someone, like he belonged there. He walked up to my desk, swiped my laptop and just strolled out.
At the bank, they've replaced the door with mechanical gates and a security guard. The gates - physically - only let one person to walk through at a time. You can't hold a gate open any more. And the security guards stop anyone who tries.
Is it 100% foolproof? No. But it's way more secure. It would have stopped that laptop thief.
There's this pernicious, defeatist attitude that if you can't make a system 100% secure, so you shouldn't try. That's misguided. Most systems can be made orders of magnitude more secure than they are today. It just takes a bit of care and work.
5 replies →
In cryptography there are protocols that account for malicious actors
sel4's guarantees break if you have DMA, e.g., from your NIC. It doesn't help with timing attacks. It doesn't cover your network stack. AFAIK, no one has taken up the mantle from Project Everest, so you'll need to write a verified TLS library. Once you've built all that, you can start thinking about your database/application/whatever. Then of course you'll have to verify all your dev's machines and scripts to ensure nobody is misusing a credential that can get stolen.
"So what?" you say. "Making a heavier-than-air metal tube take off and land millions of times per year without a catastrophe is also hard, and we no longer expect most or even many of those tubes to blow up or fall down."
Mother nature is not spending $$$ using AI and HI adversarially trying to find the exact combination of atoms that will cause your device to fail.
> sel4's guarantees break if you have DMA, e.g., from your NIC.
Modern CPUs support IOMMU. If you set that up, your NIC can only DMA to virtual addresses, managed by the operating system.
> It doesn't help with timing attacks. It doesn't cover your network stack
It does help with all this stuff, because your network stack and whatever else can be split off into isolated processes which talk over capabilities. Compromises in those processes are of course terrible. But they don't automatically allow kernel level takeover of the whole machine like on windows / linux.
> The problem isn’t that we lack the capability to make secure computers.
Depends on the "we". "We" have the capability to make secure computers like how "we" have the capability to make EUV lithography machines. There exists a relatively small number of people and organizations in the world who can do so. Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc. All of those organizations have tried for literal decades, thumped their chests about how they have awesome security year after year, and yet have totally and utterly failed despite their best efforts.
Acquiring the capability to do so is difficult and challenging and requires years to invent if you start right this very second and know what you need to do, which these organizations emphatically do not. We need security at scale and fast. The only way forward is to scale up working solutions rather than letting the bozos who put us in this spot fail at scale with yet another promise that this time for sure they will solve the problem they have repeatedly failed at for decades.
It's nothing like EUV.
> Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc
This is all by choice. They could easily have that capsbility, very unlike EUV.
And the reasoning behind that choice is looking at the tradeoffs and saying “shipping more features faster is more important”
1 reply →
Oh, Microsoft can just figure out how to make unhackable systems, they just choose not to. They spend all of those billions of dollars per year on security and spent all of those decades on failed attempts as a prank.
You really think that if they could have they would not have, even just for bragging rights? Or are we going with that it is some kind of task demanding enormous expenditure even though the organizations that have made secure systems are infinitesimally small in comparison?
Microsoft has spent orders of magnitude more money and time than the organizations that have succeeded and the result of their efforts is Windows. That says everything you need to know about their capabilitys.
Multiple literal trillion dollars organizations have spent literal decades failing at it. You are really underselling the capability gap.
Yeah it's about costs. I care about a lot of things, but the causes I actually give money to is a much shorter list. If personal data was radioactive, and leaking it cost companies real money, then more money would get spent on security. (and insurance, and lawyers.)
Yeah I've long said we should treat data leaks like food safety. The only way we'll see software security improve is if there were serious fines and/or jail time for leaking user data due to negligence.
PCI compliance works pretty well.
I would like to see restricted access to only us workers for us citizens data though.
Transferring us customer data outside the country should be illegal
2 replies →
Cybersecurity is a fragile system in Taleb's fragile-robust-antifragile framework. That is, as t->infinity, the probability of a hack approaches 1, because you only need to slip up once, and there is a small probability of the system's maintainers slipping up each day.
[dead]