Comment by 9x39
7 hours ago
If you believe some of the emerging comments out there - try "Danfoss CVE" - you'll see that some of the suspects are 15-20 year old controllers like the Danfoss SM800 (EOS 2021) and SC355 (EOS 2016).
Why don't they just--- no margin. But there were CVEs-- no margin.
The laptop class doesn't always grok that the shift-work labor class has nothing in terms of luxuries like security reviews and gear replaced on cycles and other basics taken for granted.
But I'd setup Tailscale and-- would you, the grocery store manager running on 3% margin, really? What if you're part of a chain and it's centrally controlled elsewhere, like all the other decade-old tech you use but have to call and wait for a dispatched tech to handle?
Secure-by-default laws will eventually help, but there's always going to be groaning infrastructure particularly in commodity industries, and we still haven't onboarded most of the disaffected men to the Internet that will commit innumerable more hacks and disruptions.
> The laptop class doesn't always grok that the shift-work labor class ...
I think it's more than this. Industrial control system upgrades are serious business and (at least if you're doing them properly) require a lot of planning and consideration. Particularly when the subsystem being replaced is decades old already.
I think the overwhelming majority of people (including on hn) have no understanding of how complex industrial systems are - even as-built, but particularly after decades of evolution. Doing proper engineering in these spaces is expensive.
You can reasonably blame supermarket chains for being insecure, but we certainly aren't currently enabling them (or anybody else) when it comes to fixing these issues.
> I think the overwhelming majority of people (including on hn) have no understanding of how complex industrial systems are - even as-built, but particularly after decades of evolution
I may not have experience, but it is standard security practices that monitoring and control must live in separate channels for security. This is not a "planning and consideration" concern, this is conceptual design of systems.
Is it more expensive to keep your systems upgraded and maintained or to have all your product get destroyed one arbitrary day and then have to pay to do the upgrade anyways so that the arbitrary property destruction doesn't keep happening?
In before "but insurance"; yes insurance can in theory mitigate some of this, but I feel like people on HN have no understanding what it's like to deal with a multi-million dollar insurance claim that the insurance company 100% does not want to pay out. The legal battles over such things take years and are on their own incredibly expensive.
> Is it more expensive to keep your systems upgraded and maintained or to have all your product get destroyed one arbitrary day and then have to pay to do the upgrade anyways so that the arbitrary property destruction doesn't keep happening?
It happened once. Did you do the maths here? Can you tell us which one was Kore expensive and what percentage chance there was of this hack happening?
3 replies →
That class of risk was just theoretical for decades, so the price on it was basically 0.
Now that we see large scale incidents will there be an actual money value associated with upgrading systems and security for supermarket appliances.
4 replies →
In my recent experience (last 3 months) with a cyber claim getting insurance to pay out was actually pretty straightforward. It was about a $250K USD loss on a phishing attempt (someone hijacked and rerouted payment details for a wire)
But you don't need to replace entire system to massively reduce surface area for the attack. Just do what was done for decades, and put the vulnerable devices between VPN and/or additional authorisation layer.
It's far less work than touching anything on controller side (at most changing some network settings) and requires attacker to at the very least penetrate your network perimeter and snoop around to find it rather than "run nmap on a controller's default port over country's IP range"
I work in SCADA development, and believe me, there's a reason the industry moves slowly.
This slowness also means it sometimes has a hard time understand new risks and a lot of industrial systems are not operated by people who understand those risks and do…very stupid things.
This comment could be a lot more interesting, useful and worth typing than it is. :(
3 replies →
Go on.
The fix is taking them off the internet, use DOS, and require manual insertion of a floppy to make updates.
Use CP/M. It's the only way to be sure.
I get the first and last point, but how does DOS help?
You can use a modern operating system just fine.
1 reply →
Yeah the margins are tiny because private equity has spent the better part of the century destroying American dynamism and outsourcing all externalities including getting hacked to the people.
This is only partly true, historically before private equity was a thing grocery was always a razor thin margin business that operated on volume. PE just made it worse.
2 replies →
This is happening in France.
Doesn't sound all that convincing to me. There is no need for network controlled fridges in the first place. And it's not like setting up that network control was free. This is not about saving money, this is about ignoring what IT people preach since multiple decades now, spending more for the attack vector and then loosing a lot of money when that attack vector gets used. Margins? If it were about margins the problem wouldn't exist.
(I can see the need for remote monitoring, that the freezer actually worked. Controlling the temperature is the issue.)
Do you say that as a supermarket manager, or a random person on an Internet forum? Because if it's the first, I'm very interested in hearing more on how you actually manage all the devices in your store and which ones require remote automation and which ones don't.
Automation doesn't require connecting to Internet.
4 replies →
I know somebody that works in HVAC. Two or three years ago they asked me about exposing the web-based controls for a meat processor's refrigeration system to the internet for remote administration. I advised them not to because it was an old janky-looking turd that hadn't been updated in something like a decade and suggested enlisting an IT person to stick a cheap OpenWRT router with Tailscale in front to protect it. Of course there was no money to do this so they asked what would happen if they did it anyway. The first thing I thought of was a disgruntled former employee turning the system off to spoil six figures of inventory and stink up the building. They paused for a moment and said "hmm... yeah, it sounds bad when you say it like that".
I've no idea what they ended up doing after that.
E. Leclerc - $56.87bn revenue (2019)
Intermarché - €44.5bn revenue (2024)
Cooperative U - €28.35bn revenue (2025)
These aren't struggling grocery stores. They are massive businesses collectively running several thousand stores and employing hundreds of thousands of people. They have IT departments.
Revenue tells you scant little about the health and profitability of that business. Care to share profit and gross margin numbers? Grocery stores are notoriously low-margin.
Those numbers aren't available, but they're irrelevant to the point anyway.
The post I replied to (currently the top post on the article) is trying to paint a picture of some harried manager of a struggling grocery store having to fire up networking tools, which is ridiculous. A company doing tens of billions in revenue employs or contracts some of the "laptop class" (to use that post's terminology) to do that, regardless of what their profit margin is. They don't get a "shift-work labor class" employee to take a break from stacking shelves and check the firewall config.
This is a very weird thread.
3 replies →
I don't get it - it is not like the people working in a supermarket are expected to pay for securing the hardware in it. It is up to the owners who clearly have the money to run a supermarket chain not to keep insecure shit exposed to the Internet.
We all end up paying for that though
Thank you for saying this. It's quite frustrating to read some comments here that have obviously lost touch with the reality of the vast majority of people and businesses across the world.
There really ought to be a market for a little device that goes between the IoT/industrial device and the rest of the world and that acts like a fancy, self-updating proxy.
Conceptually, you have a physical plant and a PLC and maybe an HMI. Quite sensibly, no one wants to modify this once it works. But you also have a remote access system, and all the remote part should be through a proxy that is continuously updated and only grants access to authorized users. And preferably logs everything and also can act as a sort of web application firewall.
Tailscale is fairly close, and they should consider selling a secured (attested boot, tamper resistant) device with two Ethernet ports (PoE on the uplink side) and WiFi client device capability. Stick a serial number on it, let it be added to a tailnet, and make it impossible for anyone to add anyone else’s device to their own tailnet.
BOM cost would be maybe $20 - we’re talking higher end ESP32-class hardware plus the Ethernet hardware. 100Mbps would be adequate.
Hmm, startup idea: anyone could sell this. The device has a little bootloader and two modes. In provisioning mode, it attests to the mothership (mothership is the vendor, but enterprise plans give an air gapped mode where the vendor can provide a signed config blob and the device can fetch it over TFTP, PXE-style) for configuration. In provisioned mode, it does what it’s configured to do, and it can attest to its firmware + config for purposes of authenticated to, say, Tailscale.
It’s called a firewall?
So what you're telling us is that there is tremendous opportunity for a startup to disrupt the space with new Refrigeration-as-a-Service (RaaS) products controlled by AI. That way we can replace those legacy microcontrollers with modern, secure GPUs and vibe-coded AI agents.
I find the case of having enough money to put all of that on internet and centralize management and not having enough of it to set up a rudimentary VPN unlikely. The money is not the reason.
Those systems are not cheap. It's not the case of lacking few hundred bucks to put it between a VPN box.
It is simply just "well, we weren't hacked before, why change?". Back when those were installed that was just not in the field of view and most those people worried about is changing default password.
And there is another case for industrial equipment, even if it is out of support, what you gonna do, trash a machine because of it ? No, it will be running till it falls apart or it will be more cost effective to replace.
> But I'd setup Tailscale and-- would you, the grocery store manager running on 3% margin, really? What if you're part of a chain and it's centrally controlled elsewhere, like all the other decade-old tech you use but have to call and wait for a dispatched tech to handle?
You either pay small cost now, or bigger cost when someone finds out. So, yes, really. We live in times where you no longer have to pay $500+ for enterprise grade VPN box, you can do very acceptable security wise solutions on cheap hardware or on literally "any old piece of junk"
I’d be willing to bet many more HN users work for companies who are the ones creating unnecessarily network connected smart devices than work in infrastructure, trades, retail-adjacent positions that implement/use many of these network connected devices.
Lots of completely out of touch comments
I don't know how it is in Europe, but here in Canada, looking at grocery stores in isolation makes little sense when most of the market is controlled by a handful of vertically integrated conglomerates. On top of the stores, they own a lot of the distribution, wholesale operations, real estate, and even some suppliers. Their own financials show record revenue and profit, year after year, and their margins have increased significantly over the last few years too. We can't reduce the economics of such integrated corporate groups to their retail net margins alone to judge profitability. As far as Canada goes, small margins my ass.
> their margins have increased significantly over the last few years too.
Empire (Sobeys et al) has seen consistent profit margin of ~2% for the last 2 decades.
Loblaws and Metro have higher profit margins at ~3-4% attributable largely to their acquisition of pharmacy brands which function as high margin grocer/convenience storefronts.
This is a pet peeve of mine (as a Canadian): I'm not going to write an apologia for the oligarchic grocery system we have in Canada, but when I hear about headlines about "record profits" it's alway about the amount — which of course will go up if revenues are up because of inflation of inputs/costs. But I always ask about the margins: have those gone up? (They generally are pretty steady.)
The margin is certainly blown, now
> The margin is certainly blown, now
For (just?) this fiscal. By saving on IT costs in previous fiscal years they were perhaps able to book profits.
One incident every (say) decade or so may be determined to be worth the trade of between the costs of 'proper' IT practices and getting pwned (and the resulting spoilage) so they can book profits for the other nine years.
They probably have insurance for things like this. Speculation, but still. Lots of businesses mitigate risk by just buying insurance.
Insurance for cyber incidents usually only cover you if you took reasonable measures. Like not exposing your internal systems to the Internet.
2 replies →
I can imagine that there's already an industrial security company at their door, offering a solution to prevent this from happening ever again, already tailored just for them.
Maybe it's even honestly not related to the attack.
The hardware vendor has long offered software updates. Unclear if the groceries were paying for support, they certainly weren't installing them.
Boeing still took the blame for MAX, when their customers didn't pay for training.
1 reply →
Target learned the security lesson the hard way, but they definitely learned it. When I was there, security was one of the rare parts of the tech org I found legitimately impressive. (But also: I am not a security guy! Maybe their actual practices were compelling but flawed! But the organization as a whole was legitimately serious about it )
Target learned the security lesson the hard way
About the 2013 Target data breach, https://krebsonsecurity.com/2015/09/inside-target-corp-days-...
Target is a far outlier in the insane shit that they do in terms of retail security. No one else really approaches them in this regard.
I worked with the Target security team on a few projects and I definitely agree that they take it seriously
Apparently Home Depot has a very good developer culture too.
Exactly, and "Early reports indicate losses are already approaching 300,000 euros, but could run into several million euros nationwide." - IOW, the attack has cost far less money than it would have cost to prevent it with better security.
See also: the McDonald’s ice cream machines by Taylor, their half assed and overpriced vendor locked in system, the raspberry pi based alternative, and then the ensuing legal and commercial chicanery
The profit motive absolutely favors neglect, and may even punish attempts at reasonable maintenance
It's not entirely clear which comments/criticisms you're responding to when pointing out the lack of margin. Whose excuse is that? And what's it an excuse for?
Security isn't optional just because margins are low. If eg FedEx decided to disregard security because they only made 3% of the value of the packages they ship in profit, they'd lose most of that business fast. But yes, let's not shame folks for failing to preempt theoretical-yet-unseen threats, if that's what you mean.
I think it's that people who've never faced a security threat and never had to think to are among the most vulnerable. And grocery stores are notoriously thin margin businesses who actually strive not to spend unnecessary the most.
At least that's what I got from it.
Ok, then why were their damned fridges connected to a network then?
"The chain wanted to have Internet enabled control and dashboards about their fridges" or something?
You don't get to say you want the perks of being Internet enabled and then turn around and say that protecting against the downsides would cut into your margins too much. Sheesh
2 replies →
If they aren't going to update insecure controllers then they absolutely must isolate them from the internet.
Framing IT vs OT differences as a class struggle?
Sir, I think you just invented entirely new type of technomarxism. Sunrise, parabellum!
...
As fun as it is, this has nothing to do with laptops and everything to do with someone not implementing Purdue model right.
The reason why safety regulations are written in blood - is because no one wants to pay until the consequences become blatantly obvious and impossible to ignore.
This is a clear example.
[flagged]
Of course not. It is precisely because they have to fight for scraps that this is the place where workers are beaten down the most, because the owners are making pretty damn sure they will be the ones to get that margin lining their pockets, everyone else be damned.
In high-margin industries, you can give a 3% salary increase and free coffee and snacks, it is a rounding error compared to the 20% yoy increase applied to your multi-billion dollar compensation.
> Of course not. It is precisely because they have to fight for scraps that this is the place where workers are beaten down the most, because the owners are making pretty damn sure they will be the ones to get that margin lining their pockets, everyone else be damned.
My class-brained friend, margin is what you get after wages, not before.
I was talking about the supposed oppression of customers through high prices, but it’s typical that communists imagine that the purpose of an enterprise, private or public is to provide jobs to union workers rather than to actually useful products or services to customers.
They had plenty of margin to pay for unnecessarily cloud-connected refrigerators.