Comment by Quothling
5 days ago
I think it depends on what area of the world you work. We recenty had one of our plant managers build a web portal to keep track of some of our operational tech, with features you wouldn't find in standard products. I've been turning it into a container app that can actually deploy safely into our cloud infrastrcture. It's quite frankly better quality than what most external software companies have provided us with in the past.
Don't get me wrong. It's not great. It would never pass any of our policies for things that actually operate stuff on the power grid, but as an administrative tool that can live in total isolation from the vital networks. It's perfectly fine. It's also not like we would have hired the best software companies to build it otherwise. We'd hire some low-level cheap consultant house who would then likely get cheap student labour to build it. With that in mind though, the AI is much better than what the realistic alternative would be.
Money wise it's also cheaper. It's been roughly €1000 + the time it's taken us both. If I had known they were doing it, I would have rolled out the developer cowork app/skills/whateveryoucallconfigurationsthesedays to them. This would have avoided their AI building it to be depoyed on a VM rather than in our managed k8s in our Azure. It would also have written the code a little different, used UV and maybe django rather than flask. But hey. For what it is, it's like a 90% cost saving compared to buying what would've been a less maintainable and lower quality system.
I think perhaps the greater issue will be finding people who want to extract the gold from the heap of shit and getting it to run in production. I don't personally mind, but it's not like any of my colleagues would've wanted the task.
By coincidence, we've been building software for five years that other plant managers are now considering using to keep track of some operational metrics. And I don't think we should put LLMs into our development cycle, simply because the data our software is built for can have critical meaning in some situations.
The insecurity in a vibe-coded web portal isn't that someone hacks it with XSS, it's that after the next vibe-coded release, some X quietly becomes −Y somewhere no one expects.
From this perspective, having no software at all might be better, or as in your case, safer.
I don't think you should keep AI away from software development for security reasons. We're in the European economic area and are completely NIS2 compliant and we use LLM's to aid in our software development for high risk systems. None of that is vibe coded though, so it's not like I disagree with you either.
As you point out this portal isn't that, but what protects us is the processes around compliance. This can't grow from X to Y because not even the CEO has the authority to overwrite our compliance gates. The EU is a tremendous help in this area since personal liability changed things completely.
> Quothling 6 months ago [1] I work in a NIS2 regulated sector and I'm not sure we can ever let any AI agent run in anything we do.
> We're in the European economic area and are completely NIS2 compliant and we use LLM's to aid in our software development for high risk systems.
Please pick one. Either way this is a nightmare level of threat to sovereignty.
1. https://news.ycombinator.com/item?id=47063153
7 replies →
Denial of Service is still an issue whether its from an external attack or spaghetti soup code bug. Outcome for your users is the same.
From my perspective it looks like were just allowing hostile developers within our environments now lol.
1 reply →
If you aren't using AI to write your code you should definitely be using it to find bugs in the code you write by hand.
I wonder how much of this "simple but bespoke" stuff would be better served with some customized off the shelf software rather than a vibe coded tool.
There's not a lot of software where users dont really care if it goes wrong.
I doubt it would be better served as a customized off the shelf software. They certainly could have done it with our existing systems with SmartSheet, Microsoft Fabric and/or Power Apps with SharePoint for document storage.
From an enterprise perspective this becomes complicated for various reasons. RBAC is one area. In the perfect world you have a system to handle roles and rights to every system, something that you can give managers access to so they can maintain the access available to their employees, something linked with HR. In reality you have EntraID with a hieracy which is sort of automated by HR data, but not really, because sometimes HR puts everyone on the CEO level by mistake, and, if you trusted HR as authoritative that would've just broken all the EU laws. So you have all those Entra groups and you need IT Operations to maintain them and since you want to build it on job roles and not people you'll typically not be able to maintain them in the off the shelf system. Which means that you would have had to build a web portal for the plant managers manager where they could maintain a couple of Entra groups in a web interface. That or you'll have to setup an IT support flow where you add yet another system that IT has to maintain access for.
Then we get to the actual customization. Maybe you buy a custom API on top of your BC365 platform. Maybe your C-levels deciced that paying €50k a year to avoid outages on major updates isn't worth the cost. Then when things predictably and completely avoidable fail you're going to hav to deal with the literal shitstorm. You'd think that all the people being locked out of their jobs and the €150k cost of getting an immediate and prioritised update to the system would mean you'd start paying for that $50k service after this. You'd be wrong. Ok, to be fair, in this particular example it would be a different scenario. For a small system like this you'd find a cheap consultant house in your area and get them to build the customization for you. Only they would outsource it to some solo developer who will build it in a way that basically requires that specific person to alter it. Then when it breaks or needs to be customized futher a year down the line, that person is no longer a solo developer. So you reach out to another cheap consultant house and do it all over again, from scratch.
This doesn't even mention how poorly all those 300 off the shelf systems work together. I mean, I don't maintain a SDK delivering a way to use Apache-Arrow to write and read parquet files from our datalake in the same manner for fun. I do it because those 600 container apps which basically simply translate data from one system to another need it to be as slim as possible.
Am I jaded? Sure. But who isn't in enterprise IT?
for an administrative tool the criticality is not determined by the network boundary but by which decisions within the plant are based on these figures
With the models we got at out disposal, you can easily deploy a kickass internal tool, it's fine even for mvp's unless you're not handling sensitive user data, shooting yourself in the foot is easier than ever, and talking about code smell for most application (especially frontend) does it even matter?