Comment by mlyle
8 hours ago
> It'll be a hard sell for critical infrastructure of any kind
Remember, critical infrastructure attached SCADA systems to the internet and has had trouble removing them all despite 30 years of warnings and audits.
There's definite advantages to having a LLM in the loop. The best we can hope for is that it's architected in a way that it fails safe and damages are limited, so we get some of the advantages but less of the downside.
That's a configuration issue, not people preferring a bad idea. SCADA itself is not a hard sell because it's ostensibly deterministic.
If AI cannot provide the same baseline promises while only bring right a fraction of the time, then it's not the right solution. I don't want my car's AC to ask Anthropic what the right temperature is; it's a waste of tokens that isn't optimizing jack shit 99.99% of the time. If a system isn't failing currently, then it's probably in the same boat as car ACs. Shoehorning AI into it won't solve anything.