Comment by alastairr
21 hours ago
If it's real this is a terrible thing to have happen.
However the moral of this story is nothing to do with AI and everything to do with boring stuff like access management.
21 hours ago
If it's real this is a terrible thing to have happen.
However the moral of this story is nothing to do with AI and everything to do with boring stuff like access management.
^This.
One of the top replies on twitter to the OP can be boiled down to "you treat AI as a junior dev. Why would you give anyone, let alone a junior dev, direct access to your prod db?"
And yeah, I fully agree with this. It has been pretty much the general consensus at any company I worked at, that no person should have individual access to mess with prod directly (outside of emergency types of situations, which have plenty of safeguards, e.g., multi-user approvals, dry runs, etc.).
I thought it was a universally accepted opinion on HN that if an intern manages to crash prod all on their own, it is ultimately not their fault, but fault of the organizational processes that let it happen in the first place. It became nearly a trope at this point. And I, at least personally, don't treat the situation in the OP as anything but a very similar type of a scenario.
The LLM didn't have a prod key. It found a prod key in the source base and used that instead of the key it was given.
The access is supposed to be managed in a way that prod would only be accessible with multi-user approval. And that's without even mentioning the fact that storing a key in the source code is a big no-no.
If an LLM can just do whatever after discovering a magic key (in the source code, of all places), with no multi-user approval, it is pretty much the poster child example of an issue with the process that I was talking about earlier.