Comment by DanielSlauth
2 years ago
First of all its pretty awesome your permissions are very tight. You are definitely on the other side of the spectrum compared to the rest. I get it that there is a lot of skepticism because of people hyping LLM's so indeed for now we use it as Copilot and not the driver. Hopefully you can agree though its pretty random that we are still manually creating IAM policies and need to get accustomed with the thousands of different permissions :)
To add a plus one here, as soon as I learned there's LLMs involved this became a non starter to me. I'd rather have less granular policies than risk some LLM doing something crazy.
I can justify to management that we have limited time for IAM and something was missed that we can fix / create tests / scans for after an incident. It's harder to explain that we chose a vendor that uses a non deterministic tool that can hallucinate for one of the most core security pieces of the puzzle.
We are actively working on reining in permissions, I would not call them "tight". It's just not a top 3 priority, though that is likely changing with the upcoming SOC2 efforts. I still don't see us reaching for LLMs to help us here.
I'm not saying don't use them, just use them as an extra pair of eyes, mostly to catch errors rather than to drive and architect
> get it that there is a lot of skepticism because of people hyping LLM's
The skepticism is not from the hype, it's from experiencing LLM output personally. They are fine if the output can be fuzzy, like a blog post or a function signature, not so much if there is a specific and fragile target.