Comment by verdverm
2 years ago
Repost: https://news.ycombinator.com/item?id=34038663 (11 months ago)
> We employ Large Language Models (currently OpenAI GPT-4)
For IAM, this seems like a disaster waiting to happen. Combining hallucination problems with security settings is not a path I would consider
Do you think humans are doing a better job? Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools. Why not use Slauth and other checks such as policy simulators to get tightened policies pre-deployed
I'm not your target user, I don't feel the priority on this problem even though our permissions are more permissive than we'd like. Thing is, to rein them in typically requires application changes. You cannot just sprinkle magic LLM dust on IAM and make things better.
My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM and system architecture to know if what the LLM is saying is correct, so where does that leave us?
I think they can be an extra pair of eyes, but not the driver. Still, there is a signal to noise problem that remains, due to the inherent hallucinations.
Absolutely not. Anywhere where accuracy, precision and safety matters, throwing LLMs in the mix is irresponsible IMHO or being too optimistic or possibly not understanding how these giant arrays of floating point numbers work or just hoping for the best.
Similarly, LLMs used for SQL generation meant for business analytics is also a critical area where if numbers are wrong, it might lead to a business going bankrupt.
For Prototype, fun exercise, sure go all in.
First of all its pretty awesome your permissions are very tight. You are definitely on the other side of the spectrum compared to the rest. I get it that there is a lot of skepticism because of people hyping LLM's so indeed for now we use it as Copilot and not the driver. Hopefully you can agree though its pretty random that we are still manually creating IAM policies and need to get accustomed with the thousands of different permissions :)
2 replies →
what kind of application changes are you thinking it would equire?
my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).
3 replies →
> Research shows that 95% of the permissions granted to users aren't used
These would be the "s3:*" and "Resources: *" scoped permissions I assume? I can't imagine users are explicitly typing out permissions, 95% of which are not relevant for the task.
> which creates huge problems
Such as? What is the material impact of a workflow or a user having too many permissions?
> and is a reason for spending millions in security tools
Are you claiming that overscoped IAM permissions alone are responsible for 1M+ security tooling bills in companies? Would you be willing to share information on which tools these are?
> Such as? What is the material impact of a workflow or a user having too many permissions?
Security obviously https://en.wikipedia.org/wiki/Principle_of_least_privilege
8 replies →
It's the constant tug of war between the idealized security status where users have just enough access to do their jobs and the fact that it's hard to know the precise access you need until you get the task at which point the idealized process of review to grant access takes too long and really drags down your development pace.
At my job for example we don't have a separate support team for the ETL work we do so I have a lot of access I don't use unless things are breaking and then I can't wait for the access approval process to get added to database XXX or bucket YYY to diagnose what data has broken our processes.
> Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools.
It'd potentially cost millions more to recover from a GPT-4 disaster.
that's a false dichotomy. there are approaches to this problem that are powered by neither humans nor LLMs -- see https://github.com/Netflix/Repokid as an example
One challenge will be similar to self driving cars. The error / fatality rates need to be several orders of magnitude lower than for human operators for it to be acceptable.
AWS and GCP already provide tools to show excess permissions...
The pain there is often a pre-configured role with a slew of permissions was used and you actually need to craft a new role with the right permissions.
I wrote some code once to fetch all those preconfigured role permissions and then present them in a more digestible way
It feels like taking one security problem and creating another problem.
Yeah, when it breaks, and the human didn't write it, it will be a lot harder to fix. It's like being responsible for the output of a junior programmer
My general approach is to spend more time up-front, so when you are in the heat of a problem, you don't have to learn under pressure. I think my beard is graying
I dunno. LLM generated config + formal verification could work.
This would be the way to go with the initial offering. Adding static code analysis + LLMs will help with reducing LLM usage and hallucinations and then adding a way to test out the policies to make sure that they are enough to run the code without being too broad will increase trust in the results.
If it was me, I’d still run QC tools on the generated policy just like I would for manually authored policies. Specific to AWS, the IAM Access Analyzer will confirm that you’re using correct grammar. Further, there are techniques like SCP and permission boundaries to downscope what would normally be all actions/resources.
The space of "real" options in IAM is small enough that hallucination is not a real problem.
Anecdotally I've used copilot to help write a lot of IAM polities in Terraform and the accuracy is basically 100% already.
Same could be said for ECS container definitions yet ChatGPT will happily give you a set of parameters which don't exist.
human in the loop, during the prompt->gen phase, makes a huge difference. You can hit backspace and try different things
With an API that has a hidden / predefined prompt, you'll run into hallucinations that are harder or impossible to handle