Comment by throwaway_z0om
3 days ago
> the "policy people" will climb out of their holes
I am one of those people and I work at a FANG.
And while I know it seems annoying, these teams are overwhelmed with not only innovators but lawyers asking so many variations of the same question it's pretty hard to get back to the innovators with a thumbs up or guidance.
Also there is a real threat here. The "wiped my hard drive" story is annoying but it's a toy problem. An agent with database access exfiltrating customer PII to a model endpoint is a horrific outcome for impacted customers and everyone in the blast radius.
That's the kind of thing keeping us up at night, not blocking people for fun.
I'm actively trying to find a way we can unblock innovators to move quickly at scale, but it's a bit of a slow down to go fast moment. The goal isn't roadblocks, it's guardrails that let you move without the policy team being a bottleneck on every request.
I know it’s what the security folk think about, exfiltrating to a model endpoint is the least of my concerns.
I work on commercial OSS. My fear is that it’s exfiltrated to public issues or code. It helpfully commits secrets or other BS like that. And that’s even ignoring prompt injection attacks from the public.
In the end if the data goes somewhere public, it'll be consumed and in today's threat model another GenAI tool is going to exploit faster than any human will.
I am sure there are many good corporate security policy people doing important work. But then there are people like this;
I get handed an application developed by my company for use by partner companies. It's a java application, shipped as a jar, nothing special. It gets signed by our company, but anybody with the wherewithal can pull the jar apart and mod the application however they wish. One of the partner companies has already done so, extensively, and come back to show us their work. Management at my company is impressed and asks me to add official plugin support to the application. Can you guess where this is going?
I add the plugin support,the application will now load custom jars that implement the plugin interface I had discussed with devs from that company that did the modding. They think it's great, management thinks its great, everything works and everybody is happy. At the last minute some security policy wonk throws on the brakes. Will this load any plugin jar? Yes. Not good! It needs to only load plugins approved by the company. Why? Because! Never mind that the whole damn application can be unofficially nodded with ease. I ask him how he wants that done, he says only load plugins signed by the company. Retarded, but fine. I do so. He approves it, then the partner company engineer who did the modding chimes in that he's just going to mod the signature check out, because he doesn't want to have to deal with this shit. Security asshat from my company has a melt down and long story short the entire plugin feature, which was already complete, gets scrapped and the partner company just keeps modding the application as before. Months of my life down the drain. Thanks guys, great job protecting... something.
So why are these people not involved from the first place? Seems like a huge management/executive failure that the right people who needs to check off the design weren't involved until after developers implemented the feature.
You seem to blame the person who is trying to save the company from security issues, rather than placing the blame on your boss that made you do work that would never gotten approved in the first place if they just checked with the right person first?
Because they don't respond to their emails until months after they were nominally brought into the loop. They sit back jerking their dicks all day, voicing no complaints and giving no feedback until the thing is actually done.
Yes, management was ultimately at fault. They're at fault for not tard wrangling the security guys into doing their jobs up front. They're also at fault for not tard wrangling the security guys when they object to an inherently modifiable application being modified.
6 replies →
The bikeshedding is coming from in the room. The point is that the feature didn't cause any regression in capability. And who tf wants a plugin system with only support for first party plugins?
1 reply →
> he's just going to mod the signature check out, because he doesn't want to have to deal with this shit
Fine. The compliance catastrophe will be his company's not yours'.
> I'm actively trying to find a way we can unblock innovators to move quickly at scale
So did "Move fast and break things" not work out? /i
The main problem with many IT and security people at many tech companies is that they communicate in a way that betrays their belief that they are superior to their colleagues.
"unlock innovators" is a very mild example; perhaps you shouldn't be a jailor in your metaphors?
I find it interesting that you latched on their jailor metaphor, but had nothing to say about their core goal: protecting my privacy.
I'm okay with the people in charge of building on top of my private information being jailed by very strict, mean sounding, actually-higher-than-you people whose only goal is protecting my information.
Quite frankly, if you changed any word of that, they'd probably be impotent and my data would be toast.
A bit crude, maybe a bit hurt and angry, but has some truth in it.
A few things help a lot (for BOTH sides - which is weird to say as the two sides should be US vs Threat Actors, but anyway):
1. Detach your identity from your ideas or work. You're not your work. An idea is just a passerby thought that you grabbed out of thin air, you can let it go the same way you grabbed it.
2. Always look for opportunities to create a dialogue. Learn from anyone and anything. Elevate everyone around you.
3. Instead of constantly looking for reasons why you're right, go with "why am I wrong?", It breaks tunnel vision faster than anything else.
Asking questions isn't an attack. Criticizing a design or implementation isn't criticizing you.
Thank you,
One of the "security people".