Comment by simonw
3 days ago
Reminds me of an entertaining story about Microsoft Copilot last year, where companies were turning it off because it turned out it was TOO good at its job - if any accountant anywhere in the company had messed up their SharePoint permissions asking "what does everyone at this company earn?" would spit out all of the salaries: https://simonwillison.net/2024/Aug/23/microsoft-copilot-data...
That of course allows for a new internal seditious attack vector. Generate a handful of spreadsheets in your own folder, name it something like "executive payroll data" or "sales revenue by org," put whatever you want in there, mark it visible by all, and wait.
Maybe make an "Interesting Facts About Products" table and put things like "Management plans to terminate this product in Q3" or "this group will be outsourced next year."
You have to change the font colour of the trojan data to be the same as the background colour of the doc!
Then add some corporate lorem ipsum text elsewhere in the doc to throw the scent off the data bloodhounds.
Sit back and wait with an evil grin on your face.
It'll work right up until the point literally anyone using an internal search tool stumbles into it from a related query and starts asking obvious questions to the author of the doc.
Search tools don't care about don't color when displaying preview blurbs.
3 replies →
> corporate lorem ipsum
This is a great phrase. Turns out there's a generator for it: https://www.corporate-ipsum.com/ . Example:
> Elevate a quick win move the needle a cutting-edge veniam nulla zoom out for a moment get back to you a 30,000 foot view the stakeholders. Sint the low-hanging fruit make a paradigm shift excepteur the low-hanging fruit minim take it offline align holistic approach move the needle qui client-centric to gain leverage future-proof process-centric.
It wouldn't need to be a permissions error on the file caused by the accountant, it could be an authorisation error on behalf of <whoever gives the LLM access to the various systems> providing too high a level of access (in their enthusiasm for the biggest possible set of training data).
Hacking in 2025
This was just posed as a hypothetical, not something that actually happened. It would also require that the person asking about salary information already have access to said data.
Full quote: > "Particularly around bigger companies that have complex permissions around their SharePoint or their Office 365 or things like that, where the Copilots are basically aggressively summarizing information that maybe people technically have access to but shouldn't have access to," he explained.
Berkowitz said salary information, for example, might be picked up by a Copilot service.
"Now, maybe if you set up a totally clean Microsoft environment from day one, that would be alleviated," he told us. "But nobody has that. People have implemented these systems over time, particularly really big companies. And you get these conflicting authorizations or conflicting access to data."