Comment by mlissner
2 months ago
No, we worked with researchers that developed that kind of system, but didn't broadcast our work b/c the research was too sensitive. Seems the cat is out the bag now though.
I think the combination of AI and font-metrics is going to be wild though. You ought to be able to make a system that can figure out likely words based on the unredacted ones and the redaction's size. I haven't seen any redaction system yet that protects against this.
I thought glyph spacing attacks are an old idea; like I recall reading about such ideas 10-20 years ago unless I’m misremembering. Can you clarify why it was considered “too sensitive” if the whole point of this effort is to showcase these attacks?
It’s a fine line. Most redactions are for the good, to protect someone or something. For example even in the Epstein files, where some redactions are being abused, most redactions are protecting victims.
If there’s a way to undo huge amounts of redactions, that’d certainly be a net negative. Sort of like if encryption were suddenly broken, you wouldn’t publish a paper saying so.
Our goal has always been to educate about the problem so that it can be addressed. We didn’t have resources to push on the font metrics approach, so we stayed mostly quiet about it.
> If there’s a way to undo huge amounts of redactions, that’d certainly be a net negative. Sort of like if encryption were suddenly broken, you wouldn’t publish a paper saying so.
I can't state emphatically enough how this is not the right mental playbook.
If you have found a vulnerability, it's likely someone else has too. By sitting on it, you only create more future victims.
Disclosure will lead to fixing this issue, mitigating it's precense, or switching tools/workflows, possibly a combination of. Sitting on it only ensures that folks who think they are protected, actually aren't.
3 replies →
Given that hiding among and behind victims is how abusers continue, I’m not so sure redactions really are all that beneficial when you count future victims in the pool of interested parties. And the public interest certainly isn’t helped by secrecy and redactions and selective release.
While protecting victims is noble, something like this really needs the light of day and a truth and reconciliation commission so that everyone associated with the crime ring is punished and accounted for.
And no, if you do find somehow all encryption is mathematically broken, it’s your duty to publicize it even if existing secrets are jeopardized (you mitigate as best you can obviously in the short term) because it’s likely people more powerful than you might have that knowledge anyway and are engaged in asymmetric warfare.
> I haven't seen any redaction system yet that protects against this.
The linked article suggests widening redacted areas more than needed with some randomization applied to the width. Strikes me that that wouldn't do much except add a few more possible solutions.
Yeah, the more robust protection is to widen to a constant. But in the general case that could require reflowing the pdf. But honestly single word redactions are really probably useless with cheap AI that can highly accurately fill in the gaps
Depends what you're trying to hide.
If the redaction is a person's name, and there's nothing else to give the person's identity away, single word redaction probably works reasonably well, AI or no AI.
4 replies →
This is going to be a disaster IMO because AI will just hallucinate what it thinks is the most probable redacted word and people will take that as gospel.
"don't redact or we will hallucinate something worse and make people believe it as gospel" is nice deterrent
We don’t need a “deterrent” against things being redacted in publicly released documents. We can have transparency without the whole world finding out the names of victims and witnesses, people’s phone numbers and SSNs, etc., every time a document is released.
Maybe we should all just use mono-space fonts for everything