Comment by gojomo
6 months ago
Look, we just need to add some new 'planes' to Unicode - that mirror all communicatively-useful characters, but with extra state bits for...
guaranteed human output - anyone who emits text in these ranges that was AI generated, rather than artisanally human-composed, goes straight to jail.
for human eyes only - anyone who lets any AI train on, or even consider, any text in these ranges goes straight to jail. Fnord, "that doesn't look like anything to me".
admittedly AI generated - all AI output must use these ranges as disclosure, or – you guessed it - those pretending otherwise go straight to jail.
Of course, all the ranges generate visually-indistinguishable homoglyphs, so it's a strictly-software-mediated quasi-covert channel for fair disclosure.
When you cut & paste text from various sources, the provenance comes with it via the subtle character encoding differences.
I am only (1 - epsilon) joking.
Just like with food: there will be a market value in content that is entirely “organic” (or in some languages “biological”). I.e. written, drawn, composed, edited, and curated by humans.
Just like with food: defining the boundaries of what’s allowed will be a nightmare, it will be impossible to prove content is organic, certifying it will be based entirely on networks of trust, it will be utterly contaminated by the thing it professes to be clean of, and it may even be demonstrably worse while still commanding a higher price point.
The entire world operates on trust of some form. Often people are acting in good faith. But regulation matters too.
If you don't go after offenders then you create a lemon markets. Most customers/people can't tell, so they operate on what they can. That doesn't mean they don't want the other things, it means they can't signal what they want. It is about available information, that's what causes lemon markets, information asymmetry.
It's also just a good thing to remember since we're in tech and most people aren't tech literate. Makes it hard to determine what "our customers" want
> If you don't go after offenders then you create a lemon markets.
Btw, private markets are perfectly capable of handling 'markets for lemons'. There might be good excuses for introducing regulation, but markets for lemons ain't.
As a little thought exercise, you can take two minutes and come up with some ways businesses can 'fix' markets for lemons and make a profit in the meantime. How many can you find? How many can you find already implemented somewhere?
12 replies →
I do wonder what would be an acceptable level of guarantee to trigger a “human written” bit.
I actually think a video of someone typing the content, along with the screen the content is appearing on, would be an acceptably high bar at this present moment. I don’t think it would be hard to fake, but I think it would very rarely be worth the cost of faking it.
I think this bar would be good for about 60 days, before someone trains a model that generates authentication videos for incredibly cheap and sells access to it.
Pen on paper, written without consulting any digital display. Just like exams used to be, before the pandemic.
Of course, the output will be no more valuable to the society at large than what a random student writes in their final exam.
1 reply →
Fully in agreement with you. There'll be ultimately two groups of consumers of "organic" content:
1. Those who just want to tick a checkbox will buy mass produced "organic" content. AI slop that had some woefully underpaid intern in a sweatshop add a bit of human touch.
2. People who don't care about virtue signalling but genuinely want good quality will use their network of trust to find and stick to specific creators. E.g. I'd go to the local farmer I trust and buy seasonal produce from them. I can have a friendly chat with them while shopping, they give me honest opinions on what to buy (e.g. this year was great for strawberries!). The stuff they sell on the farm does not have to go through the arcane processes and certifications to be labelled organic, but I've known the farmer for years, I know that they make an effort to minimize pesticide use, they treat their animals with care and respect and the stuff they sell on the farm is as fresh as it can be, and they don't get all their profits scalped by middlemen and huge grocery chains.
You're capturing nicely how the relationship with the farmer is an essential part of the "product" you buy when you buy high-end organic. I think that will continue to be true in culture/info markets.
Unicode has a range of Tag Characters, created for marking regions of text as coming from another language. These were deprecated for this purpose in favor of higher level marking (such as HTML tags), but the characters still exist.
They are special because they are invisible and sequences of them behave as a single character for cursor movement.
They mirror ASCII so you can encode arbitrary JSON or other data inside them. Quite suitable for marking LLM-generated spans, as long as you don’t mind annoying people with hidden data or deprecated usage.
https://en.m.wikipedia.org/wiki/Tags_(Unicode_block)
Can't I get around this by starting my text selection one character after the start of some AI-generated text and ending it one character before the end, Ctrl-C, Ctrl-V?
Yes, that’s correct. All of these measures, of course, stand as a courtesy and are trivial to bypass, as ema notes.
Finding cryptographic-strength measures to identify LLM-generated content is a few orders of magnitude harder than optimistically marking them. Besides, it also relies on the content producer adding those indicators so that can’t be ignored as a major source of missing metadata.
But sometimes lossy mechanisms are still helpful because people who aren’t out with malicious purposes might copy and paste without being aware that the content is generated, while an auditor (be it anyone who inspects one level deeper) can discover in some (most?) cases the source of the content.
There are many ways to get around this since it is trivial to write code that strips those tags.
> emits text in these ranges that was AI generated
How would you define AI generated? Consider a homework and the following scenarios:
1. Student writes everything themselves with pen & paper.
2. Student does some research with an online encyclopedia, proceeds to write with pen and paper. Unbeknownst to them, the online encyclopedia uses AI to answer their queries.
3. Student asks an AI to come up with the structure of the paper, its main points and the conclusion. Proceeds with pen and paper.
4. Student writes the paper themselves, runs the text through AI as a final step, to check for typos, grammar and some styling improvements.
5. Student asks the AI to write the paper for them.
The first one and the last one are obvious, but what about the others?
Edit, bonus:
6. Student writes multiple papers about different topics; later asks an AI to pick the best paper.
7. Student spent the entire high school and bachelor's degree learning from content that teachers generate using AI and using it to do homework, hence becoming AI-contaminated
This is about the characters themselves, therefore:
1. Not AI 2. Not AI 3. Not AI 4. The characters directly generated by AI are AI characters 5. AI 6. Not AI
The student dictates a paper word for word exactly
The student is missing arms and so dictates a paper word for word exactly
5 replies →
6 is extremely interesting, in that it's tantamount to asking a panel of innumerably many people to give an opinion on which paper is best for a general audience.
It's hard to imagine that NOT working unless it's implemented poorly.
Twelve millisecond after this law gets into effect, typing factories open in India, where human operators hand-recopy text from AI sources to perform "data laundering".
If somebody writes in a foreign language and asks Chat GPT to translate to English, is that AI generated content? What about if they write on paper and use an LLM to OCR? What if they give the AI a very detailed outline, constantly ask for rewrites and are ruthless in removing any facts they're not 100% sure of if they slip in? What if they only use AI to fix the grammar and rewrite bad English into a proper scientific tone?
My answer would be a clear "no" to all of these, even though the content ultimately ends up fully copy-pasted from an LLM in all those cases.
My answer is clear "yes" to most of those.
Yes, machine translations are AI-generated content - I read foreign-language news sites which sometimes has machine translation articles and the quality stands out and not in a good way.
"Maybe" for "writing on paper and using LLM for OCR". It's like automatic meeting transcript - if the speaker has perfect pronunciation, it works well. If they don't, then the meeting notes still look coherent but have little relationship to what speaker said and/or will miss critical parts. Sadly there is no way for reader to know that from reading the transcript, so I'd recommend labeling "AI edited" just in case.
Yes, even if "they give the AI a very detailed outline, constantly ask for rewrites, etc.." it's still AI generated. I am not sure how can you argue otherwise - it's not their words. Also, it's really easy to convince yourself that you are "ruthless in removing any facts they're not 100% sure" while actually you are anything but.
"What if they only use AI to fix the grammar and rewrite bad English into a proper scientific tone?" - I'd label it "AI-edited" if the rewrites are minor or "AI-generated" if the rewrites are major. This one is especially insidious as people may not expect rewrites to change meaning, so they won't inspect them too much, so it will be easier for hallucinations to slip in.
> they give the AI a very detailed outline […]
Honestly, I think that's a tough one.
(a) it "feels" like you are doing work. Without you the LLM would not even start. (b) it is very close to how texts are generated without LLMs. Be it in academia, with the PI guiding the process of grad students, or in industry, with managers asking for documentation. In both cases the superior takes (some) credit for the work that is in large parts by others.
1 reply →
It really depends on the context, e.g. if you need texts for a database of word frequencies, then the answer is a clear "yes", and LLMs have already ruined everything [1]. The only exception from your list would be OCR where a human proofreads the output.
[1] https://github.com/rspeer/wordfreq/blob/master/SUNSET.md
For the translate part let me just point out the offensively bad translations that reddit (sites with an additional ?tl=foo) and YouTube automatic dubbing force upon users.
These are immediately, negatively obvious as AI content.
For the other questions the consensus of many publications/journals has been to treat grammar/spellcheck just like non-AI but require that other uses have to be declared. So for most of your questions the answer is a firm "yes".
If the purpose is to identify text that can be used as training data, in some ways it makes sense to me to mark anything and everything that isn't hand-typed as AI generated.
Like for your last example: to me, the concept "proper scientific tone" exists because humans hand-typed/wrote in a certain way. If we use AI edited/transformed text to act as a source for what "proper scientific tone" looks like, we still could end up with an echo chamber where AI biases for certain words and phrases feed into training data for the next round.
Being strict about how we mark text could mean a world where 99% of text is marked as AI-touched and less than 1% is marked as human-originated. That's still plenty of text to train on, though such a split could also arguably introduce its own (measurable) biases...
> we still could end up with an echo chamber where AI biases for certain words and phrases feed into training data for the next round.
That’s how it works with humans too. “That sounds professional because it sounds like the professionals”.
All four of your examples are situations where an LLM has potential to contaminate the structure or content of the text, so in all four cases it is clear-cut that the output poses the same essential hazards to training or consumption as something produced "whole cloth" from a minimal prompt; post-hoc human supervision will at best reduce the severity of these risks.
OK, sure, there are gradations.
The new encoding can contain a FLOAT32 side channel on every character, to represent its proportional "AI-ness" – kinda like the 'alpha' transparency channel on pixels.
Yes yes yes yes
Stop ruining my simple and perfect ideas with nuance and complexity!
Nuance and complexity are a thing, but many of the GP's examples should be clearly AI labeled...
> What if they give the AI a very detailed outline, constantly ask for rewrites and are ruthless in removing any facts they're not 100% sure of if they slip in?
1 reply →
I’ll take the contrarian view. I don’t care if content is generated by a human or by an AI. I care about the quality of the content, and in many cases, the human does a better job currently.
I would like a search engine algorithm that penalizes low quality content. The ones we currently have do a piss poor job of that.
> I would like a search engine algorithm that penalizes low quality content. The ones we currently have do a piss poor job of that.
Without knowing the full dataset that got trimmed to the search result you see, how do you evaluate the effectiveness?
You’re asking a fair question but I think you’re approaching it from a POV that’s maybe a bit more of an engineering mindset than the person you’re responding to is using
A brilliant algorithm that filters out some huge amount of AI slop is still frustrating to the user if any highly ranked AI slop remains. You still click it, immediately notice what it is, and wonder why the algo couldn’t figure this out if you did so quickly
It’s like complaining to a waiter that there’s a fly in your soup, and the waiter can’t understand why you’re upset because there were many more flies in the soup before they brought it to the table and they managed to remove almost all of them
It doesn’t matter how much it filters out, if the top results are still spam.
I barely use Google anymore. Mostly just when I know the website I want, but not the URL.
Maybe have the glyph be zero width by default but have way to show them? I think begin-end markers would work better to make a whole range. It would need support from editor to manage the ranges and change editing AI generated text to mixed.
What might make sense is source marking. If you copy and paste text, it becomes a citation. AI source is always cited.
I havebeen thinking that there should be metadata in images for the provenance. Maybe a list of hashes of source images. Real cameras would include the raw sensor data. Again, AI image would be cited.
It would be much less disruptive to require that any network traffic containing AI generated content must have the IP evil bit set.
I have long thought that we should extend the plain text format to allow putting provenance metadata into substrings in the file.
This is that, but a different implementation. Plain text is like two conductor cables; it’s so useful and cost effective but the moment you add a single abstraction layer above it (a data pin) you can do so much more cool stuff.
That would be an evolution of HTML. Plain text is just plain text by definition, it can't include markup and annotations etc.
> for human eyes only - anyone who lets any AI train on, or even consider, any text in these ranges goes straight to jail. Fnord, "that doesn't look like anything to me".
Won't work because on day 0 someone will write a conversion library and apparently if you are big enough and have enough lawyers you can just ignore the jail threat (all popular LLMs just scrape internet and skip licensing any text or code. Show me one that isn't)
You'd probably want to distinguish between content being readable by AI and being trainable by AI.
E.g. you might be fine with the search tool in chatgpt being able to read/link to your content but not be fine with your content being used to improve the base model.
Each character should be, in effect, a signed git commit: in addition to a few bits for the Unicode code point itself, it should store a pointer back to the previous character's hash, plus a digital signature identifying the keyboard that typed it.
Rather than new planes, some sort of combining-character or even just an invisible signifying-mark would achieve the same purpose with far less encoding space. Obviously this would still be a nightmare for everyone who has to process text regardless.
Nope. Too easy to accidentally strip out. Each and every glyph must carry the taint.
We don’t want to send innocent people to jail! (Use UCS-18 for maximum benefit.)
But why? It's nice that somebody's collecting sources of pre-AI content that might be useful for curiosity or research or something. But other than that, why does it matter? AI text can still be perfectly good text. What's the psychological need behind this popular anti-AI ludditism?
You’re absolutely right that AI-generated text can be good—sometimes even great. But the reason people care about preserving or identifying pre-AI content isn’t always about hating AI. It's more about context and trust.
Think of it like knowing the origin of food. Factory-produced food can be nutritious, but some people want organic or local because it reflects a different process, value system, or authenticity. Similarly, pre-AI content often carries a sense of human intention, struggle, or cultural imprint that people feel connected to in a different way.
It’s not necessarily a “psychological need” rooted in fear—it can be about preserving human context in a world where that’s becoming harder to spot. For researchers, historians, or even just curious readers, knowing that something was created without AI helps them understand what it reflects: a human moment, not a machine-generated pattern.
It’s not always about quality—it’s about provenance.
Edit: For those that can't tell this is obviously just copy and pasted from chatgpt response.
I feel like the em-dashes and "You're absoultely right" already kinda serve the purpose of special AI-only glyphs
1 reply →
OK, so they can choose to read material from publishers that they trust to only produce human generated content. Similar to buying organic food. Pay a bit more for the feeling. No need for those idealists to drag everybody else into it.
2 replies →
Sounds like the plot of God Shaped Hole
Seems kind of excessive to send them to jail when the prisons are already pretty full. Might be more productive to do summary executions?
I understand that you're not completely serious about it, but you're proposing a very brittle technical solution for what is fundamentally a social and motivational issue.
The core flaw is that any such marker system is trivially easy to circumvent. Any user intending to pass AI content as their own would simply run the text through a basic script to normalize the character set. This isn't a high-level hack; it's a few dozen lines in Python and trivially easy to write for anyone who can follow a few basic Python tutorials or a 5-second task for ChatGPT or Claude.
Technical solutions to something like this exist in the analog world, of course, like the yellow dots on printers that encode date, time, and the printer's serial number. But, there is a fundamental difference: The user has no control over that enforcement mechanism. It's applied at a firmware/hardware layer that they can't access without significant modification. Encoding "human or AI" markers within the content itself means handing the enforcement mechanism directly to the people you're trying to constrain.
The real danger of such a system isn't even just that it's blatantly ineffective; it's that it creates a false sense of security. The absence of "AI-generated" markers would be incorrectly perceived as a guarantee for human origin. This is a far more dangerous state than even our current one, where a healthy level of skepticism is required for all content.
It reminds me of my own methods of circumventing plagiarism checkers back in school. I'm a native German speaker, and instead of copying from German sources for my homework, I would find an English source on the topic, translate it myself, and rewrite it. The core ideas were not my own, but because the text passed through an abstraction layer (my manual translation), it had no direct signature for the checkers to match. (And in case any of my teachers from back then read this: Obviously I didn't cheat in your class, promise.)
Stripping special Unicode characters is an even simpler version of the same principle. The people this system is meant to catch - those aiming to cheat, deceive, or manipulate - are precisely the ones who will bypass it effortlessly. Apart from the most lazy and hapless, of course. But we are already catching those constantly from being dumb enough to include their LLM prompts, or "Sure, I'll do that for you." when copying and pasting. But if you ask me, those people are not the ones we should be worried about.
//edit:
I'm sure there are way smarter people than me thinking about this problem, but I genuinely don't see any way to solve this problem with technology that isn't easily circumvented or extremely brittle.
The most promising would likely be something like unperceivable patterns in the content itself, somehow. Like hiding patterns in the length of words used, length of sentences, punctuation, starting letters for sentences, etc. But even if the big players in AI were to implement something like this immediately, it would be completely moot.
Local open-source models that can be run on consumer hardware already are more than capable enough to re-phrase input text without altering the meaning, and likely wouldn't contain these patterns. Manual editing breaks stylometric patterns trivially - swap synonyms, adjust sentence lengths, restructure paragraphs. You could even attack longer texts piecemeal by having different models rephrase different paragraphs (or sentences), breaking the overall pattern. And if all else fails, there's always my manual approach from high school.