Comment by miki123211
2 years ago
You still have some that control, but in a much more indirect way.
There are three kinds of data now, synthetic, pre-2022 and current. Everything pre-2022 is definitely written by humans, synthetic data is still synthetic, and post-2022 is a mix of both.
I wouldn't be surprised if "AI detectors" work somewhat for this use case. They're biased, far from accurate and a terrible idea if you need to make important decisions (like whether to expel a student for cheating), but there's quite a large room for errors here.
> Everything pre-2022 is definitely written by humans
I'm not sure if methods like article spinning counts as written by humans. This is something you could automate before AI and it would take a human written article and randomly swap words with similar meaning throughout to make it seem original.
Don’t forget machine-translated texts, where until ~2017 the translation was likely done by something much dumber / semantically lossy than an LLM, and after 2017 was basically done by an early form of LLM (the Transformers architecture originating in Google Translate.)
Many historical English-language news reports published on the English-language websites of foreign news media from non-English-speaking countries, from 1998 (Babelfish era) to ~a few months ago, may be unreliable training data for this reason.
They do work detecting LLM outputs that are sampled "naively" (when the model/user is really not trying to pass it as human output).
I copied a prompt translated from spanish to english using ChatGPT Plus in a GPT-4o Azure OpenAI Service endpoint. It did work in Spanish but didn't run in english because the default AOS Content Filters detected a jailbreak intent. It was quite weird.