Comment by nyrikki
2 years ago
Cheese and Chalk.
It is very different to generate synthetic datasets to assist in targeted training , vs ingesting LLM output from web scraping.
2 years ago
Cheese and Chalk.
It is very different to generate synthetic datasets to assist in targeted training , vs ingesting LLM output from web scraping.
Yes -- said another way, if you're an ML researcher and you have human-provided (scraped) data, and an ability to generate synthetic data, then until recently, you had a controllable parameter: how much of your training data for your new model should be synthetic? You can vary this, run multiple experiments, and choose how much synthetic data to use -- and you can vary the specific configs about how that synthetic data is generated.
If synthetic data is mixed into your upstream data sources in a way you cannot control, then your ML team loses a valuable controllable parameter.
You still have some that control, but in a much more indirect way.
There are three kinds of data now, synthetic, pre-2022 and current. Everything pre-2022 is definitely written by humans, synthetic data is still synthetic, and post-2022 is a mix of both.
I wouldn't be surprised if "AI detectors" work somewhat for this use case. They're biased, far from accurate and a terrible idea if you need to make important decisions (like whether to expel a student for cheating), but there's quite a large room for errors here.
> Everything pre-2022 is definitely written by humans
I'm not sure if methods like article spinning counts as written by humans. This is something you could automate before AI and it would take a human written article and randomly swap words with similar meaning throughout to make it seem original.
1 reply →
They do work detecting LLM outputs that are sampled "naively" (when the model/user is really not trying to pass it as human output).
I copied a prompt translated from spanish to english using ChatGPT Plus in a GPT-4o Azure OpenAI Service endpoint. It did work in Spanish but didn't run in english because the default AOS Content Filters detected a jailbreak intent. It was quite weird.
Yeah, I raised the same issue before reading your post; ninja'd I am.
I like your "cheese and chalk".
I always preferred sugar and shit. Obviously, that is profane. But, I consider profanity be seen as the part of speech it really is.
Profane is, if you will, fucking fine.
But "cheese and chalk" is a great analogy because both are sources of calcium, but cheese is much better for the human body. It carries useful info.
Also drives your point home more efficiently. While it may be profane, there's far more speech available with far less "use" that is intentionally profane to spark a reaction without regard to what that reaction may be. Shock value for attention, rather than to carry home a point.
I think this is it.
Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in.
Basically still needs a human in the loop.
> Basically still needs a human in the loop.
Yes, and big LLM developers have millions of humans in the loop. That's why they provide free access, for human in the loop filtering & guidance.
If I go to chatGPT and solve a coding task, maybe the first 3 ideas don't work and the 4th works. It can do RLHF setting the first 3 with negative and the fourth with positive score. They just used me to test their model and create a datapoint.
Using LLM is useful both ways - for humans, we get assistance, and LLMs get feedback for their outputs. This seems like the new form of "you are the product".
Then why not remove this crap (LLMs) from the loop altogether? How did we get from "AI will replace you" to "your new job will be an AIs janitor" in the space of about 12 months?
there is nothing wrong with being a janitor. you could also call it "AI editor" instead of you want to insert a job title sounds more prestigious. some people find it easier and more enjoyable to edit a first draft generated by a language model based on instructions than writing that first draft themselves.
3 replies →
>Then why not remove this crap (LLMs) from the loop altogether
Because reading is faster than writing.
Someone could spend a few years or even most of their life writing a book that can be read in a matter of hours days or weeks.
Humans writing have to proofread their own work. Or occasionally even pay someone else to do it.
No, bad/wrong/nonsense is not the only risk here. You're missing the main point that the authors are making: the shape of the distribution gets changed by this process. A model trained on human data will produce fewer high-perplexity examples than it was trained on (you can see this in Fig 1b, even between generation 0 and 1). In a literal information theory sense, these perplexity values indicate how much information is in each example. Over successive generations models have less actual information to learn from even if they have the same volume of text.
LLMs are milking us of knowledge and skills, repackage them and give it back to us. Models interact with the internet, humans and code execution. They are exploring. Lots of exploring now happens in the chat room, a place where ideas are first tried out. With billions of users, the volume of information LLMs collect from us is huge. We bring references, guidance and feedback right into its mouth, the LLM doesn't even need to do anything like crawling.
Imagine how many things we know, things we accumulated in our life experience, that were never written down anywhere. That information was lost to others. But now we use LLM assistants, so they get to be in the loop and collect tidbits of human life experience that is not written on the internet. And soon they will also work on audio/video and travel with us everywhere, seeing what we show them.
I think that maybe we are too harsh in expecting LLMs to be perfect. If they are based off of human input that is incorrect then we might propagate such errors. But they will still be quicker and much more reliable than most people. Isn’t this good enough? After all, we are willing to accept flaws in people, even including the president. I suspect that the way forward will be to progressively clean the LLM input data as each error gets identified.