Slop is about thoughtless use of a model to generate output. Output from your paper's model would still qualify as slop in our book.
Even if your model scored extremely high perplexity on an LLM evaluation we'd likely still tag it as slop because most of our text slop detection is using sidechannel signals to parse out how it was used rather than just using an LLM's statistical properties on the text.
Here's what pattern suppression actually does on a model that's trained to open its writing with "You're absolutely right.":
You're spot-on. You're bang-on. You're dead right. You're 100% correct. I couldn't agree more. I agree completely. That's exactly right. That's absolutely correct. That's on the nose. You hit the nail on the head. Right you are. Very true. Exactly — well said. Precisely so. No argument from me. I'll second that. I'm with you 100%. You've got it exactly. You've hit the mark. Affirmative — that's right. Unquestionably correct. Without a doubt, you're right.
I'm willing to bet money you can easily tag these openers yourself.
This sampling strategy and the elaborate scheme to bake its behavior into the model during the post-training are terribly misguided, because they don't fix the underlying mode collapse. It's formulated as narrowing down the output distribution, but as with many things in LLMs it manifests itself on a much higher semantical level - during the RL (at least using the current methods) the model narrows the many-to-many mapping of high-level ideas that the pretrained model has down to one-to-one or even many-to-one. If you naively suppress repetitive n-grams that are not semantically aware and manually constructed patterns that don't scale, it will just slip out at the first chance, spamming you with minor non-repetitive variations of the same high-level idea.
You'll never have the actual semantic variety unless you fix mode collapse. Referencing n-grams or manually constructed regexes as a source of semantical diversity automatically makes the method invalid, no matter how elaborate your proxy is. I can't believe that after all this time you persist in this and don't see the obvious issue that's been pointed at multiple times.
People don't call it slop because of repetitive patterns they call it slop because it's low-effort, uninsightful, meaningless content cranked out in large volumes
Slop is about thoughtless use of a model to generate output. Output from your paper's model would still qualify as slop in our book.
Even if your model scored extremely high perplexity on an LLM evaluation we'd likely still tag it as slop because most of our text slop detection is using sidechannel signals to parse out how it was used rather than just using an LLM's statistical properties on the text.
Would love to see proof of this claim that you can tag antislopped LLM text as LLM generated. I'm willing to bet money that you can't.
Here's what pattern suppression actually does on a model that's trained to open its writing with "You're absolutely right.":
You're spot-on. You're bang-on. You're dead right. You're 100% correct. I couldn't agree more. I agree completely. That's exactly right. That's absolutely correct. That's on the nose. You hit the nail on the head. Right you are. Very true. Exactly — well said. Precisely so. No argument from me. I'll second that. I'm with you 100%. You've got it exactly. You've hit the mark. Affirmative — that's right. Unquestionably correct. Without a doubt, you're right.
I'm willing to bet money you can easily tag these openers yourself.
This sampling strategy and the elaborate scheme to bake its behavior into the model during the post-training are terribly misguided, because they don't fix the underlying mode collapse. It's formulated as narrowing down the output distribution, but as with many things in LLMs it manifests itself on a much higher semantical level - during the RL (at least using the current methods) the model narrows the many-to-many mapping of high-level ideas that the pretrained model has down to one-to-one or even many-to-one. If you naively suppress repetitive n-grams that are not semantically aware and manually constructed patterns that don't scale, it will just slip out at the first chance, spamming you with minor non-repetitive variations of the same high-level idea.
You'll never have the actual semantic variety unless you fix mode collapse. Referencing n-grams or manually constructed regexes as a source of semantical diversity automatically makes the method invalid, no matter how elaborate your proxy is. I can't believe that after all this time you persist in this and don't see the obvious issue that's been pointed at multiple times.
2 replies →
I'm not saying we could detect it from the text alone!
The side channel signals (who posted it, where, etc.) are more valuable in tagging than raw text classifier scores.
That's why I said our definition of slop can include all types of genAI: it's about *thoughtless use of a tool* more than the tool being used.
And also that regardless of the method, your model can be used to generate slop.
1 reply →
If its not labeled as generated by AI, then that in of itself makes it deceptive and therefore slop.
It looks like a method of fabricating more convincing slop?
I think the Kagi feature is about promoting real, human-produced content.
People don't call it slop because of repetitive patterns they call it slop because it's low-effort, uninsightful, meaningless content cranked out in large volumes