Comment by joshuamorton
1 month ago
I don't think that's what it's saying at all. It's saying that there's a level of creativity in model creation that isn't present in distillation.
1 month ago
I don't think that's what it's saying at all. It's saying that there's a level of creativity in model creation that isn't present in distillation.
Maybe, but it's not like their AI is likely to repeat it back verbatim so it's unlikely to be a copyright violation. It seems like at most, they would be breaking Anthropic's terms of service?
Or maybe they're going through an intermediary "transfer station" that's breaking terms of service:
https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...
Yes, it's just a ToS violation at present. Those are legally binding though, despite the common adage. What that really translates to here though, anyone's guess.
Anthropic's own copyright infringement could apparently be forgiven for 1.5B USD after all, so maybe there's a price that breaking the distillation clause for is acceptable too. Or some other arrangement.
But surely at least one of the websites Anthropic scraped to make Claude had a ToS forbidding automatic access?
Why is Anthropic's ToS any more binding than that of a rabidly-anti-ai literature blog with 50 readers?
3 replies →
>a level of creativity in model creation that isn't present in distillation.
the same argument - a level of creativity in the world knowledge creation that ins't present in the model training on that knowledge.
Or in other words - model creation and training is just a distilling of the world knowledge.
I don't disagree. I'm not sure why that's a relevant reply though.
If you think that the addition of a less creative process (model creation) to a more creative corpus ("art") is problematic, then it follows that you should think the addition of a less creative process (distillation) to a more creative corpus (a model) is also problematic.
I think both are natural and fine. Otherwise we'd have to outlaw analytical thinking.
Yes, this is what I was getting at.
There is an even higher level of creativity in creating books, songs and all sorts of art used in model training though. That's your apparent blindspot.
There is no world in which me vacuuming the entirety of human knowledge to make a genai model is ok but hoovering my model answers is not. The hypocrisy is stunning and risible.
Now if you go and make a model based on purely synthetic data and not a single work made by humans, you would have a valid point.
> There is an even higher level of creativity in creating books, songs and all sorts of art used in model training though.
No argument here, I completely agree.
> There is no world in which me vacuuming the entirety of human knowledge to make a genai model is ok but hoovering my model answers is not.
I disagree with this though. Clearly LLMs owe a huge debt to everything that has come before, but surely you'd agree that the models that are produced are something substantial and new and novel which didn't exist before and have lots of value in their own right. Let's be a bit reductive and pretend Moonshot had just outright stolen the weights from Fable somehow, clearly that wouldn't be contributing anything really new or novel. Now of course they've distilled rather than stolen, but the point is similar: how much value have they added along the way?
1 reply →
So, I'm not the person you were responding to. I'd like you to take a moment and suggest where anyone in the thread you're replying to, either me or Remus, has said anything that suggests disagreement with the statement
> There is an even higher level of creativity in creating books, songs and all sorts of art used in model training though.
He claimed there was more creativity in model training than in model distillation. That makes no claim about the relationship between the creativity in model creation and art. Why are you continuing to attack a claim that was never made, after a sub thread very explicitly clarifying that that claim was not made?
1 reply →