Comment by simonw
4 years ago
> It doesn't have any thought or understanding of its own
Of course it doesn't. Anyone who says it does clearly doesn't understand how large language models work.
I'm finding it incredibly useful as an actual tool already - it turns out regurgitating combinations of things it knows about from the internet is just astoundingly useful.
I'm having it teach me Rust for example. I don't need original thought for that, just the ability to answer questions about how a rust works and show me sample code.
https://github.com/simonw/advent-of-code-2022-in-rust/issues...
I'm not sure why you distinguish large language models as not having understanding, as it shows a high degree of comprehension; in a de facto sense, it understands an awful lot. It understands your inputs well enough to give you useful outputs, and it can absolutely reason creatively about problems you pose to it. It doesn't have it's "own" though or understanding only because it doesn't have it's "own" story and experience to add that element of unique personality.
We may need to rethink what exactly 'understanding' means in the context of ChatGPT. It's capable of a bunch of things that classically would be defined as understanding. We can claim that language models can't 'understand' things but then we have to explain why x, y or z that language models can do don't count as understanding.
Do you see it as a problem that it’s stripping a large number of people who wrote the resources it regurgitates (such as on teaching Rust) from traffic, potential income and attribution while the money you are charged goes to a VC-funded SV company?
I've thought about this issue a lot - I participated in some deep research into the training data that went into Stable Diffusion, for example: https://simonwillison.net/2022/Sep/5/laion-aesthetics-weekno...
I'm beginning to settle into the conclusion that the trade-offs here are worth it.
The positive value provided by these tools outweighs the massive number of people who have each been hurt in a very small way.
I'm not happy about this. But I also don't think it's possible to build tools that are this powerful without massive amounts of training data that have been gathered without consent.
I've written a lot of material that has gone into training these models. Copilot for example seems to know how to write Datasette Plugins, and I've written over 100 of those over the past five years (way more than anyone else) and shared the code on GitHub. I'm delighted that my code has been used in this way, even though I don't get attributed for it.
But I also don't make a living in a way that's damaged by these models (I don't think they're going to replace me as a programmer: I just think they're going to let me be more productive and effective).
If I was an illustrator working on commissions who was losing work to generative image models that perfectly initiated my style I would likely feel very different about this.
So it's complicated. I still like my AI vegan analogy from a few months ago: https://simonwillison.net/2022/Aug/29/stable-diffusion/#ai-v...
I'm most definitely not a vegan, but I do think about how the AIs I am using have been created. And then (like eating meat) I decide to use them anyway!
I’ve also thought about this issue a lot.
> in a very small way
Financially it is certainly a small way for me, as I made a conscious choice to not depend on content creation or OSS dev as primary revenue streams and (while I can’t say the same about future generations) I’m somewhat sure in my work I won’t be replaced by an OpenAI product within the years I have left.
However, I have gotten where I am in many respects thanks to countless people who shared information in an open way—and some of those people either made living from ads/donation buttons/upsells or enjoyed knowing that somebody appreciates the work they do, maybe even writes them about it, etc. All of those incentives depend on people visiting their web property, and all of those incentives are being taken away.
(It’s adjacent but slightly different than the plight of illustrators or writers some of whom justifiably feel as if they dug their jobs’ graves by doing what they love in the open. Having a few close friends in art space, I feel ashamed for the bait-and-switch that people in ‘my’ industry have served them.)
> I'm not happy about this. But I also don't think it's possible to build tools that are this powerful without massive amounts of training data that have been gathered without consent.
Attribution would solve part of the abovementioned issues, and intuitively it seems trivial—some extra layers that let the network reflect on where it might’ve gotten the inspiration for this or that wouldn’t require any true “understanding” by ML (which like you I consider an oxymoron), and surely the investment they raised or profits they make could handle it. Call me a cynic but I suspect they intentionally don’t make this a priority.
> I’m most definitely not a vegan
So far as meat-eating is legal, I’m not a fan of this analogy. While I know for a fact that many people would find this useful, “whether to respect IP or not” is just not up to personal choice—IMO the only factor that makes ChatGPT’s a slightly vague issue (as in “not definitely illegal but still ethically dubious”), like SD’s, is that IIRC the model is supposed to be open to all for private use.
What about Google search, then? You’d have to browse very many webpages to find the same thing you do with a single googling. That browsing would have generated revenue that Google now took away.
If Google did this, as in stopped sending users to content producers and just returned aggregated and processed content from all the pages in its index, they’d be met with robots.txt DENY entries and regulated out of existence in developed countries within days. They got burned once[0], so they know to tread carefully.
While we can (should?) be offended at the amount of featured snippets Google still shows today, it doesn’t seem comparable in scale.
[0] https://www.wired.co.uk/article/australia-media-code-faceboo...