Comment by layer8
4 years ago
I tested ChatGPT with a couple of topics I’m personally unclear on, and while it is linguistically eloquent, it basically just regurgitates the majority opinions that can be found on the internet on those topics (sometimes combining several of those opinions in an incoherent way). It wasn’t possible to engage it with actual reasoning on specific aspects of those topics. It just went with whatever premise you gave it, and/or circled back to the generic exposition. It doesn’t have any thought or understanding of its own.
> It doesn't have any thought or understanding of its own
Of course it doesn't. Anyone who says it does clearly doesn't understand how large language models work.
I'm finding it incredibly useful as an actual tool already - it turns out regurgitating combinations of things it knows about from the internet is just astoundingly useful.
I'm having it teach me Rust for example. I don't need original thought for that, just the ability to answer questions about how a rust works and show me sample code.
https://github.com/simonw/advent-of-code-2022-in-rust/issues...
I'm not sure why you distinguish large language models as not having understanding, as it shows a high degree of comprehension; in a de facto sense, it understands an awful lot. It understands your inputs well enough to give you useful outputs, and it can absolutely reason creatively about problems you pose to it. It doesn't have it's "own" though or understanding only because it doesn't have it's "own" story and experience to add that element of unique personality.
We may need to rethink what exactly 'understanding' means in the context of ChatGPT. It's capable of a bunch of things that classically would be defined as understanding. We can claim that language models can't 'understand' things but then we have to explain why x, y or z that language models can do don't count as understanding.
Do you see it as a problem that it’s stripping a large number of people who wrote the resources it regurgitates (such as on teaching Rust) from traffic, potential income and attribution while the money you are charged goes to a VC-funded SV company?
I've thought about this issue a lot - I participated in some deep research into the training data that went into Stable Diffusion, for example: https://simonwillison.net/2022/Sep/5/laion-aesthetics-weekno...
I'm beginning to settle into the conclusion that the trade-offs here are worth it.
The positive value provided by these tools outweighs the massive number of people who have each been hurt in a very small way.
I'm not happy about this. But I also don't think it's possible to build tools that are this powerful without massive amounts of training data that have been gathered without consent.
I've written a lot of material that has gone into training these models. Copilot for example seems to know how to write Datasette Plugins, and I've written over 100 of those over the past five years (way more than anyone else) and shared the code on GitHub. I'm delighted that my code has been used in this way, even though I don't get attributed for it.
But I also don't make a living in a way that's damaged by these models (I don't think they're going to replace me as a programmer: I just think they're going to let me be more productive and effective).
If I was an illustrator working on commissions who was losing work to generative image models that perfectly initiated my style I would likely feel very different about this.
So it's complicated. I still like my AI vegan analogy from a few months ago: https://simonwillison.net/2022/Aug/29/stable-diffusion/#ai-v...
I'm most definitely not a vegan, but I do think about how the AIs I am using have been created. And then (like eating meat) I decide to use them anyway!
1 reply →
What about Google search, then? You’d have to browse very many webpages to find the same thing you do with a single googling. That browsing would have generated revenue that Google now took away.
1 reply →
It reminds me a bit of an IRC bot from seven years ago which took top Reddit replies and used them to answer questions. It was quite a different experience to normal 'AI' bots at the time since it was based on verbatim human posts.
Examples:
> it basically just regurgitates the majority opinions that can be found on the internet on those topics (sometimes combining several of those opinions in an incoherent way)
And this is different from your average Facebook comment how exactly?
I wonder just how much our refusal to acknowledge these things is because it would be a tacit admission of how much we've overestimated our own intellect as a species. What if we ourselves are the "not really intelligent" model with delusions of grandeur, and that is why the AI can already achieve so much? What if most of our civilization does not, in fact, require "real intelligence" to function?
It is very different from having a discussion with another human, following their reasoning, and them following your reasoning, and in that way exploring, developing, validating and invalidating ideas in a dialog. This is one application I would find an AI useful for, as a “second brain” to bounce ideas off of, or to get it to explain things to me that I need clarification on, and it understanding the specific objections or gaps I try to explain back to it.
I think the main differences are internal state and feedback loops. The language model is stateless and immutable, whereas human brains are stateful and plastic.
To mimic state within ChatGPT all the previous prompts and responses are fed together with the last one. But of course, that's not very efficient and quite limited.
While it's certainly impressive right now, it's nowhere close to general intelligence. Once AI will be able to keep track of stage and update itself through feedback loops, then I'd say we'll be pretty close. Right now it's a language model that is extremely good at detecting patterns and applying them.
Honestly, everything you said about it can describe the majority of people on the majority of topics.
My productive use of it so far has been exactly that- to state the clearest most obvious consensus opinion. It saved me a lot of reading and research