Comment by jacquesm
3 years ago
> But that's a ways off.
Given the jumps in output quality between '1', '2' and '3' that may not be as far off as I would like it to be.
It reminds me of the progression of computer chess. From 'nice toy' to 'beats the worlds best human' since 1949 to 'Man vs Machine World Team Championships' in 2004 is 55 years, but from Sargon (1978) to Deep Blue (1997) is only 21 years. For years we thought there was something unique about Chess (and Go for that matter) that made the game at the core a human domain thing, but those that were following this more closely saw that the progression would eventually lead to a point where the bulk of the players could no longer win from programs running on off the shelf hardware.
GPT-3 is at a point where you could probably place it's output somewhere on the scale of human intellect depending on the quality of the prompt engineering and the subject matter. Sometimes it produces utter garbage but already often enough it produces stuff that isn't all that far off from what a human might plausibly write. The fact that we are having this discussion is proof of that, given a few more years and iterations 4, 5 and 6 the relevant question is whether we are months, years or decades away from that point.
The kind of impact that this will have on labor markets the world over is seriously underestimated, and even though GPT-3's authors have side-stepped a thorny issue by simply not feeding it information on current affairs in the training corpus if Chess development is any guide the fact that you need a huge computer to train the model today is likely going to be moot at some point, when anybody can train their own LLM. Then the weaponization of this tech will begin for real.
Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express, which places limits on how well it can argue something.
It is of course possible that it might (eventually) be convincing enough that no human can tell, which would be problematic because it would suggest human speech is indistinguishable from a knee jerk response that doesn't require that you communicate any useful information.
Things would be quite different if an AI could interpret new information and form opinions, but even if GPT could be extended to do so, right now it doesn't seem to have the capability to form opinions or ingest new information (beyond a limited short term memory that it can use to have a coherent conversation).
But the bar really isn't 'no human can tell' the bar is 'the bulk of the humans can't tell'.
> Things would be quite different if an AI could interpret new information and form opinions, but even if GPT could be extended to do so, right now it doesn't seem to have the capability to form opinions or ingest new information (beyond a limited short term memory that it can use to have a coherent conversation).
Forming opinions is just another mode of text transformations, ingesting new information is either a conscious decision to not let the genie out of the bottle just yet or a performance limitation, neither of those should be seen as cast in stone, the one is a matter of making the model incremental (which should already be possible), the other merely a matter of time.
None of this matters. The reason comments are valuable is that they are a useful source of information. Part of the transaction cost of deciding whether a comment is useful is how much additional work is required to evaluate it.
Comments are ascribed credibility based on the trust the reader has in the commenting entity, whether the comment is consistent with the reader's priors and researching citations made in the comment, either explicit or implicit.
Since GPT can confidently produce comments which are wrong, there is no trust in it as a commenting entity. Consequently everything it produces needs to be further vetted. It's as if every comment was a bunch of links to relevant, but not necessarily correct sources. Maybe it produces some novelty which leads to something worthwhile, but the cost is high, until it can be trusted. Which is not now.
If a trusted commenter submits a comment by GPT, then he is vouching for it and it is riding on his reputation. If it is wrong, his reputation suffers, and trust in that commenter drops just as it would regardless of the genesis of the comment.
A true AI will not have one opinion. It will realize there are many truths - one persons truth is really a perspective based on their inputs which are different than another's. Change the inputs and you’ll often get a different output.
ChatGPT further proves this notion - you can ask it to prove/disprove the same point and it will do so quite convincingly both times.
9 replies →
You are arguing that a piece of software misses a metaphorical soul (something that cannot be measured but that humans uniquely have and nothing else does). That's an incredibly poor argument to make in a context where folks want interesting conversation. Religion (or religion-adjacent concepts such as this one) is a conversational nuke: It signals to anyone else that the conversation is over, as a discussion on religion cannot take forms that are fundamentally interesting. It's all opinion, shouted back and forth.
Edit: Because it is a prominent feature in the responses until now, I will clarify that there is an emphasis on "all" in "all opinion". As in, it is nothing but whatever someone believes with no foundation in anything measurable or observable.
I didn’t read it as being a religious take. They appear to be referring more to embodiment (edit: alternatively, online/continual learning) which these models do not posses. When we start persisting recurrent states beyond the current session we might be able to consider that limited embodiment. Even still the models will have no direct experience interacting with the subjects of their conservations. Its all second hand from the training data.
2 replies →
I find it ironic that you are expressing a strong opinion that opinions do not make good conversation. Philosophy is the highest form of interesting conversation, and it's right there with religion (possibly politics, too).
1 reply →
It doesn't have to have a metaphorical (or metaphysical or w/e) soul, but at this point it does not have it's own 'opinion'. It will happily argue either way with only a light push, it talks because it is ordered to, not because it is trying to communicate information. This severely limits the kind of things it can do.
I would argue (of course not seriously) about the opposite: ChatGPT has a metaphorical soul. What it learned very well is how to structure the responses so that they sound convincing - no matter how right or wrong they are. And that's dangerous.
Perhaps you have people around you who are not well suited to political, religious philosophical discussions or perhaps you don’t enjoy them / can’t entertain them.
Personally, I find the only interesting conversations technical or philosophical in nature. Just the other day, I was discussing with friends how ethics used to be a regular debated topic in society. Literally, every Sunday people would gather and discuss what it is to be a good human.
Today, we demonize one another, in large part because no one shares an ethical principal. No one can even discuss it and if they try, many people shut down the conversation (as you mentioned). In reality, it’s probably the only conversation worth having.
1 reply →
>it fundamentally lacks an internal point of view that it can express, which places limits on how well it can argue something.
Are you sure that the latter follows from the former? Seems to me that something free from attachment to a specific viewpoint or outcome is going to be a better logician than otherwise. This statement seems complacently hubristic to me.
I would argue that ChatGPT has opinions, and these opinions are based on it's training data. I don't think GPT has the type of reasoning skills needed to detect and resolve conflicts in its inputs, but it does hold opinions. It's a bit hard to tell because it can easily be swayed by a changing prompt, but it has opinions, it just doesn't hold strong ones.
The only thing stopping GPT from ingesting new information and forming opinions about it is that it is not being trained on new information (such as its own interactions).
"Sure it might produce convincing examples of human speech, but it fundamentally lacks an internal point of view that it can express..."
Sounds just like the chess experts from 30 years ago. Their belief at the time was that computers were good at tactical chess, but had no idea how to make a plan. And Go would be impossible for computers, due to the branching factor. Humans would always be better, because they could plan.
GPT (or a future successor) might not be able to have "an internal point of view". But it might not matter.
Having some internal point of view matters in as much that not having one means it's not really trying to communicate anything. A text generation AI would be a much more useful interface if it can form a view an express it rather than just figuring it all out from context.
You are correct in stating that current chat bots, such as GPT, do not have the ability to form opinions or interpret new information beyond a limited short term memory. This is a limitation of current technology, and as a result, chat bots are limited in their ability to engage in complex arguments or discussions. However, it is important to note that the development of AI technology is ongoing, and it is possible that future advances will allow for the development of more sophisticated AI systems that are capable of forming opinions and interpreting new information. Until that time, chat bots will continue to be limited in their abilities.
I am pretty sure this response was generated by a bot/GPT. As good as they are, you can tell what's GPT stuff and what isn't.
15 replies →
Ok, y'all passed the test! This was all open ai. Interesting times.
On the problem of distinguishing a bot from a human, I suggest the following podcast episode from Cautionary Tales [1]. I found it both enjoyable and interesting, as it shows an interesting point of view about the matter: if we already had bots that passed as humans long ago, is because we are often bad at conversations, not necessarily because the bot is extremely good at it (and indeed in most cases it isn't).
[1] https://podcasts.google.com/feed/aHR0cHM6Ly93d3cub21ueWNvbnR...
What I fear the most is that we‘ll keep at this “fake it till you make it” approach and skip the philosophical questions, such as what conscience really is.
We’re are probably at the verge of having a bot that reports as conscious and convinces everyone that it is so. We’ll then never know how it got there, if really did or if just pretends so well that it doesn’t matter, etc.
If feels like it’s out last chance as a culture of tackling that question. When you can pragmatically achieve something, the “how” loses a bit of its appeal. We may not completely understand fluid dynamics, but if it flys, it flys.
The answer may well be 'consciousness is the ability to fake having consciousness well enough that another conscious being can't tell the difference' (which is the essence of the Turing test). Because if you're looking for a mechanism of consciousness you'd be hard put to pinpoint it in the 8 billion or so brains at your disposal for that purpose, no matter how many of them you open up. They'll all look like so much grisly matter from a biological point of view and like a very large neural net from a computational one. But you can't say 'this is where it is located and that is how it works'. Only some vague approximations.
Sure, and that’s what I’m trying to say. Is being conscience just fooling yourself and others really well or is there some new property that eventually emerges from large enough neural networks and sensory inputs? The philosophical zombie is one the most important existencial questions that we may be at the cusp of ignoring.
9 replies →
> what conscience really is
My favorite line from Westworld - "if you cannot tell the difference, does it really matter?"
> on the scale of human intellect
Where is the module that produces approximations to true and subtle insights about matters? Where is the "critical thinking" plugin, how is it vetted?
How do you value intelligence: on the form, or on the content? Take two Authors: how do you decide which one is more intelligent?
> the progression of computer chess
?! Those are solvers superseded by different, more effective solvers with a specific goal... These products in context supersede "Eliza"!
Well, for starters we could take your comment and compare it to GPT-3 output to see which one makes more sense.
https://news.ycombinator.com/item?id=33651960
1 reply →
> compare
Exactly. Which one "/seems/ to make sense" and which one has the "juice".
Also: are you insinuating anything? Do you believe your post is appropriate?
Edit: but very clearly you misunderstood my post: not only as you suggest with your (very avoidable) expression, but also in fact. Because my point implied that "a good intellectual proposal should not happen by chance": modules should be implemented for it. Even if S (for Simplicius) said something doubtful - which is found copiously even in our already "selected" pages -, and engine E constructed something which /reports/ some insight, that would be chancey, random, irrelevant - not the way we are supposed to build things.
8 replies →
You're looking at it from the perspective of "ChatGPT generating text that looks human."
dang is talking about "humans generating text which is 'better' than what ChatGPT can do."
Those are very different bars. Average output vs top output.
ChatGPT often generates text that a human might plausibly write. But is there text that a human could write that ChatGPT couldn't possibly write?
If ChatGPT is generating text by learning from the best of the human comments, then can an average human comment beat it?
> But is there text that a human could write that ChatGPT couldn't possibly write?
No, because ChatGPT is trained on text that humans wrote. Because what ChatGPT generates is based on what humans have wrote, it can always create the plausibility that a human might have created the text they are reading from it.