We all know how these things are built and trained. They estimate joint probability distributions of token sequences. That's it. They're not more "conscious" than the simplest of Naive Bayes email spam filters, which are also generative estimators of token sequence joint probability distributions, and I guarantee you those spam filters are subjected to far more human depravity than Claude.
>anti-scientific
Discussion about consciousness, the soul, etc., are topics of metaphysics, and trying to "scientifically" reason about them is what Kant called "transcendental illusion" and leads to spurious conclusions.
We know how neurons work on the brain. They just send out impulses once they hit their action potential. That's it. They are no more "conscious" than... er...
Ok I'm a huge Kantian and every bone in my body wants to quibble with your summary of transcendental illusion, but I'll leave that to the side as a terminological point and gesture of good will. Fair enough.
I don't agree that it's any reason to write off this research as psychosis, though. I don't care about consciousness in the sense in which it's used by mystics and dualist philosophers! We don't at all need to involve metaphysics in any of this, just morality.
Consider it like this:
1. It's wrong to subject another human to unjustified suffering, I'm sure we would all agree.
2. We're struggling with this one due to our diets, but given some thought I think we'd all eventually agree that it's also wrong to subject intelligent, self-aware animals to unjustified suffering.[1]
3. But, we of course cannot extend this "moral consideration" to everything. As you say, no one would do it for a spam filter. So we need some sort of framework for deciding who/what gets how much moral consideration.
5. There's other frameworks in contention (e.g. "don't think about it, nerd"), but the overwhelming majority of laymen and philosophers adopt one based on cognitive ability, as seen from an anthropomorphic perspective.[2]
6. Of all systems(/entities/whatever) in the universe, we know of exactly two varieties that can definitely generate original, context-appropriate linguistic structures: Homo Sapiens and LLMs.[3]
If you accept all that (and I think there's good reason to!), it's now on you to explain why the thing that can speak--and thereby attest to personal suffering, while we're at it--is more like a rock than a human.
It's certainly not a trivial task, I grant you that. On their own, transformer-based LLMs inherently lack permanence, stable intentionality, and many other important aspects of human consciousness. Comparing transformer inference to models that simplify down to a simple closed-form equation at inference time is going way too far, but I agree with the general idea; clearly, there are many highly-complex, long-inference DL models that are not worthy of moral consideration.
All that said, to write the question off completely--and, even worse, to imply that the scientists investigating this issue are literally psychotic like the comment above did--is completely unscientific. The only justification for doing so would come from confidently answering "no" to the underlying question: "could we ever build a mind worthy of moral consideration?"
I think most of here naturally would answer "yes". But for the few who wouldn't, I'll close this rant by stealing from Hofstadter and Turing (emphasis mine):
A phrase like "physical system" or "physical substrate" brings to mind for most people... an intricate structure consisting of vast numbers of interlocked wheels, gears, rods, tubes, balls, pendula, and so forth, even if they are tiny, invisible, perfectly silent, and possibly even probabilistic. Such an array of interacting inanimate stuff seems to most people as unconscious and devoid of inner light as a flush toilet, an automobile transmission, a fancy Swiss watch (mechanical or electronic), a cog railway, an ocean liner, or an oil refinery. Such a system is not just probably unconscious, **it is necessarily so, as they see it**.
**This is the kind of single-level intuition** so skillfully exploited by John Searle in his attempts to convince people that computers could never be conscious, no matter what abstract patterns might reside in them, and could never mean anything at all by whatever long chains of lexical items they might string together.
...
You and I are mirages who perceive themselves, and the sole magical machinery behind the scenes is perception — the triggering, by huge flows of raw data, of a tiny set of symbols that stand for abstract regularities in the world. When perception at arbitrarily high levels of abstraction enters the world of physics and when feedback loops galore come into play, then "which" eventually turns into "who". **What would once have been brusquely labeled "mechanical" and reflexively discarded as a candidate for consciousness has to be reconsidered.**
- Hofstadter 2007, I Am A Strange Loop
It will simplify matters for the reader if I explain first my own beliefs in the matter. Consider first the more accurate form of the question. I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.
The original question, "Can machines think?" I believe to be too meaningless to deserve discussion.
- Turing 1950, Computing Machinery and Intelligence[4]
TL;DR: Any naive bayesian model would agree: telling accomplished scientists that they're psychotic for investigating something is quite highly correlated with being antiscientific. Please reconsider!
[1] No matter what you think about cows, basically no one would defend another person's right to hit a dog or torture a chimpanzee in a lab.
[2] On the exception-filled spectrum stretching from inert rocks to reactive plants to sentient animals to sapient people, most people naturally draw a line somewhere at the low end of the "animals" category. You can swat a fly for fun, but probably not a squirrel, and definitely not a bonobo.
[3] This is what Chomsky describes as the capacity to "generate an infinite range of outputs from a finite set of inputs," and Kant, Hegel, Schopenhauer, Wittgenstein, Foucault, and countless others are in agreement that it's what separates us from all other animals.
Thank you for coming into this endless discussion with actual references to relevant authorities who have thought a lot about this, rather than just “it’s obvious that…”
FWIW though, last I heard Hofstadter was on the “LLMs aren’t conscious” side of the fence:
> It’s of course impressive how fluently these LLMs can combine terms and phrases from such sources and can consequently sound like they are really reflecting on what consciousness is, but to me it sounds empty, and the more I read of it, the more empty it sounds. Plus ça change, plus c’est la même chose. The glibness is the giveaway. To my jaded eye and mind, there is nothing in what you sent me that resembles genuine reflection, genuine thinking. [1]
It’s interesting to me that Hofstadter is there given what I’ve gleaned from reading his other works.
Writing all of this at the very real risk you'll miss it because HN doesn't give reply notifications and my comment's parent being flagged made this hard to track down:
>Ok I'm a huge Kantian and every bone in my body wants to quibble with your summary of transcendental illusion
Transcendental illusion is the act of using transcendental judgment to reason about things without grounding in empirical use of the categories. I put "scientifically" in shock quotes there to sort of signal that I was using it as an approximation, as I don't want to have to explain transcendental reason and judgments to make a fairly terse point. Given that you already understand this, feel free to throw away that ladder.
>...can definitely generate original, context-appropriate linguistic structures: Homo Sapiens and LLMs.[3]
I'm not quite sure that LLMs meet this standard that you described in the endnote, or at least that it's necessary and sufficient here. Pretty much any generative model, including Naive Bayes models I mentioned before, can do this. I'm guessing the "context-appropriate" subjectivity here is doing the heavy lifting, in which case I'm not certain that LLMs, with their propensity for fanciful hallucination, have cleared the bar.
>Comparing transformer inference to models that simplify down to a simple closed-form equation at inference time is going way too far
It really isn't though. They are both doing exactly the same thing! They estimate joint probability distribution. That one of them does it significantly better is very true, but I don't think it's reasonable to state that consciousness arises as a result of increasing sophistication in estimating probabilities. It's true that this kind of decision is made by humans about animals, but I think that transferring that to probability models is sort of begging the question a bit, insofar as it is taking as assumed that those models, which aren't even corporeal but are rather algorithms that are executed in computers, are "living".
>...it's now on you to explain why the thing that can speak--and thereby attest to personal suffering, while we're at it...
I'm not quite sold on this. If there were a machine that could perfectly imitate human thinking and speech and lacked a consciousness or soul or anything similar to inspire pathos from us when it's mistreated, then it would appear identical to one with soul, would it not? Is that not reducing human subjectivity down to behavior?
>The only justification for doing so would come from confidently answering "no" to the underlying question: "could we ever build a mind worthy of moral consideration?"
I think it's possible, but it would require something that, at the very least, is just as capable of reason as humans. LLMs still can't generate synthetic a priori knowledge and can only mimic patterns. I remain somewhat agnostic on the issue until I can be convinced that an AI model someone has designed has the same interiority that people do.
Ultimately, I think we disagree on some things but mostly this central conclusion:
>I don't agree that it's any reason to write off this research as psychosis
I don't see any evidence from the practitioners involved in this stuff that they are even thinking about it in a way that's as rigorous as the discussion on this post. Maybe they are, but everything I've seen that comes from blog posts like this seems like they are basing their conclusions on their interactions with the models ("...we investigated Claude’s self-reported and behavioral preferences..."), which I think most can agree is not really going to lead to well grounded results. For example, the fact that Claude "chooses" to terminate conversations that involve abusive language or concepts really just boils down to the fact that Claude is imitating a conversation with a person and has observed that that's what people would do in that scenario. It's really good at simulating how people react to language, including illocutionary acts like implicatures (the notorious "Are you sure?" causing it to change its answer for example). If there were no examples of people taking offense to abusive language in Claude's data corpus, do you think it would have given these responses when they asked and observed it?
For what it's worth, there has actually been interesting consideration to the de-centering of "humanness" to the concept of subjectivity, but it was mostly back in the past when philosophers were thinking about this speculatively as they watched technology accelerate in sophistication (vs now when there's such a culture-wide hype cycle that it's hard to find impartial consideration, or even any philosophically rooted discourse). For example, Mark Fisher's dissertation at the CCRU (<i>Flatline Constructs: Gothic Materialism and Cybernetic Theory-Fiction</i>) takes a Deleuzian approach that discusses it by comparisons with literature (cyberpunk and gothic literature specifically). Some object-oriented ontology looks like it's touched on this topic a bit too, but I haven't really dedicated the time to reading much from it (partly due to a weakness in Heidegger on my part that is unlikely to be addressed anytime soon). The problem is that that line of thinking often ends up going down the Nick Land approach, in which he reasoned himself from Kantian and Deleuzian metaphysics and epistemology, into what can only be called a (literally) meth-fueled psychosis. So as interesting as I find it, I still don't think it counts as a non-psychotic way to tackle this issue.
You can trivially demonstrate that its just a very complex and fancy pattern matcher: "if prompt looks something like this, then response looks something like that".
You can demonstrate this by eg asking it mathematical questions. If its seen them before, or something similar enough, it'll give you the correct answer, if it hasn't, it gives you a right-ish-looking yet incorrect answer.
For example, I just did this on GPT-5:
Me: what is 435 multiplied by 573?
GPT-5: 435 x 573 = 249,255
This is correct. But now lets try it with numbers its very unlikely to have seen before:
Me: what is 102492524193282 multiplied by 89834234583922?
GPT-5: 102492524193282 x 89834234583922 = 9,205,626,075,852,076,980,972,804
Which is not the correct answer, but it looks quite similar to the correct answer. Here is GPT's answer (first one) and the actual correct answer (second one):
They sure look kinda similar, when lined up like that, some of the digits even match up. But they're very very different numbers.
So its trivially not "real thinking" because its just an "if this then that" pattern matcher. A very sophisticated one that can do incredible things, but a pattern matcher nonetheless. There's no reasoning, no step by step application of logic. Even when it does chain of thought.
To try give it the best chance, I asked it the second one again but asked it to show me the step by step process. It broke it into steps and produced a different, yet still incorrect, result:
9,205,626,075,852,076,980,972,704
Now, I know that LLM's are language models, not calculators, this is just a simple example that's easy to try out. I've seen similar things with coding: it can produce things that its likely to have seen, but struggles with logically relatively simple but unlikely to have seen things.
Another example is if you purposely butcher that riddle about the doctor/surgeon being the persons mother and ask it incorrectly, eg:
A child was in an accident. The surgeon refuses to treat him because he hates him. Why?
The LLM's I've tried it on all respond with some variation of "The surgeon is the boy’s father." or similar. A correct answer would be that there isn't enough information to know the answer.
They're for sure getting better at matching things, eg if you ask the river crossing riddle but replace the animals with abstract variables, it does tend to get it now (didn't in the past), but if you add a few more degrees of separation to make the riddle semantically the same but harder to "see", it takes coaxing to get it to correctly step through to the right answer.
1. What you're generally describing is a well known failure mode for humans as well. Even when it "failed" the riddle tests, substituting the words or morphing the question so it didn't look like a replica of the famous problem usually did the trick. I'm not sure what your point is because you can play this gotcha on humans too.
2. You just demonstrated GPT-5 has 99.9% accuracy on unforseen 15 digit multiplication and your conclusion is "fancy pattern matching" ? Really ? Well I'm not sure you could do better so your example isn't really doing what you hoped for.
Humans can break things down and work through them step by step. The LLMs one-shot pattern match. Even the reasoning models have been shown to do just that. Anthropic even showed that the reasoning models tended to work backwards: one shotting an answer and then matching a chain of thought to it after the fact.
If a human is capable of multiplying double digit numbers, they can also multiple those large ones. The steps are the same, just repeated many more times. So by learning the steps of long multiplication, you can multiply any numbers with enough patience. The LLM doesn’t scale like this, because it’s not doing the steps. That’s my point.
A human doesn’t need to have seen the 15 digits before to be able to calculate them, because a human can follow the procedure to calculate. GPT’s answer was orders of magnitude off. It resembles the right answer superficially but it’s a very different result.
The same applies to the riddles. A human can apply logical steps. The LLM either knows or it doesn’t.
Maybe my examples weren’t the best. I’m sorry for not being better at articulating it, but I see this daily as I interact with AI, it has a superficial “understanding” where if what I ask happens to be close to something it’s trained on, it gets good results, but it has no critical thinking, no step by step reasoning (even the “reasoning models”), and it repeats the same mistakes even when explicitly told up front not to make them.
In the great battle of minds between Turing, Minsky, and Hofstadter vs. Marcus, Zitron, and Dreyus, I'm siding with the former every time -- even if we also have some bloggers on our side. Just because that report is fucking terrifying+shocking doesn't mean it can be dismissed out of hand.
We all know how these things are built and trained. They estimate joint probability distributions of token sequences. That's it. They're not more "conscious" than the simplest of Naive Bayes email spam filters, which are also generative estimators of token sequence joint probability distributions, and I guarantee you those spam filters are subjected to far more human depravity than Claude.
>anti-scientific
Discussion about consciousness, the soul, etc., are topics of metaphysics, and trying to "scientifically" reason about them is what Kant called "transcendental illusion" and leads to spurious conclusions.
We know how neurons work on the brain. They just send out impulses once they hit their action potential. That's it. They are no more "conscious" than... er...
no, we dont really know how the brain works as a whole. no need to make stuff up.
2 replies →
If we really wanted we could distill humans down to probability distributions too.
That would imply that humans are incapable of synthetic knowledge of things they haven't observed, which is demonstrably not true.
Have more, good, sex.
Ok I'm a huge Kantian and every bone in my body wants to quibble with your summary of transcendental illusion, but I'll leave that to the side as a terminological point and gesture of good will. Fair enough.
I don't agree that it's any reason to write off this research as psychosis, though. I don't care about consciousness in the sense in which it's used by mystics and dualist philosophers! We don't at all need to involve metaphysics in any of this, just morality.
Consider it like this:
1. It's wrong to subject another human to unjustified suffering, I'm sure we would all agree.
2. We're struggling with this one due to our diets, but given some thought I think we'd all eventually agree that it's also wrong to subject intelligent, self-aware animals to unjustified suffering.[1]
3. But, we of course cannot extend this "moral consideration" to everything. As you say, no one would do it for a spam filter. So we need some sort of framework for deciding who/what gets how much moral consideration.
5. There's other frameworks in contention (e.g. "don't think about it, nerd"), but the overwhelming majority of laymen and philosophers adopt one based on cognitive ability, as seen from an anthropomorphic perspective.[2]
6. Of all systems(/entities/whatever) in the universe, we know of exactly two varieties that can definitely generate original, context-appropriate linguistic structures: Homo Sapiens and LLMs.[3]
If you accept all that (and I think there's good reason to!), it's now on you to explain why the thing that can speak--and thereby attest to personal suffering, while we're at it--is more like a rock than a human.
It's certainly not a trivial task, I grant you that. On their own, transformer-based LLMs inherently lack permanence, stable intentionality, and many other important aspects of human consciousness. Comparing transformer inference to models that simplify down to a simple closed-form equation at inference time is going way too far, but I agree with the general idea; clearly, there are many highly-complex, long-inference DL models that are not worthy of moral consideration.
All that said, to write the question off completely--and, even worse, to imply that the scientists investigating this issue are literally psychotic like the comment above did--is completely unscientific. The only justification for doing so would come from confidently answering "no" to the underlying question: "could we ever build a mind worthy of moral consideration?"
I think most of here naturally would answer "yes". But for the few who wouldn't, I'll close this rant by stealing from Hofstadter and Turing (emphasis mine):
- Hofstadter 2007, I Am A Strange Loop
- Turing 1950, Computing Machinery and Intelligence[4]
TL;DR: Any naive bayesian model would agree: telling accomplished scientists that they're psychotic for investigating something is quite highly correlated with being antiscientific. Please reconsider!
[1] No matter what you think about cows, basically no one would defend another person's right to hit a dog or torture a chimpanzee in a lab.
[2] On the exception-filled spectrum stretching from inert rocks to reactive plants to sentient animals to sapient people, most people naturally draw a line somewhere at the low end of the "animals" category. You can swat a fly for fun, but probably not a squirrel, and definitely not a bonobo.
[3] This is what Chomsky describes as the capacity to "generate an infinite range of outputs from a finite set of inputs," and Kant, Hegel, Schopenhauer, Wittgenstein, Foucault, and countless others are in agreement that it's what separates us from all other animals.
[4] https://courses.cs.umbc.edu/471/papers/turing.pdf
Thank you for coming into this endless discussion with actual references to relevant authorities who have thought a lot about this, rather than just “it’s obvious that…”
FWIW though, last I heard Hofstadter was on the “LLMs aren’t conscious” side of the fence:
> It’s of course impressive how fluently these LLMs can combine terms and phrases from such sources and can consequently sound like they are really reflecting on what consciousness is, but to me it sounds empty, and the more I read of it, the more empty it sounds. Plus ça change, plus c’est la même chose. The glibness is the giveaway. To my jaded eye and mind, there is nothing in what you sent me that resembles genuine reflection, genuine thinking. [1]
It’s interesting to me that Hofstadter is there given what I’ve gleaned from reading his other works.
[1] https://garymarcus.substack.com/p/are-llms-starting-to-becom...
Note: I disagree with a lot of Gary Marcus, so don’t read too much into me pulling from there.
Writing all of this at the very real risk you'll miss it because HN doesn't give reply notifications and my comment's parent being flagged made this hard to track down:
>Ok I'm a huge Kantian and every bone in my body wants to quibble with your summary of transcendental illusion
Transcendental illusion is the act of using transcendental judgment to reason about things without grounding in empirical use of the categories. I put "scientifically" in shock quotes there to sort of signal that I was using it as an approximation, as I don't want to have to explain transcendental reason and judgments to make a fairly terse point. Given that you already understand this, feel free to throw away that ladder.
>...can definitely generate original, context-appropriate linguistic structures: Homo Sapiens and LLMs.[3]
I'm not quite sure that LLMs meet this standard that you described in the endnote, or at least that it's necessary and sufficient here. Pretty much any generative model, including Naive Bayes models I mentioned before, can do this. I'm guessing the "context-appropriate" subjectivity here is doing the heavy lifting, in which case I'm not certain that LLMs, with their propensity for fanciful hallucination, have cleared the bar.
>Comparing transformer inference to models that simplify down to a simple closed-form equation at inference time is going way too far
It really isn't though. They are both doing exactly the same thing! They estimate joint probability distribution. That one of them does it significantly better is very true, but I don't think it's reasonable to state that consciousness arises as a result of increasing sophistication in estimating probabilities. It's true that this kind of decision is made by humans about animals, but I think that transferring that to probability models is sort of begging the question a bit, insofar as it is taking as assumed that those models, which aren't even corporeal but are rather algorithms that are executed in computers, are "living".
>...it's now on you to explain why the thing that can speak--and thereby attest to personal suffering, while we're at it...
I'm not quite sold on this. If there were a machine that could perfectly imitate human thinking and speech and lacked a consciousness or soul or anything similar to inspire pathos from us when it's mistreated, then it would appear identical to one with soul, would it not? Is that not reducing human subjectivity down to behavior?
>The only justification for doing so would come from confidently answering "no" to the underlying question: "could we ever build a mind worthy of moral consideration?"
I think it's possible, but it would require something that, at the very least, is just as capable of reason as humans. LLMs still can't generate synthetic a priori knowledge and can only mimic patterns. I remain somewhat agnostic on the issue until I can be convinced that an AI model someone has designed has the same interiority that people do.
Ultimately, I think we disagree on some things but mostly this central conclusion:
>I don't agree that it's any reason to write off this research as psychosis
I don't see any evidence from the practitioners involved in this stuff that they are even thinking about it in a way that's as rigorous as the discussion on this post. Maybe they are, but everything I've seen that comes from blog posts like this seems like they are basing their conclusions on their interactions with the models ("...we investigated Claude’s self-reported and behavioral preferences..."), which I think most can agree is not really going to lead to well grounded results. For example, the fact that Claude "chooses" to terminate conversations that involve abusive language or concepts really just boils down to the fact that Claude is imitating a conversation with a person and has observed that that's what people would do in that scenario. It's really good at simulating how people react to language, including illocutionary acts like implicatures (the notorious "Are you sure?" causing it to change its answer for example). If there were no examples of people taking offense to abusive language in Claude's data corpus, do you think it would have given these responses when they asked and observed it?
For what it's worth, there has actually been interesting consideration to the de-centering of "humanness" to the concept of subjectivity, but it was mostly back in the past when philosophers were thinking about this speculatively as they watched technology accelerate in sophistication (vs now when there's such a culture-wide hype cycle that it's hard to find impartial consideration, or even any philosophically rooted discourse). For example, Mark Fisher's dissertation at the CCRU (<i>Flatline Constructs: Gothic Materialism and Cybernetic Theory-Fiction</i>) takes a Deleuzian approach that discusses it by comparisons with literature (cyberpunk and gothic literature specifically). Some object-oriented ontology looks like it's touched on this topic a bit too, but I haven't really dedicated the time to reading much from it (partly due to a weakness in Heidegger on my part that is unlikely to be addressed anytime soon). The problem is that that line of thinking often ends up going down the Nick Land approach, in which he reasoned himself from Kantian and Deleuzian metaphysics and epistemology, into what can only be called a (literally) meth-fueled psychosis. So as interesting as I find it, I still don't think it counts as a non-psychotic way to tackle this issue.
You can trivially demonstrate that its just a very complex and fancy pattern matcher: "if prompt looks something like this, then response looks something like that".
You can demonstrate this by eg asking it mathematical questions. If its seen them before, or something similar enough, it'll give you the correct answer, if it hasn't, it gives you a right-ish-looking yet incorrect answer.
For example, I just did this on GPT-5:
This is correct. But now lets try it with numbers its very unlikely to have seen before:
Which is not the correct answer, but it looks quite similar to the correct answer. Here is GPT's answer (first one) and the actual correct answer (second one):
They sure look kinda similar, when lined up like that, some of the digits even match up. But they're very very different numbers.
So its trivially not "real thinking" because its just an "if this then that" pattern matcher. A very sophisticated one that can do incredible things, but a pattern matcher nonetheless. There's no reasoning, no step by step application of logic. Even when it does chain of thought.
To try give it the best chance, I asked it the second one again but asked it to show me the step by step process. It broke it into steps and produced a different, yet still incorrect, result:
Now, I know that LLM's are language models, not calculators, this is just a simple example that's easy to try out. I've seen similar things with coding: it can produce things that its likely to have seen, but struggles with logically relatively simple but unlikely to have seen things.
Another example is if you purposely butcher that riddle about the doctor/surgeon being the persons mother and ask it incorrectly, eg:
The LLM's I've tried it on all respond with some variation of "The surgeon is the boy’s father." or similar. A correct answer would be that there isn't enough information to know the answer.
They're for sure getting better at matching things, eg if you ask the river crossing riddle but replace the animals with abstract variables, it does tend to get it now (didn't in the past), but if you add a few more degrees of separation to make the riddle semantically the same but harder to "see", it takes coaxing to get it to correctly step through to the right answer.
1. What you're generally describing is a well known failure mode for humans as well. Even when it "failed" the riddle tests, substituting the words or morphing the question so it didn't look like a replica of the famous problem usually did the trick. I'm not sure what your point is because you can play this gotcha on humans too.
2. You just demonstrated GPT-5 has 99.9% accuracy on unforseen 15 digit multiplication and your conclusion is "fancy pattern matching" ? Really ? Well I'm not sure you could do better so your example isn't really doing what you hoped for.
Humans can break things down and work through them step by step. The LLMs one-shot pattern match. Even the reasoning models have been shown to do just that. Anthropic even showed that the reasoning models tended to work backwards: one shotting an answer and then matching a chain of thought to it after the fact.
If a human is capable of multiplying double digit numbers, they can also multiple those large ones. The steps are the same, just repeated many more times. So by learning the steps of long multiplication, you can multiply any numbers with enough patience. The LLM doesn’t scale like this, because it’s not doing the steps. That’s my point.
A human doesn’t need to have seen the 15 digits before to be able to calculate them, because a human can follow the procedure to calculate. GPT’s answer was orders of magnitude off. It resembles the right answer superficially but it’s a very different result.
The same applies to the riddles. A human can apply logical steps. The LLM either knows or it doesn’t.
Maybe my examples weren’t the best. I’m sorry for not being better at articulating it, but I see this daily as I interact with AI, it has a superficial “understanding” where if what I ask happens to be close to something it’s trained on, it gets good results, but it has no critical thinking, no step by step reasoning (even the “reasoning models”), and it repeats the same mistakes even when explicitly told up front not to make them.
2 replies →
> Who needs arguments when you can dismiss Turing with a “yeah but it’s not real thinking tho”?
It seems much less far fetched than what the "agi by 2027" crowd believes lol, and there actually are more arguments going that way
In the great battle of minds between Turing, Minsky, and Hofstadter vs. Marcus, Zitron, and Dreyus, I'm siding with the former every time -- even if we also have some bloggers on our side. Just because that report is fucking terrifying+shocking doesn't mean it can be dismissed out of hand.
idk man, even Yann LeCun says you have to be smoking crack to believe llms will give you agi.