Comment by ivraatiems
5 days ago
That doesn't seem right to me. If I understand it right, your logic is:
1. Humans intellect is Turing computable. 2. LLMs are based on Turing-complete technology. 3. Therefore, LLMs can eventually equal human intellect.
But if that is the right chain of assumptions, there's lots of issues with it. First, whether LLMs are Turing complete is a topic of debate. There are points for[0] and against[1].
I suspect they probably _are_, but that doesn't mean LLMs are tautologically indistinguishable from human intelligence. Every computer that uses a Turing-complete programming language can theoretically solve any Turing-computable problem. That does not mean they will ever be able to efficiently or effectively do so in real time under real constraints, or that they are doing so now in a reasonable amount real-world time using extant amounts of real-world computing power.
The processor I'm using to write this might be able to perform all the computations needed for human intellect, but even if it could, that doesn't mean it can do it quickly enough to compute even a single nanosecond of actual human thought before the heat-death of the universe, or even the end of this century.
So when you say:
> Without that any limitations borne out of what LLMs don't currently do are irrelevant.
It seems to me exactly the opposite is true. If we want technology that is anything approaching human intelligence, we need to find approaches which will solve for a number of things LLMs don't currently do. The fact that we don't know exactly what those things are yet is not evidence that those things don't exist. Not only do they likely exist, but the more time we spend simply scaling LLMs instead of trying to find them, the farther we are from any sort of genuine general intelligence.
[0] https://arxiv.org/abs/2411.01992 [1] https://medium.com/heyjobs-tech/turing-completeness-of-llms-...
> 1. Humans intellect is Turing computable. 2. LLMs are based on Turing-complete technology. 3. Therefore, LLMs can eventually equal human intellect.
Yes, with an emphasis on can. That does not mean they necessarily will. Though I would consider it unlikely that they won't, we the only way of proving that they will would be to do it.
> But if that is the right chain of assumptions, there's lots of issues with it. First, whether LLMs are Turing complete is a topic of debate. There are points for[0] and against[1].
It's trivial to prove that a system comprised of an LLM with a loop is Turing computable. A single inference step can not be Turing computable, but one with a loop only requires the LLM to be capable of executing 6 distinct steps with temperatur set to 0. You can wire up a toy neural network by hand that can do this.
This is in fact a far more limited claim than what the paper you linked to makes.
The article you linked to, on the other hand, is discussing if an LLM can act like a Turing machine without that loop. That is why "state management" matters. State management is irrelevant when you wrap a loop around because you can externalise the state, and you only need 2 states (and 3 symbols, or 3 states and 2 symbols) for the smallest known universal Turing machine.
The entire article is thus entirely irrelevant to this question. Sure, you will struggle to make an LLM act as a Turing machine without going "off the rails". But that is irrelevant - you only need it to be able to execute one state transition by deterministically producing the right next tape operation and next state when given the next symbol and current state.
From that you can build up to arbitrarily complex computation, because for every given Turing machine, you can construct a larger Turing machine that uses additional symbols or states to encode an operation that takes multiple steps for the smaller machine.
> The processor I'm using to write this might be able to perform all the computations needed for human intellect, but even if it could, that doesn't mean it can do it quickly enough to compute even a single nanosecond of actual human thought before the heat-death of the universe, or even the end of this century.
Irrelevant, because absent physics that exceeds the Turing computable, the human brain is an existence-proof for the possibility of computing everything the human brain does in a package the size of an average human brain.
It is very likely that we will need architectural changes to compute any given model efficiently enough, but to suggest it is not possible is an extraordinary claim not supported by anything.
If you take LLM to mean a very specific architecture, or specific computational methods to execute a model, then you have a point. If so we are talking about very different things.