Comment by butterisgood
9 hours ago
People need to stop redefining and trying to capture the term AGI. None of this is AGI. Not even close. Can it do things an AGI could do? Yeah some of it, but the difference really matters. These things still regularly fail, and gaslight about answers to questions like "how many r's in strawberry" or "s's in espresso".
An AGI wouldn't struggle with that.
The last version to fail on those questions was GPT 4.5.
Meanwhile most humans fail to correctly answer how many f's are in the sentence, "Finished files are the result of years of scientific study combined with the experience of many years.".
It comes and goes... My point is we're not near AGI.
Nooo. I just asked Claude (Sonnet 5 Medium) how many I's are in assassin, and it said 2. Granted, it got several words correct before this. But no, they still aren't great at this letter-counting thing.
I don't think we should count the lower tier models if we're discussing what the top ones are capable of. No one was suggesting that Sonnet is AGI.
Meanwhile Qwen3.8 27B got both the 'f's and the 'i's questions right.
Anyone know why they aren't good at this?
3 replies →
A lot of movies gave the impression that making phone calls and balancing checkbooks would be the easiest tasks for consumer AI to solve, while math and science might require exceedingly advanced AI. Turns out to be the opposite: computers are great at math and bad at conversation.
That's because movies were based on the "general" nature of AI, assuming we would create intelligence that would learn and grow.
Pretty much the opposite of what we can't up with, if you're willing to call what we have intelligence.
> "how many r's in strawberry" or "s's in espresso".
And what percentage of your red retina receptors are firing?
The "number of letters" critique was broken before it was introduced the first time. The models were specifically designed with preprocessing to not be able to perceive their input as strings of letters. Blind people are not dumb. (True as a pun and in context.)
Sub-access sensory questions, or do-you-know-a-fact questions (which is what spelling becomes when you can't see the letters, and are not specifically trained to match all token encoded words to their letters) are not intelligence questions.
These are like saying someone isn’t human because they have a speech impediment or an auditory processing disorder.
AGI doesn’t mean infallible, it just means it can have a reasonable crack at things it hasn’t seen or done before.
I don't understand why you're being downvoted... that's literally the definition of AGI.
I'm going to go out on a limb and guess that there are plenty of savants who can't tell you how many r's are in strawberry.
> gaslight about answers to questions like "how many r's in strawberry" or "s's in espresso".
this has been debunked too many times to bother rebutting. they struggle with those things because of the way they are.
it's completely irrelevant.
If it allows you to distinguish readily between human intelligence and computer intelligence, it seems quite relevant to the question of whether computers have achieved something akin to human intelligence.
It may not be useful for anything else, but at least it can say that.
But it's not relevant as a metric to gauge distance to human intelligence. Humans see individual letters, LLMs do not. If I asked you the relative activation of the cones in your retina as I showed you some solid color image, you couldn't do it. You simply do not have cognitive access to that information. But that says nothing about your intelligence.
A more accurate test would be to give it a list of words (or anything represented as a single token) and ask it how many times that token appeared. I'm sure they have no trouble at that task.
3 replies →
I am not enthusiastic about criteria for human-like intelligence that imply that dyslexic people don't have human-like intelligence.
[EDITED to add:] I actually don't know whether dyslexic people find it difficult to count letters in words, if they have them already written down by someone else. I suspect they find it harder than people who aren't dyslexic. But perhaps "blind people whose spelling is poor" would have been better; I would not want to deny them human-like intelligence either.
2 replies →
which just brings us back to the whole birds vs planes thing.
turns out that flapping wings is not the right way to unlock human flight.
computers could count the Rs in strawberry since vacuum tubes. that measure is irrelevant.
10 replies →
> they struggle with those things because of the way they are. it's completely irrelevant.
I mean, they seem like fair game if you’re ever participating in a Turing Test.