Comment by rwmj
2 months ago
Can modern humans count the number of tokens in "blueberry"? (You're not allowed to cheat and use https://platform.openai.com/tokenizer)
2 months ago
Can modern humans count the number of tokens in "blueberry"? (You're not allowed to cheat and use https://platform.openai.com/tokenizer)
I see this argument every time this particular failing is brought up and like, regardless of the reasoning, it's still completely failing to do something a second grader should be able to do and until it stops failing in that way, being like "Um technically it's a lot harder than you think" isn't going to win over AI skeptics.
A Chinese second grader would probably not be able to answer "How many Rs are in the word 草莓?" It's a valid critique.
You're right they probably wouldn't be able to to it, but that child isn't being marketed as an invaluable business tool to assist doctors in diagnosing cancer, or whatever.
2 replies →
One is an unambiguous question, the other depends on the tokenizing algorithm.
No, but I can recognize my inability to do so instead of making the answer up.
Uh, who cares?
That is literally useless knowledge.
When's the last time you needed to know the number of Rs in "blueberry"?
When's the last time you needed to count the number of things inside a larger thing?
5 replies →