Comment by mjdv
3 months ago
> We never talk about it now because we obviously blew past it years ago.
It's shocking to me that (as far as I know) no one has actually bothered to do a real Turing test with the best and newest LLMs. The Turing test is not whether a casual user can be momentarily confused about whether they are talking to a real person, or if a model can generate real-looking pieces of text. It's about a person seriously trying, for a fair amount of time, to distinguish between a chat they are having with another real person and an AI.
Q: Do you play chess? A: Yes. Q: I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play? A: (After a pause of 15 seconds) R-R8 mate.
A:I don’t know chess notation
Tbf, a machine is more likely to be versed in this ancient descriptive notation than a human is who is maybe just playing casually. R1 and K1 have not been around since the 80s.