Comment by zmmmmm

Sure, but there's no reason there can't be a correlation between us "thinking" it's intelligent and it actually being intelligent. What other proxy should we use? I can't think of a scenario where it's actually intelligent but humans don't think it is that has a good practical ending. It's at least necessary even if it isn't sufficient.