Comment by sanderjd
2 years ago
> It's the closest we can easily get to the 'average user experience'.
Maybe it's the closest we can get (though I doubt it), but it definitely isn't close enough to tell us anything about the "average user experience".
The average user has been using google for years, without taking any steps to avoid personalization. An incognito session (on a browser / machine / network that is probably fingerprinted...) is pretty much the opposite of that typical usage pattern.
I recognize that just writing a blog post or comment on HN is not a research project so needs to do something quick, but I think it mostly invalidates the experiment. What would get closer would be to devise a few user personas and attempt to search and browse for awhile within those personas before trying the experiment. Or much better yet, put together a focus group comprised of real people within the personas you're interested in, and run the experiment using their real accounts.
> If we posit that every user eventually trains Google to avoid SEO spam
I don't think it's that, I think it's that every user trains it to return results more likely to improve the metric of "more likely to click one of the links", and I think that makes it more, not less, likely that they see what most of us here consider to be spam.
But I don't know! Maybe that's not what this experimental setup would show. But it would be a lot more enlightening than a setup using a fresh incognito window, which reflects the usage pattern of a proportion of search queries that is a tiny rounding error above zero.
Why are you assuming all users are logged in to google all the time?
Because it is objectively the case that the "average user" of the internet has a google cookie in their browser. It doesn't require that they be logged in - though I believe it's likely also the case that the "average user" is indeed logged into a google account - it just requires that they use google search without turning off cookies or specifically blocking google's. Essentially everybody uses google search and essentially nobody cares enough (or would know how) to turn off cookies or block google's cookie.
If this doesn't describe most people you know, you're in a very small bubble. (I'm somewhat in that bubble too, but I still have lots of family and friends who use the internet the normal way.)
Google has billions of user accounts ....