← Back to context

Comment by VikingCoder

7 months ago

Ugh. I have a feeling I know what happened.

People ran experiments where they showed big, ugly, profitable ads, and they convinced themselves that the metrics meant it was a positive experience for users.

p-Hacking, again and again.

I have a ton of sympathy for everyone involved in this. It's incredibly hard to have a good model of what is good for users, and to have metrics that measure relevant things, and to have the discipline to make yourself test a real hypothesis rather than hunt for evidence that proves your foregone conclusion. And to reward people for negative experiments.

All very hard to do.

And frankly, if Google can't do it right - who can?

I think you need really powerful product managers who happen to be right. And that's not sustainable. Not something you can plan on or measure. Only reward if you happen to be lucky enough to notice it. Ow.