Comment by kevincox

I think it is worse than that. Yes they A/B test. But they also have incentive to show how great their new feature idea is. So they are always picking the metrics that make their feature look better and ignoring the ones that make it look worse. So there is a thumb on the scale here.