Comment by junon
11 hours ago
This. If I'm forced to use a feature I hate because it's the only way to do something, the "ground truth" reflects that I like that feature. It doesn't tell the whole story.
11 hours ago
This. If I'm forced to use a feature I hate because it's the only way to do something, the "ground truth" reflects that I like that feature. It doesn't tell the whole story.
Most metrics teams are reasonably competent and are aware of that. Excepting "growth hackers"
I haven't been in a single metrics discussion where we didn't talk about what we're actually measuring, if it reflects what we want to measure, and how to counterbalance metrics sufficiently so we don't build yet another growthhacking disaster.
Doesn't mean that metrics are perfect - they are in fact aggravatingly imprecise - but the ground truth is usually somewhat better than "you clicked it, musta liked it!"
And yet, the observable evidence of changes in software that collect metrics directly contradict this.
Eh, there are a lot of cases where teams A/B test their way into a product that sucks.