Comment by mulmen

I think this is one of the greatest fallacies in modern data science. We only know what we can measure. There's no data on the opportunity cost of design decisions.

In other words Discourse only has metrics on people who use Discourse.

I think this is probably why sign-up metrics are so common but those are perverse as well. How much do you set yourself back if you only work for people willing to sign up?