Comment by godelski
7 months ago
What's odd to me is how everything is so metricized. Clearly over metricization is the downfall of any system that looks meritocratic. Due to the limitations of metrics and how they are often far easier to game than to reach through the intended means.
An example of this I see is how new leaders come in and hit hard to cut costs. But the previous leader did this (and the one before them) so the system/group/company is fairly lean already. So to get anywhere near similar reductions or cost savings it typically means cutting more than fat. Which it's clear that many big corps are not running with enough fat in the first place (you want some fat! You just don't want to be obese!). This seems to create a pattern that ends up being indistinguishable from "That worked! Let's not do that anymore."
Agree you have to mix qualitative with the quantitative, but the best metrics systems don't just measure one quantity metric. They should be paired with a quality metric.
Example: User Growth & Customer Engagement
Have to have user growth and retention. If you looked at just one or the other, you'd be missing half the equation.
I think that a good portion of the problem is that there are groups involved in metrics:
1) People setting the metrics
2) People implementing/calculating the metrics
3) People working on improving the metrics (ie product work)
2 is specially complicated for a lot of software products because it can some times be really hard to measure and can be tweaked/manipulated. For example, the MAU twitter figures from the buyout that Musk keeps complaining about, or Blizzard constantly switching their MAU definition.
Often 2 and 3 are the same people and 1 is almost always upper management. I argue that 1 and 2 should be a single group of people (that doesn't work on the product at all) and not directly subject to upper management and not tracked by the same metrics they implement (or tracked by any metrics at all).
Absurdity, unfairness, and failure often result from selective blindness to reality, whether willful or unintentional. Hyperlogical people sometimes lack empathy or an ability to conceive of, to understand, or prefer to trivialize ambiguous situations, politics, biases, human factors, or nonfunctional requirements. Always keep looking for one's own and organizational blind spots.
Oh god. The blind faith in reductive, objectivist, rationalist meritocracy that somehow "everything can be measured perfectly" and "whatever happens is completely unbiased as proscribed by a black-and-white, mechanical formula". No, sorry, that's insufficiently holistic in accounting for intangibles and supportive effort, and more of a throwback ideology that should've died in the 1920's. Some degree of discretion is needed because there is no shortcut to "measuring" performance.