Comment by arjie
3 days ago
This kind of thing is totally fine if it's being done (it's believable because Meta internally incentivized tokenmaxxing). When you're trying to change the behavior of a large number of people, only blunt instruments are available if you want to get quick outcomes. The edge cases where people Goodhart very hard are all right. You can just human-in-the-loop them away. The opportunity cost for most organizations of not moving to use AI tools as productivity enhancers is currently gauged by them (rightfully, in my opinion) to be too high to allow for osmotic adoption.
Most people look at sea changes come and go. They all have a story of how they "could have bought Bitcoin when it was $100" or whatever. In an org, you don't want to have the story of "we could have done that when nobody else had", so you incentivize adoption of the tool as hard as possible and hope that dipping feet in the water makes people want to swim. If you don't already have a culture of early adoption (and no large company can) then you have to use blunt incentives. I don't think anyone has demonstrated otherwise.
So you're saying that if you ignore all the downsides with "you can just human in the loop them away" then it works great?
Even if you don't, it's the only way to ensure adoption and most workplaces consider the lack of adoption a greater danger than a Goodharted adoption. Overall, I've observed that the US has very low barrier to starting companies so, considering companies of all sizes are doing this, if it's a mistake those startups will get beaten by the ones doing other things.