Comment by jaredklewis
3 days ago
I’m confused as to why anyone would think this would be possible to determine.
Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs?
What job out there is so simple that we can meaningfully measure all the positive and negative effects of the worker as well as account for different conditions between workers.
I could probably get behind the idea that you could measure productivity for professional poker players (given a long enough evaluation period). Hard to think of much else.
People in charge love to measure productivity and, just as harmfully, performance. The main insight people running large organisations (big business and governments) have into how they are doing is metrics, so they will use what measures they can have regardless of how meaningful they are.
The British government (probably not any worse than anyone else, just what I am most familiar with) does measure the productivity of the NHS: https://www.england.nhs.uk/long-read/nhs-productivity/ (including doctors, obviously).
They also try to measure the performance of teachers and schools and introduced performance league tables and special exams (SATS - exams sat at various ages school children in the state system, nothing like the American exams with the same name) to do this more pervasively. They made it better by creating multi-academy trusts which adds a layer of management running multi-schools so even more people want even more metrics.
The same for police, and pretty much everything else.
Won't stop MBAs from trying though.
Yet paradoxically, the user knows instinctively. I know exactly when I'll get my next medical checkup, and when the test results will arrive. I know if a software app improves my work, and what it will cost to get a paid license.
The hard thing is occupations where the quantity of effort is unrelated to the result due to the vast number of confounding factors.
Theres an interesting recent study showing how poorly developers measure their own productivity:
https://secondthoughts.ai/p/ai-coding-slowdown
HN discussion: https://news.ycombinator.com/item?id=44526912
We can determine the productivity of factory workers, and that is still(!) how we are seen by some managers.
And to be fair, some crud work is repetitive enough so it should be possible to get a fair measure of at least the difference in speed between developers.
But that building simple crud services with rest interfaces takes as much time as it does is a failure of the tools we use.
Duly upvoted! I tend to agree. Yet the shibboleth of productivity haunts us still.
> Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs?
Yes, yes we can.
Programmers really need to stop this cope about us being such special snowflakes that we can't be assessed and that our maangers just need to take that we're worth keeping around on good faith.
News to me. How do you determine the productivity of a doctor? Patients seen? Patients cured? (for real, where did you get that data?) Number of medicines prescribed? Procedures performed? Does a triple bypass surgery count the same a pap smear? Hours worked? Amount of help they provided to colleagues? Easy to come up with another 100 other metrics that might be worth looking out. How are they all weighted?
Like I get that in SWE (like all other fields), managers have to make judgement calls and try to evaluate which reports contribute the most, but the GP post seemed surprised that this wasn't a solved problem by now, which just seems incomprehensible to me.
> How do you determine the productivity of a doctor?
At the end of the road. Patient outcome and contentedness compared to others with similar indications. Patients seen and all that is that sort of short-term BS that you see everywhere that's giving metrics a bad name. It'd be like determining a mechanic's productivity by how many times he twisted a wrench.
4 replies →
life expectancy, standardized qol metrics. or patients seen, revenue per patients, hours worked etc can be metrics if those _were_ what you wanted. the point is, the answer is yes, they have measures of better/worse docs in their field.
1 reply →
> Yes, yes we can.
Of course we can. But can we do it in a meaningful way, such that the metric itself doesn't become a subject to optimization?
"When a measure becomes a target, it ceases to be a good measure"
> the metric itself doesn't become a subject to optimization
By making the metrics part of a sustaintable company-wide goal. If there's a company-wide goal to increase X kind of revenue by Y% making actionable targets on how a team can contribute (not lazy shit like "our changes should contribute Z% of that Y%"), and within that create for a person another smaller metric based on that.
3 replies →
> Yes, yes we can.
Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion.
> Programmers really need to stop this cope about us being such special snowflakes
Which is not at all what is happening in your parent comment. On the contrary, they’re putting developers on even footing with other professions.
>Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion.
You can look at the kind of work they're doing, how effective their solutions are, and how long it takes them to do it. That's the basics of it across a wide range of professions. Now, there's no one-size-fits-all metric or formula you can just calculate based on objective facts for most of this, because the work is more varied than e.g. factory work, but it's also not impossible to make the comparison, if you actually understand the work reasonably and you use judgement.
In the case of this study, because the assignment of the comparison they were doing was random, then just measuring time to completion across a range of tasks is a perfectly reasonable metric, because there's nothing to really bias the outcome, just a lot of factors that add noise instead. But it is worth noting that the result is a very broad average, and there is likely a very complicated distribution of details underneath, which is much harder to measure.
1 reply →
"Software engineers can be qualitatively assessed for the purposes of pay and promotion" and "software engineers can have their productivity measured and quantified" are two very different things.
So what methods do you use?
[dead]