Comment by layer8
13 hours ago
Labeling people as villains (as opposed to condemning acts), in particular those you don’t know personally, is almost always an unhelpful oversimplification of reality. It obscures the root causes of why the bad things are happening, and stands in the way of effective remedy.
In this case they hadn’t labeled anyone as villains, though. They could have omitted that section entirely.
I happen to agree that labeling them as villains wouldn’t have been helpful to this story, but they didn’t do that.
> It obscures the root causes of why the bad things are happening, and stands in the way of effective remedy.
There’s a toxic idea built into this statement: It implies that the real root cause is external to the people and therefore the solution must be a systemic change.
This hits a nerve for me because I’ve seen this specific mindset used to avoid removing obviously problematic people, instead always searching for a “root cause” that required us all to ignore the obvious human choices at the center of the problem.
Like blameless postmortems taken to a comical extreme where one person is always doing some careless that causes problems and we all have to brainstorm a way to pretend that the system failed, not the person who continues to cause us problems.
> Like blameless postmortems taken to a comical extreme where one person is always doing some careless that causes problems and we all have to brainstorm a way to pretend that the system failed, not the person who continues to cause us problems.
Well, I'd argue the system failed in that the bad person is not removed. The root is then bad hiring decision and bad management of problematic people. You can do a blameless postmortem guiding a change in policy which ends in some people getting fired.
> You can do a blameless postmortem guiding a change in policy which ends in some people getting fired.
In theory maybe, but in my experience the blameless postmortem culture gets taken to such an extreme that even when one person is consistently, undeniably to blame for causing problems we have to spend years pretending it’s a system failure instead. I think engineers like the idea that you can engineer enough rules, policies, and guardrails that it’s impossible to do anything but the right thing.
This can create a feedback loop where the bad players realize they can get away with a lot because if they get caught they just blame the system for letting them do the bad thing. It can also foster an environment where it’s expected that anything that is allowed to happen is implicitly okay to do, because the blameless postmortem culture assigns blame on the faceless system rather than the individuals doing the actions.
3 replies →
Blameless postmortems are for processes where everyone is acting in good faith and a mistake was made and everyone wants to fix it.
If one party decides that they don’t want to address a material error, then they’re not acting in good faith. At that point we don’t use blameless procedures anymore, we use accountability procedures, and we usually exclude the recalcitrant people from the remediation process, because they’ve shown bad faith.
> Well, I'd argue the system failed in that the bad person is not removed.
This is just a proxy for "the person is bad" then. There's no need to invoke a system. Who can possibly trace back all the things that could or couldn't have been spotted at interview stage or in probation? Who cares, when the end result is "fire the person" or, probably, "promote the person".
2 replies →
> There’s a toxic idea built into this statement: It implies that the real root cause is external to the people and therefore the solution must be a systemic change.
Not necessarily, although certainly people sometimes fall into that trap. When dealing with a system you need to fix the system. Ejecting a single problematic person doesn't fix the underlying problem - how did that person get in the door in the first place? If they weren't problematic when they arrived, does that mean there were corrosive elements in the environment that led to the change?
When a person who is a cog within a larger machine fails that is more or less by definition also an instance of the system failing.
Of course individual intent is also important. If Joe dropped the production database intentionally then in addition to asking "how the hell did someone like him end up in this role in the first place" you will also want to eject him from the organization (or at least from that role). But focusing on individual intent is going to cloud the process and the systemic fix is much more important than any individual one.
There's also a (meta) systemic angle to the above. Not everyone involved in carrying out the process will be equally mature, objective, and deliberate (by which I mean that unfortunately any organization is likely to contain at least a few fairly toxic people). If people jump to conclusions or go on a witch hunt that can constitute a serious systemic dysfunction in and of itself. Rigidly adhering to a blameless procedure is a way to guard against that while still working towards the necessary systemic changes.
Often institutions develop fundamental problems because individuals gradually adjust their behaviors away from the official norms. If it goes uncorrected, the new behavior becomes the unofficial norm.
One strategy for correcting the institution is to start holding individuals accountable. The military does this often. They'll "make an example" of someone violating the norms and step up enforcement to steer the institutional norms back.
Sure it can feel unfair, and "everyone else is doing it" is a common refrain, but holding individuals accountable is one way to fix the institution.
I agree with most of what you said but i'd like to raise 2 points
1) the immediate action _is more important immediately_ than the systemic change. We should focus on maximizing our "fixing" and letting a toxic element continue to poison you while you waste time wondering how you got there is counterproductive. It is important to focus on the systemic change, but once you have removed the person that will destroy the organization/kill us all.
2) I forgot. Sorry
1 reply →
> how did that person get in the door in the first place?
is answered by:
> any organization is likely to contain at least a few fairly toxic people
2 replies →
> Ejecting a single problematic person doesn't fix the underlying problem - how did that person get in the door in the first place? If they weren't problematic when they arrived, does that mean there were corrosive elements in the environment that led to the change?
This is exactly the toxicity I’ve experienced with blameless postmortem culture:
Hiring is never perfect. It’s impossible to identify every problematic person at the interview stage.
Some times, it really is the person’s own fault. Doing mental gymnastics to assume the system caused the person to become toxic is just a coping mechanism to avoid acknowledging that some people really are problematic and it’s nobody’s fault but their own.
1 reply →
> There’s a toxic idea built into this statement: It implies that the real root cause is external to the people and therefore the solution must be a systemic change.
It's both obviously. To address the human cause, you have to call out the issues and put at risk the person's career by damaging their reputation. That's what this article is doing. You can't fix a person, but you can address their bad behavior in this way by creating consequences for the bad things.
Part of the root cause definitely is the friction aspect. The system is designed to make the bad thing easier, and when designing a system you need the good outcomes to be lower friction.
> This hits a nerve for me because I’ve seen this specific mindset used to avoid removing obviously problematic people, instead always searching for a “root cause” that required us all to ignore the obvious human choices at the center of the problem.
The real conversations like that take place in places where there's no recordings, or anything left in writing. Don't assume they aren't taking place, or that they go how you think they go.
> Like blameless postmortems taken to a comical extreme where one person is always doing some careless that causes problem
Post-mortems are a terrible place for handling HR issues. I'd much rather they be kept focused on processes and technical details, and human problem be kept private.
Dogpiling in public is an absolutely awful thing to encourage, especially as it turns from removing a problematic individual to looking for whoever the scapegoat is this time.
The prior is stating an extreme case, eg "comical extreme".
One problem is that if you behave as if a person isn't the cause, you end up with all sorts of silly rules and processes, which are just in place to counter "problematic individual".
You end up using "process" as the scapegoat.
I agree, but in this hypothetical situation the HR part needs to happen, despite the fact that most people don't want to be the squeaky wheel that explicitly starts pointing fingers..
It's way too easy to pretend the system is the problem while sticking your head in the sand because you don't want to solve the actual human problem.
Sure, use the post mortem to brainstorm how to prevent/detect/excise the systematic problem ("How do we make sure no one else can make the same mistake again"), but eventually you just need to deal with the repeat offender.
People don’t really understand what this stuff means and create fucked up processes.
In a blame focused postmortem you say “Johnny fucked up” and close it.
When you are about accountability, the responsible parties are known or discovered if unknown and are responsible for prevention/response/repair/etc. The corrective action can incorporate and number of things, including getting rid of Johnny.
This hits the nail on the head. I liken it to a scale or ladder, each rung representing a new level of understanding:
1) Basic morality (good vs evil) with total agency ascribed to the individual
2) Basic systems (good vs bad), with total agency ascribed to the system and people treated as perfectly rational machines (where most of the comments here seem to sit)
3) Blended system and morality, or "Systemic Morality": agency can be system-based or individual-based, and morality can be good or bad. This is the single largest rung, because there's a lot to digest here, and it's where a lot of folks get stuck on one ("you can't blame people for making rational decisions in a bad system") or the other ("you can't fault systems designed by fallible humans"). It's why there's a lot of "that's just the way things are" useless attitudes at present, because folks don't want to climb higher than this rung lest they risk becoming accountable for their decisions to themselves and others.
4) "Comprehensive Morality": an action is net good or bad because of the system and the human. A good human in a bad system is more likely to make bad choices via adherence to systemic rules, just as a bad human in a good system is likely to find and exploit weaknesses in said system for personal gain. You cannot ascribe blame to one or the other, but rather acknowledge both separately and together. Think "Good Place" logic, with all of its caveats (good people in bad systems overwhelmingly make things worse by acting in good faith towards bad outcomes) and strengths (predictability of the masses at scale).
5) "Historical Morality": a system or person is net good or bad because of repeated patterns of behaviors within the limitations (incentives/disincentives) of the environment. A person who routinely exploits the good faith of others and the existing incentive structure of a system purely for personal enrichment is a bad person; a system that repeatedly and deliberately incentivizes the exploitation of its members to drive negative outcomes is a bad system. Individual acts or outcomes are less important than patterns of behavior and results. Humans struggle with this one because we live moment-to-moment, and we ultimately dread being held to account for past actions we can no longer change or undo. Yet it's because of that degree of accountability - that you can and will be held to account for past harms, even in problematic systems - that we have the rule of law, and civilization as a result.
Like a lot of the commenters here, I sat squarely in the third rung for years before realizing that I wasn't actually smart, but instead incredibly ignorant and entitled by refusing to truly evaluate root causes of systemic or personal issues and address them accordingly. It's not enough to merely identify a given cause and call it a day, you have to do something to change or address it to reduce the future likelihood of negative behaviors and outcomes; it's how I can rationalize not necessarily faulting a homeless person in a system that fails to address underlying causes of homelessness and people incentivized not to show empathy or compassion towards them, but also rationalize vilifying the wealthy classes who, despite having infinite access to wealth and knowledge, willfully and repeatedly choose to harm others instead of improving things.
Villainy and Heroism can be useful labels that don't necessarily simplify or ignorantly abstract the greater picture, and I'd like to think any critically-thinking human can understand when someone is using those terms from the first rung of the ladder versus the top rung.
I'm not sure the problems we have at the moment are a lack of accountability. I mean, I think let's go a little overboard on holding people to account first, then wind it back when that happens. The crisis at the moment is mangeralism across all of our institutions which serves to displace accountability .
Labeling people as villains used to be effective deterrence against doing villainous things. When did that change?
When we began blaming society instead.
I've read multiple times that a large percentage of the crime comes from a small group of people. Jail them, and the overall crime rate drops by that percentage.
Which group is that?
5 replies →
It's also pretty clearly a deterrence against people admitting and fixing their own mistakes, both individually and as institutions. Which is exactly what we're seeing here...
You wouldn't be a villain from doing one bad thing, but a pattern.
Correlation is not causation.
Was it ever, though? This is an easy thing to say, but how would we demonstrate that it worked?
Ah yes, the mythical past when nobody did bad things because we punished them correctly.
The crime rate does change dramatically over time. For example, the homicide rate during the pandemic was about double what it is today.
1 reply →
Questions:
1. Who is responsible for adding guardrails to ensure all papers coming in are thoroughly checked & reviewed?
2. Who review these papers? Shouldn’t they own responsibility for accuracy?
3. How are we going to ensure this is not repeated by others?
There needs to be prestige for tearing down heavily flawed work.
reviewers are unpaid. its also quite common to farm out the actual review work to grad students, postdocs and the like. if you're suggesting adding liability, then you're just undermining the small amount of review that already takes place.
Just to add on, armchair quarterbacking is a thing, it’s easy in hindsight to label decisions as the result of bad intentions. This is completely different than whatever might have been at play in the moment and retrospective judgement is often unrealistic.
Every single comment on every thread on this entire website is armchair quarterbacking. It's completely obvious that this is dishonest bad work.
It is possible that the root cause is an individual person being bad. This hasn't been as common recently because people were told not to be villains and to dislike villains, so root causes of the remaining problems were often found buried in the machinery of complex social systems.
However if we stop teaching people that villains are bad and they shouldn't be villains, we'll end up with a whole lot more problems of the "yeah that guy is just bad" variety.
As with anything, it's just highly subjective. What some call an heinous act is another person's heroic act. Likewise, where I draw the line between an unlucky person and a villain is going to be different from someone else.
Personally, I do believe that there are benefits to labelling others as villains if a certain threshold is met. It cognitively reduces strain by allowing us to blanket-label all of their acts as evil [0] (although with the drawback of occasionally accidentally labelling acts of good as evil), allowing us to prioritise more important things in life than the actions of what we call villains.
[0]: https://en.wikipedia.org/wiki/Halo_effect#The_reverse_halo_e...
It’s not really subjective if you don’t believe it’s your place to judge the human to begin with.
If you were in their exact life circumstance and environment you would do the same thing. You aren’t going to magically sidestep cause and effect.
The act itself is bad.
The human performing the act was misguided.
I view people as inherently perfect whose view of life, themselves, and their current situations as potentially misguided.
Eg, like a diamond covered in shit.
Just like it’s possible for a diamond to be uncovered and polished, the human is capable of acquiring a truer perspective and more aligned set of behaviors - redemption. Everyone is capable of redemption so nobody is inherently bad. Thinking otherwise may be convenient but is ultimately misguided too.
So the act and the person are separate.
Granted, we need to protect society from such misguidedness, so we have laws, punishments, etc.
But it’s about protecting us from bad behavior, not labeling the individual as bad.
> If you were in their exact life circumstance and environment you would do the same thing.
I don't buy that for a moment. It presumes people do not have choices.
The difference between a man and an animal is a man has honor. Each of us gets to choose if we are a man or an animal.
1 reply →
> The human performing the act was misguided.
What does this mean? If someone rapes someone else, they were inherently perfect but misguided, in your view?
Bad acts are in the past, and may be situational or isolated.
Labelling a person as bad has predictive power - you should expect them to do bad acts again.
It might be preferable to instead label them as “a person with a consistent history of bad acts, draw your own conclusion, but we are all capable of both sin and redemption and who knows what the future holds”. I’d just call them a bad person.
That said, I do think we are often too quick to label people as bad based one bad act.
The person is inseparable from the root cause.
I'm guessing you believe that a person is always completely responsible for their actions. If you are doing root cause analysis you will get nowhere with that attitude.
There’s many ways that people can fail where they aren’t the root cause.
These failures aren’t on that list because they require active intent.
In the case of software RCA, but if a crime is committed then many times there is a victim. There could be some root cause, but ignoring the crime creates a new problem for the victim (justice)
Both can be pursued without immediately jumping to defending a crime
Then "root cause" means basically nothing
I hope you don't work in technology. If you do, I hope I never work with you.
Blameless post-mortems are critical for fixing errors that allowed incident to happen.
In that case let's just shut down the FAA and any accident investigations.
It's not processes that can be fixed, it's just humans being stupid.
I'm not a bad person, I just continuously do bad things, none of which is my fault - there is always a deeper root cause \o/
On the flip side, even if you punish the villain, garbage papers still get printed. Almost like there is a root cause.
Both views are maximalistic.
On the flop side, maybe there wouldn't be as many garbage papers printed if there were any actual negative consequences. It's not so simple as you make it out to be.
3 replies →
I would argue that villainy and "bad people" is an overcomplication of ignorance.
If we equate being bad to being ignorant, then those people are ignorant/bad (with the implication that if people knew better, they wouldn't do bad things)
I'm sure I'm over simplifying something, looking forward to reading responses.
What if the root cause is that because we stopped labeling villains, they no longer fear being labeled as such. The consequences for the average lying academic have never been lower (in fact they usually don’t get caught and benefit from their lie).
Actually the risks for academic misconduct have never been higher. For quite a while now there's been borderline activism to go out and search the literature for it - various custom software solutions have been written specifically to that end. We're also rapidly approaching a reality in which automated cross checking of the literature for contradictions will be possible.
Unfortunately academia as a pursuit has never had a larger headcount and the incentives to engage in misconduct have likely never been higher (and appear to be steadily increasing).
Are we living on the same planet?
Surely the public discourse over the past decades has been steadily moving from substantive towards labeling each other villains, not the other way around.
But that kind of labeling happens because of having the wrong political stances, not because of the moral character of the person.
4 replies →
For activist, politicians scientists, civilians? be specific
It’s possible to take two opposing and flawed views here, of course.
On the one hand, it is possible to become judgmental, habitually jumping to unwarranted and even unfair conclusions about the moral character of another person. On the other, we can habitually externalize the “root causes” instead of recognizing the vice and bad choices of the other.
The latter (externalization) is obvious when people habitually blame “systems” to rationalize misbehavior. This is the same logic that underpins the fantastically silly and flawed belief that under the “right system”, misbehavior would simply evaporate and utopia would be achieved. Sure, pathological systems can create perverse incentives, even ones that put extraordinary pressure on people, but moral character is not just some deterministic mechanical response to incentive. Murder doesn’t become okay because you had a “hard life”, for example. And even under “perfect conditions”, people would misbehave. In fact, they may even misbehave more in certain ways (think of the pathologies characteristic of the materially prosperous first world).
So, yes, we ought to condemn acts, we ought to be charitable, but we should also recognize human vice and the need for justice. Justly determined responsibility should affect someone’s reputation. In some cases, it would even be harmful to society not to harm the reputations of certain people.
What specific pathologies characteristic of the materially prosperous first world? People almost universally behave better in a functional system with enough housing food education and so forth. Morality is and will always remain important but systems matter a LOT. For instance we've experienced less murder since we stopped mass lead poisoning our entire population.
It's a paradox. We know for an absolute fact that changing the underlying system matters massively but we must continue to acknowledge the individual choice because the system of consequences and as importantly the system of shame keeps those who wouldn't act morally in check. So we punish the person who was probably lead poisoned the same as any other despite knowing that we are partially at fault for the system that lead to their misbehavior.
You presumably read the piece. There was no remedy. In fact the lavishly generous appreciation of all those complexities arguably is part of the reason there was no remedy. (Or vice versa, i.e. each person's foregone conclusion that there will be no remedy for whatever reason, might've later been justified/rationalized via an appeal to those complexities.)
The act itself, of saying something other than the truth, is always more complex than saying the truth. ← It took more words to describe the act in that very sentence. Because there are two ideas, the truth and not the truth. If the two things match, you have a single idea. Simple.
Speaking personally, if someone's very first contact with me is a lie, they are to be avoided and disregarded. I don't even care what "kind of person" they are. In my world, they're instantly declared worthless. It works pretty well. I could of course be wrong, but I don't think I'm missing out on any rich life experiences by avoiding obvious liars. And getting to the root cause of their stuff or rehabilitating them is not a priority for me; that's their own job. They might amaze me tomorrow, who knows. But it's called judgment for a reason. Such is life in the high-pressure world of impressing rdiddly.
> Labeling people as villains is almost always an unhelpful oversimplification of reality
This is effectively denying the existence of bad actors.
We can introspect into the exact motives behind bad behaviour once the paper is retracted. Until then, there is ongoing harm to public science.
IMHO, you should deal with actual events, when not ideas, instead of people. No two people share the exact same values.
For example, you assume that guy trying to cut the line is a horrible person and a megalomaniac because you've seen this like a thousand times. He really may be that, or maybe he's having an extraordinarily stressful day, or maybe he's just not integrated with the values of your society ("cutting the line is bad, no matter what") or anything else BUT none of all that really helps you think clearly. You just get angry and maybe raise your voice when you're warning him, because "you know" he won't understand otherwise. So you left your values now too because you are busy fighting a stereotype.
IMHO, correct course of action is assuming good faith even with bad actions, and even with persistent bad actions, and thinking about the productive things you can do to change the outcome, or decide that you cannot do anything.
You can perhaps warn the guy, and then if he ignores you, you can even go to security or pick another hill to die on.
I'm not saying that I can do this myself. I fail a lot, especially when driving. It doesn't mean I'm not working on it.
I used to think like this, and it does seem morally sound at first glance, but it has the big underlying problem of creating an excellent context in which to be a selfish asshole.
Turns out that calling someone on their bullshit can be a perfectly productive thing to do, it not only deals with that specific incident, but also promotes a culture in which it's fine to keep each other accountable.
2 replies →
I honestly think this would qualify as "ruinous empathy"
It's fine and even good to assume good faith, extend your understanding, and listen to the reasons someone has done harm - in a context where the problem was already redressed and the wrongdoer is labelled.
This is not that. This is someone publishing a false paper, deceiving multiple rounds of reviewers, manipulating evidence, knowingly and for personal gain. And they still haven't faced any consequences for it.
I don't really know how to bridge the moral gap with this sort of viewpoint, honestly. It's like you're telling me to sympathise with the arsonist whilst he's still running around with gasoline
2 replies →
When bad behavior has been identified, reported, and repeated - as described in the article - it is no longer eligible for a good faith assumption.
I think they're actually just saying bad actors are inevitable, inconsistent, and hard to identify ahead of time, so it's useless to be a scold when instead you can think of how to build systems that are more resilient to bad acts
You have to do both. Offense and defense are closely related. You can make it hard to engage in bad acts, but if there are no penalties for doing so or trying to do so, then that means there are no penalties for someone just trying over and over until they find a way around the systems.
Academics that refuse to reply to people trying to replicate their work need to be instantly and publicly fired, tenure or no. This isn't going to happen, so the right thing to do is for the vast majority of practitioners to just ignore academia whilst politically campaigning for the zeroing of government research grants. The system is unsaveable.
4 replies →
To which my reply would be, we can engage in the analysis after we have taken down the paper.
It's still up! Maybe the answer to building a resilient system lies in why it is still up.
One thing that stands in the way of other people choosing the wrong path is the perception of consequences. Minimal consequences by milquetoast critics who just want to understand is a bug not a feature.
People are on average both bad and stupid and function without a framework of consequences and expectations where they expect to suffer and feel shame. They didn't make a mistake they stood in front of all their professional colleagues and published effectively what they knew were lies. The fact that they can publish lies and others are happy to build on lies ind indicates the whole community is a cancer. The fact that the community rejects calls for correction indicates its metastasized and at least as far as that particular community the patient is dead and there is nothing left to save.
They ought to be properly ridiculed and anyone who has published obvious trash should have any public funds yanked and become ineligible for life. People should watch their public ruin and consider their own future action.
If you consider the sheer amount of science that has turned out to be outright fraud in the last decade this is a crisis.
That comment sounds like the environment causes bad behavior. That's a liberal theory refuted consistently by all the people in bad environments who choose to not join in on the bad behavior, even at a personal loss.
God gave us free will to choose good or evil in various circumstances. We need to recognize that in our assessments. We must reward good choices and address bad ones (eg the study authors'). We should also change environments to promote good and oppose evil so the pressures are pushing in the right direction.