Comment by nonrandomstring
4 years ago
Many years ago a colleague who works in defence told me about a job posting he'd seen but was having a moral struggle with.
The opening was for "Lethality Engineer": Ideal candidate with good physics and medical background.
I said that the main perk was that at least on Halloween he wouldn't need to buy a costume. He could just go out as himself.
He didn't take the job.
I hope recent events have illustrated that if it weren't for the people who develop lethal weapons, we (as in you and I) would be helpless against the bullies of the world. Unilateral pacifism is cute philosophy only when there are rough men standing ready to do violence on their behalf.
I strongly agree with this sentiment. However, it is hard for an ethical person to participate in developing war technology when possession and usage of the weapons is purely a political question, and history also has seen our side of geopolitics commit atrocities.
My stance has previously been that I am unwilling to work on weapons technology, because history has shown that these weapons sometimes end up being used for an indefensible cause. Then all of a sudden you're an accomplice to murder, and getting away with it.
In the light of Russia's invasion of Ukraine, which is just a continuation of its historical imperialism, working on weapons is something I would be perfectly okay with and probably even motivated to do. But stop for a moment and think what a history of aggressive military actions does to our society's ability to recruit for this important job.
I was a reviewer for a book on ethical machine learning that wasn’t published. I’ll never forget, the author stated “don’t work on anything that could cause harm.” Here I am reading this while working in defense being like “that’s a lazy and dumb position.” Nearly anything in the wrong hands could cause harm.
It’s not unethical to work in the auto industry because people can die in car accidents. It’s not unethical to work in the beer business because people can become alcoholics. It’s not unethical to work for a credit card company because people can bury themselves in debt. And it’s not unethical to work in defense because the weapons may fall into the wrong hands.
What’s unethical is encouraging these problems and not trying to prevent them. And yeah, it’s hard to navigate these ethical issues, but we’re professionals like doctors and lawyers and part of the reason we get paid like we do is because we may have to wrestle with these issues.
6 replies →
You can be happily manufacturing weapons for a good cause today only to see your government turn evil the next day. Unfortunately people don't cluster around ideas but geography and that's is out of control for most.
This does not mean we shouldn't do something but we have to realize nothing is permanent and the fruits of our labor can very well be misused the next day.
Like anything difficult there are real risks and trade-offs, but just refusing to engage in difficult pragmatic issues is not the ethical position imo, it's just the easy one that feels good. It puts the burden of actual complex ethical decisions onto other people.
The west needs the capability to defend the ideals of classical liberalism and individual liberty. In order to do that it needs a strong military capability.
https://zalberico.com/essay/2020/06/13/zoom-in-china.html
4 replies →
The flaw in that logic is that, if it weren't for the people who develop lethal weapons for the bullies, we wouldn't have to fear the bullies.
Also, I think the design space of "radical defense" is under explored. Our (western) armies are still designed for attack and force projection, although we have long since renamed our war ministers secretaries of defense.
But I wonder if you could develop defense capability to make your country unattackable. Not by threat of retaliation, but for example by much much stronger missile defense. Or by educating ("indoctrinating") your own population, so that an occupier would not find a single collaborator? Or by mining your own infrastructure, and giving every citizen basic combat training (a bit like the swiss)? Or by fostering a world-wide political transformation that is designed to prevent wars from happening at all?
I think if we wanted to spend money researching stuff to keep us safe, it doesn't necessarily have to be offensive weapons.
The flaw in this logic is somewhat related to law enforcement, in that if your military is min/maxed for defense, someone who wants to do you harm only has to be right once in order to actually do you harm. Looking at nuclear weapons and missile defense (ignoring the existence of dirty bombs etc.), your opponent needs to only be right once for one of your cities and hundreds of thousands of civilians to be gone. And likewise, if you've focused on defense you're likely wholly unprepared for any sort of retaliation.
The Swiss approach what with literally bunkering in the mountains and everything is interesting, but the logistics for larger countries would be exponentially harder (and most lack the geographic help). "Fostering world-wide political transformation" is so pie in the sky it's honestly not worth serious discussion. It's fanciful.
Someone will always be willing to make weapons for the bullies because a lot of people don't view them as bullies in the first place. Ask people in Iraq, or Chechnya, or Ireland, or Pakistan, or Taiwan, who the bullies are, and you'll get wildly different answers that will cover approximately 90% of the worldwide population.
> The flaw in that logic is that, if it weren't for the people who develop lethal weapons for the bullies, we wouldn't have to fear the bullies.
You can't uninvent weapons, and you can't prevent the bullies from making their own weapons.
The problem with an impenetrable defensive shield is that it gives your potential enemies the heebie-jeebies (technical geopolitical term) that, now that you have the shield, you can attack them without fear of reprisal. If the enemy thinks you're working on a credible shield (or even a shield you think is credible) their best option is to attack now before you, emboldened by your sense of invulnerability, attack them.
1 reply →
> The flaw in that logic is that, if it weren't for the people who develop lethal weapons for the bullies, we wouldn't have to fear the bullies.
False. Bullies are a problem even if no one has weapons beyond what can be grabbed and used from the environment without any invention. Heck, bullies are a problem if everyone just has the weapons built in to their bodies.
> But I wonder if you could develop defense capability to make your country unattackable.
Not without incidentally developing a huge edge in offensive weapons that would make you attackable when it inevitably diffused to others. Uniquely defensive technology mostly doesn't exist.
> Not by threat of retaliation, but for example by much much stronger missile defense.
Much better interceptor missiles mean the technology for much better missiles generally. Directed energy interception means direct energy weapons. Hypervelocity kinetic interceptors are general purpose hypervelocity kinetic weapons.
> Or by educating ("indoctrinating") your own population, so that an occupier would not find a single collaborator?
That kind of indoctrination can also be used offensively, but the enemy doesn't need collaborators to attack you. (They might need it to conquer without genocide, but attackers willing to commit genocide for land are not unheard of, nor are attackers whose goal isn't conquest.)
> Or by mining your own infrastructure, and giving every citizen basic combat training (a bit like the swiss)?
Mining your infrastructure is itself creating a vulnerability to certain kinds of attacks.
> Or by fostering a world-wide political transformation that is designed to prevent wars from happening at all?
It's been tried, repeatedly. The League of Nations, the Kellogg-Briand Pact, the UN. It’ll be nice if someone ever finds the “one wired trick to prevent war forever”, but it seems distinctly improbable and particularly suicidal to bank your defense on the ability to find it.
Especially since we cannot really help Ukraine with anything but tech and they are outnumbered, so the only advantage we can give is how much better our weapons are than Russian ones.
Have you considered, though, that we (as in you and I) might be some of these bullies in the world and that these rough men aren't just standing by, but are actively doing violence on our behalf ? One needn't look much further past recent events to find examples aplenty.
I understand the point you are trying to make, but it's not as easy as pretending that the weapons "we" develop are purely for morally and ethically righteous purpose.
With respect you may be making some unfounded assumptions about what I've built and what I believe.
My point was really about the fact that this job title "Lethality Engineer" actually exists. And moreover, that it asked for medical qualifications, which would go against any doctor's Hippocratic oath.
Most of us who've done defence related work are happy at the edges, with tactical information systems, coms or guidance (my stuff ended up in targeting).
But when it comes down to figuring out how fragments can be arranged around a charge to make sure the waveshape optimally penetrates as many nearby skulls as possible... hmmm suddenly not so gung-ho about it.
That's not a distant, theoretical morality about tyrants and bullies. I've no problems contemplating my family's military history and am plenty proud of it, even though we'd all rather live in a world without this stuff.
Until the lethal weapons are turned on those (countries, groups, people) who develop them. Kind of like gun owners are more likely to be harmed (or harm others) by their own guns, notwithstanding the arguments about personal protection used to justify such ownership.
This has already happened with groups the US has armed in the past. The US itself has been the bad guy sometimes.
There is no proper resolution to this struggle, and people who are guided by their conscience should not be attacked for having a "cute philosophy" that relies on "rough men standing ready to do violence on their behalf."
A sibling comment put it well that refusing to wrestle with these important questions is the unethical position as it just pushes the decision off onto other people. "Cute philosophy" is a perfect way to describe that because it's completely untenable if everyone were to think that way.
The gun thing is completely tautological though. Yes, if you have a gun you're more likely to be injured by your gun than someone who doesn't. How would someone who doesn't own a gun be injured by their own guns in the first place? It's like saying you're more likely to be in a car accident if you own a car. Of course you are.
1 reply →
I generally agree with this, however, the rough men willing to do violence on our behalf are more and more becoming the quirky scientists who are very disconnected from the actual impact of their work. I think there's a big difference between those types of people. It seems like people don't feel the weight of violence as much as they used to. I imagine this will increase as we develop more AI driven weapons.
This doesn't square with my experience in defense. I worked in software and we saw plenty of combat and aftermath footage and were always aware that the design decisions we made and the tools we built meant life-and-death for someone. We did our best to make sure it was the right people.
I'd add, the weight of violence—if anything—is going up. People today are devastated when a dozen soldiers and scores of civilians are killed in a suicide bombing or urban conflict, but go back to Vietnam and those incidents barely register because they happened all the time. The number of people killed in any given armed conflict has dropped quite a bit in the last 50-or-so years. (The Syrian Civil War is one big exception.)
1 reply →
https://en.wikipedia.org/wiki/A_Taste_of_Armageddon
This comment is confusing two completely separate things. There's a world of difference between not being willing to defend yourself and actively trying to come up with more aggressive and lethal weapons.
The argument that "we need defense!" only justifies the need to stockpile and develop _sufficiently_ lethal and tactical weaponry to neutralize incoming threats (like anti-ballistic systems). It doesn't justify inventing deadlier weapons. No dispossessed victim of foreign invasion has ever needed a bioweapon to assert themselves, and there's no chance developing one would ever be used for anything but war crimes. You should absolutely turn down roles like "Lethality Engineer" from an ethical standpoint, even if you agree military defense is necessary.
People raise the spectre of deterrence as a utilitarian justification for needing more powerful weapons ("har har, they'll think twice about attacking us if they know we have nukes!"). But that's narrow thinking. Deterrence can be achieved in other less-damning ways, like strategic alliances and building more robust defense systems.
tl;dr defence != deadlier offence.
We are acting pretty helpless because the bully has nuclear weapons.
> Many years ago a colleague who works in defence told me about a job posting he'd seen but was having a moral struggle with.
This is a good struggle to have. What's ironic in many cases is that we don't experience these quandaries in other jobs, but the ethical and moral ramifications still exist. The early days of search in Google or social in Facebook probably didn't elicit the same kind thought process as a lethality engineering post. (Anecdotally I spoke some years ago with an acquaintance Googler who told me that he enjoyed working there precisely because he was working on privacy issues that worked against some of the advertising side of the business.)
I've worked in telecommunications, industrial systems engineering, and energy. There are ethical and moral issues in the work that I've done/do as a contributor in each of those domains, even though I'm not involved day-to-day in decision making that feels particularly moral.
One of the base assumptions we probably need to make in our work is that whatever we do will always be misused in the worst possible way. If we explore that idea, it might give us some sense for how to structure our output to curtail the worst of the damages.
> The early days of search in Google or social in Facebook probably didn't elicit the same kind thought process as a lethality engineering post.
It did for at least one person (me). I was 16 in 2004 with 11 years of dev experience, trying to decide whether to go out to SV, go to college for CS, or do something else. I was from the same city/community as Larry Page and in Zuck's age group, so it wasn't an absurd consideration to try. Lots of things went into my decision to do something non-CS related for college, but morals were one of the reasons I didn't go to SV (I objected to the professionalization of the web + Zuck creeped me out + I didn't agree with cutting out humans/curators from the search process like Google did).
It's just that until very recently, people either thought I was lying OR that I was just batshit insane. Who is invited to a gold rush and doesn't go?
I can't imagine I was the only one.
> I can't imagine I was the only one.
I'm sure not, and hopefully the description I provided isn't a blanket one. And, to be clear, I'm also not trying to say that working for any of those organizations is per se unethical. I don't think that this is the case.
The point, rather, is that ethical and moral considerations are actually much nearer to us than might appear at first blush. Sometimes this happens by the mere nature of the work (killing people more efficiently) and sometimes by scale (now when we surface search results, we make direct impacts on what people learn, where they shop, how they receive advertisements, etc., none of which was true in 1999). Navigating this isn't easy (indeed, you can make an argument that there is a morally good outcome for killing people more efficiently; I'm not saying it's necessarily a good one, but that one can be made), but we don't routinely equip people to think about it.
To make matters worse, our cultural assumptions shift over time. The Google/Facebook difference is illustrative. Page and Brin are a generation older than Zuckerberg, and their assumptions about what it means to be moral are probably not the same. These assumptions also change based on circumstance--when we scale a business from a garage to a billion dollars, it's hard to maintain the True North on your moral compass (assuming such a thing exists).
Anyway, I think a deep skepticism about human nature and the utility of technology is probably very useful in these situations.
But is the world better off if moral people avoid immoral jobs?
I believe the world shows there is plenty enough supply of talented people that are willing to do immoral jobs. So removing yourself from the pool of candidates makes little difference.
Alternatively, one could work in an immoral job and make a difference from the inside.
Why not do that? Perhaps to feel impotently virtuous, or perhaps the work couldn’t be stomached by the virtuous, or perhaps the virtuous but weak are scared of losing their virtuousness...
I think you were hard on him. There should be no ethical qualms when our weapons are used on enemies who seek to kill us or attack our interests.
Also, if an ethical person doesn’t take this job, someone far more unethical probably will. And they will raise no objections if they should ever be necessary. Kind of like how a lot of bad people become police officers when no one good wants to do it.
I can relate to that. In my career I stepped out and into defense, and it never really bothered me that much to be honest. But then it was always things like fighter jets and helicopters sold to NATO members, I never had to rationalize that we build the weapons carrier and not the, e.g., missiles that actually cause harm.
I always drew the line at small arms so. Way more people die because of those, they end up in every conflict and there have been too many scandals of those smalls arms manufacturers circumventing export restrictions. Quite recently I added supporting countries like Saudi and the UAE to that list, even the job would have been really interesting, providing highly sophisticated training services to the Saudis is nothing I could do and still look myself in mirror. And civil aerospace is fun as well.
I worked in defense too, might go back. When I get calls I'm like "I don't do work for the Saudi's or the DEA" and half the time the recruiter is like "Uh, I said this job is for Raytheon."
"Yeah, but who's their client?"
One way or the other, regardless of the company, properly one if not both of those countries. Sure, those countries are rich, I just hope that Ukraine showed us in the Democratic west that certain values, like human rights, shouldn't be compromised upon, which we all did in the last decades.
I do understand why those companies chase Saudi and UAE contracts, that's where the money is. Maybe that changes if NATO members increase defense spending, it would be a nice side effect, wouldn't it?
1 reply →
A friend is a very good university lecturer in physics, and a pacifist. He isn't particularly please about the fact that a decent number of his students will turn the particular lessons he teaches towards the production of weapons.
Lethality Engineer? Is that a P.Eng. kind of position? If your work doesn't actually kill anybody, could you be sued for malpractice and lose your license?
They don't take anybody, the interview is murder.