Comment by Waterluvian
14 hours ago
> what the engineers thought when doing design reviews for a "selective stand down" feature.
Possibly a version of, “I lack the freedom to operate with a moral code at work because I’m probably replaceable, the job market makes me anxious, my family’s well-being and healthcare are tied to having a job, and I don’t believe the government has my back.”
From my experience, it’s more likely that the engineers who got far enough in the company to be working on this code believed that their willingness to work on nefarious tasks that others might refuse or whistle-blow made them a trusted asset within the company.
In industries like this there’s also a mindset of “Who cares, it’s all going to corporations anyway, why not send some of that money to the corporation that writes my paychecks?”
I have noticed that in addition to this perspective there are scores of developers who espouse the idea that “we just create, what people do with our work isn’t our business.”
I understand the utilitarian qualities of the argument, but I submit that there’s a reason that capital-E-Engineering credentials typically require some kind of education in ethics-in-design.
> but I submit that there’s a reason that capital-E-Engineering credentials typically require some kind of education in ethics-in-design.
Or said differently: there’s a reason why software engineering jobs pay so well; no mandatory ethics training required!
capital E engineers have numerous other laws that protect their position.
Civil/mechanical/electrical have countless codes that must be followed with the force of law.
When we say we want engineering standards for software developers we are also asking for standards and codes to be applied to software and all that entails.
I'm not saying this is good or bad, just to consider the ramifications of this at all levels.
I suspect you are right. It reminds me of the whole "at the government you can hack legally" argument used by government intelligence agencies to recruit hackers.
I think a lot of skilled engineers want interesting challenges where they break boundaries, and being in an environment that wants you to break those boundaries allows them to legitimize why they are doing it. That is, "someone else is taking moral responsibility, so I can do my technical challenge in peace"
Do you know of anyone declining to work on a project For ethical in their view ( non military non killing) ?
I’ve led a sheltered life and never met one, people have told me they wouldn’t apply for a role with a company for ethical reasons maybe they even believed they would get the job
Sure: A couple of years ago I joined a company doing outsourced system administration. Then it was suggested I should take care of a new client: a manufacturer of weapons with a quite shady reputation. There were already other issues I had noticed. But this was the red flag for me and I left after four weeks. My then team lead was pissed and complained I should have told beforehand that I don’t want to go down that route. But it never occured to me before that to compile a blacklist of things I won’t do. And I had been in business for more than 20 years when that happened.
Hello. I have. The first time, I was offered a job working on missile guidance systems. I told them I would not work on weapons, so they offered me a job working on something else instead. Then they asked me to move to another project that would require getting government security clearance. I said I wouldn't do that either because I was not willing to make the required promises to my government, so they gave me other projects that didn't require it. It's really not that hard to have a penny's worth of a moral compass if your skill has any kind of value. I think maybe the problem is people who only have value to companies that only hire people without any morals.
I know a lot of people who won't work for some companies for ethical reasons.
Though, sometimes the exact reason is muddied, since companies that are perceived as unethical in how they behave externally are often also perceived as unethical in how they behave towards employees. So you might object on pragmatic grounds of how you'd be treated, before you ever get to, say, altruistic grounds.
Also, sometimes fashion is involved. For example, many people wouldn't work for company X, because of popular ethical objections to what they do being in the news, but some of those people would probably work for an unknown company doing the same things, without thinking much about it.
But often it's just "I don't like what company Y is doing to people, and I wouldn't work on that, even if they treated employees really well, and it was really fashionable to work there".
(See, for example, the people who refused to work for Google after the end of Don't Be Evil honeymoon phase, even though they generally treated employees pretty well, and it was still fashionable to work there.)
I've often been contacted by recruiters for companies in the gambling (in India it's called "skill-based web gaming") or the crypto/web3 space, and I've always denied those for ethical reasons.
Sure, it happens all the time. Speaking personally, for example, I walked out of an interview when I realised it was for The Sun's betting site (Sun Bet)
I worked at LivingSocial back in 2012. I was 21 and didn’t know anything about marketing. The pitch was that daily deals helped small businesses get new customers who would then become recurring, which was good. I liked helping small businesses.
Over time I realized that the company knew this wasn’t really true. Daily deal customers weren’t likely to return. They went where the deals were. The influx of cash from daily deals was a marketing expense, almost always at a loss (most deals were 50%+ off and half of the remaining revenue went to LivingSocial), and buyers rarely returned so SMBs would never recoup their loses.
Once I figured this out, I decided to leave even though I would miss my equity cliff by a month. I ended up joining ZenPayroll (now Gusto) early on because they were helping SMBs with a real problem (payroll was a fking nightmare back then.)
1 reply →
Yes! I once met a highly paid contract tech lead who had walked out of a lucrative contract with a supermarket after he became aware the new credit card product he was working on was to be exclusively targeted at customers in poor areas.
The moral fortitude on that man!
I applaud his actions, but genuinely do not know if I would have the stones to leave my job if I was in a similar position!
>Do you know of anyone declining to work on a project For ethical in their view ( non military non killing) ?
o/
i was offered a high paying job, with relocation to a 1st world country (at the time, i was living in a 3rd world country with high murder rates), to a industry that i consider quite shady (and it's not military and not around killing -- i have no issues with both of those). i politely refused.
most of my friends, at the time, told me that they would've have accepted without even thinking, but for me, it's just not worth it.
I know lots of people who had the offer to work in gambling but chose not to take it for moral reasons
1 reply →
I think most people avoid this situation one step earlier by choosing the company they work for. I.e. do you accept a job in adtech, military, adult industry, etc.
I think pretty much everyone has an internal red line, of course they will vary a lot and may even move over time.
Well kinda trivially, asides from secular ethics, you'll find that typical Muslims decline a number of jobs/projects for ethical reasons.
This is a real long look in the mirror moment.
I quit a job on contract with a major insurance provider because they asked me to perform a truncate instead of a rounding operation in a formula without any mathematically sound reason for choosing the truncate over the rounding. I figured out they wanted truncating because it would lead to more people being denied flood coverage than rounding would.
I like the idea that what makes someone a 'professional' instead of just an employee is the wherewithal, agency, and expectation to say no to a particular task or assignment.
An architect or engineer is expected to signal and object to an unsafe design, and is expected by their profession (peers, clients, future employers) to refuse said work even if it costs them their job. This applies even to professions without a formalized license board.
If you don't have the guts and ability to act ethically (and your field will let you get away with it), you're just a code monkey and not a professional software developer.
Maybe when the government and the shareholders start setting an example and hold the bosses and capital owners accountable, and reward instead of punish the whistleblowers, and when their are enough jobs so that losing the one you have is not a problem, moral behavior further down the hierarchy will improve.
This is why we need Professional Engineer licenses for software.
There are times when a product design needs to be reviewed and approved by someone who cares more about his license than about his job. It doesn't happen as often with software as it does with civil engineering, but often enough that it needs to become a thing.
And what happens when the licensing board gets politically compromised? You cant fix broken incentives by papering over another layer of administration. If the underlying incentives are opposed, the administration layer will be adapted to fit.
Civil engineering licensing works because underneath it all the incentive structure is aligned with the goals of the license. Its not about imposing morals, its about ensuring that buildings and devices are constructed to not fail, and to not fail catastrophically. The motivations of the ones who hire engineers are mostly aligned, they don't want the devices to fail either, and expose them to liability.
Medical doctor licensing also works because the incentives are mostly for patients not to be dying. But in the pharmaceuticals industry the incentive structure is different, where some rate of fatality is considered an acceptable cost of doing business, we see examples of subversion.
Sure software engineering licenses could be a great addition. But alone it will fail unless the incentive structure for those employing software engineers is aligned with the licensing goals.
This works great inside a country where said software must be written in the country for compliance reasons...
How does is work for a fungible product that can be written anywhere and shipped at the speed of light?
It doesn't, but at some point a mature industry + society decides to make that tradeoff.
We can't have it both ways: be essential digital infrastructure, AND move at "the speed of light".
Those poor guards working in the concentration camps in nazi germany just wanted job security. They can’t be blamed for their actions.
In my experience, sometimes your employer blatantly lies to you about what you're making and how it'll be used. I was once recruited to work on a software installer which could build and sign dynamic collections of software which was meant to be used to conveniently install several packages at once. Like, here's a set of handy tools for X task, here are the default apps we install on machines for QA people, here is our suite of apps for whatever. It seemed to have genuine utility because it could pull data in real time to ensure it was all patched and current and so on. That could be great for getting new machines up and running quickly. Several options exist for this use case today, but didn't then as far as I recall. This was on Windows.
Ultimately it was only used to install malware in the form of browser extensions, typically disguised as an installer for some useful piece of software like Adobe Acrobat. It would guide you through installing some 500 year old version of Acrobat and sneakily unload the rest of the garbage for which we would be paid, I don't know, 25 cents to a couple dollars per install. Sneaking Chrome onto people's machines was great money for a while. At one point we were running numbers of around $150k CAD per day just dumping trash into unsuspecting people's computers.
At no point in the development of that technology were we told it was going to ruin countless thousands of people's browsers or internet experiences in general. For quite a while the CEO played a game with me where I'd find bad actors on the network and report them to him. He'd thank me and assure me they were on top of figuring out who was behind it. Eventually I figured out that the accounts were in fact his. They let me go shortly after that with generous severance.
I don't miss anything about ad tech. It was such a disheartening introduction to the software world. It's really the armpit and asshole of tech, all at once.
> Ultimately it was only used to install malware in the form of browser extensions, ...
Like any other MDM software.[0] Everyone who has been long enough in the infosec industry knows that MDM is fundamentally nothing more than a corporate-blessed malware and spyware package.
In the past 2-3 years the criminal gangs have realised that too. The modern form of socially engineered phishing quite often entices victims to install a legit MDM software package (eg. MS InTune) and hand over their device control for remote management. Why bother writing malware that has to fiddle with hooks to syscalls and screenshot capabilities when you have a vendor approved way of doing the same?
0: https://en.wikipedia.org/wiki/Mobile_device_management
I think you can only get away with that excuse so long as you're actively looking for a new job while also collecting data to turn whistleblower (anonymously if need be) once you have one. Ultimately it falls on the employee to do the right thing or get out because they risk being held accountable for what they do. A replaceable employee (which is pretty much all of them) will be especially vulnerable since they can be thrown under the bus with minimal inconvenience to the company.
Also likely, some version of "get dat money"
More like "well, they pay well and it's interesting problem so who gives a fuck"
Ah yes let's be sure not to judge anyone for anything they do
People do not make choices in a vacuum.
But they still make their choices and should face the consequences of them.
2 replies →
You can still judge them evil even if the parent was accurate as to the motivations for their actions. Villains are more interesting when they're sympathetic.
You're in the planning meeting discussing this feature, you ask "Hey, are we allowed to do this? I thought stand downs were contractural." and your PM says yes, they got the okay from legal. Now what do you do?
> they got the okay from legal.
Now that I could definitely see happening. I would also want that in writing somewhere.
I guess discovery for the impending lawsuits should be very interesting
It’s easy, looking at the current state of affairs, to conclude that ethical behavior is incompatible with capitalist ambition. One might still choose to be ethical nonetheless, but with the understanding that you will be overtaken by those who have made a different choice.