Comment by ENOTTY

Even better

Notify someone up the chain that you want to submit malicious patches, and ask them if they want to collaborate.

If your patches make it through, treat it as though they essentially just got red teamed, everyone who reviewed it and let it slip gets to have a nervous laugh and the commit gets rejected, everyone having learned something.

Wouldn't that draw more attention to the research patches, compared to a "normal" lkml patch? If you (as a maintainer) expected the patch to be malicious, wouldn't you be extra careful in reviewing it?

Dd they keep track of and submit a list of additions to revert after they managed to get it added?

From the looks of it they didn't even when it was heading out to stable releases?

That's just using the project with no interest in not causing issues.

> This is funny, but not at all a good analogy

Yeah, for one thing, to be a good analogy, rather than lockpicking without entering when he’s not home and leaving a note, you’d need to be an actual service worker for a trusted home service business and use that trust to enter when he is home, conduct sabotage, and not say anything until the sabotage is detected and traced back to you and cited in his cancelling the contract with the firm for which you work, and then cite the “research” rationale.

Of course, if you did that you would be both unemployed and facing criminal charges in short order.

Everyone has been saying "This affects software that runs on billions of machines and could cause untold amounts of damage and even loss of human life! What were the researchers thinking?!" and I guess a follow-up thought, which is that "Maintainers for software that runs on billions of machines, where bugs could cause untold amounts of damage and even loss of human life didn't have a robust enough system to prevent this?" never occurs to anyone. I don't understand why.

It wasn't intended to be serious. But on the other hand, he has now quite openly and publicly declared himself to be part of a group of people who mess around with security related things as a "test".

He shouldn't be surprised if it has some unexpected consequences to his own personal security, like some unknown third parties porting away his phone number(s) as a social engineering test, pen testing his office, or similar.

There's also not nearly as much harm as there is in wasting maintainer time and risking getting faulty patches merged.

The main thing you want here is a demonstration that they realize they fucked up, realize the magnitude of the fuckup, and have done something reasonable to lower the risk of it happening again, hopefully very low.

Given that the professor appears to be a frequent flyer with this, the kernel folks banning him and the university prohibiting him from using Uni resources for anything kernel related seems reasonable and gets the point across.

Expel the students and fire the professor. That will demonstrate their commitment to high ethical standards.

The comment about IRB —- institutional research board —- is clear, I think.

Probably that, combined with "we informed the professor of {serious consequences} should this happen again".

Well, yes? Seems like recourse in their case would be to make a convincing plea or plan to rectify the problem that satisfies decision makers in the linux project.

> good, conscientious researchers and students at the University of Minnesota who have been working on the kernel in good faith

All you have to do is look at the reverted patches to see that these are either mythical or at least few and far in between.

> 1. The experiment is not on live, deployed, versions of the kernel.

The patches were merged and the email thread discusses that the patches made it to the stable tree. Some (many?) distributions of Linux have and run from stable.

> 2. There are mechanisms in place for preventing actual merging of the faulty patches.

Those mechanisms failed.

> 3. Even if a patch is merged by mistake, it can be easily backed out or replaced with another patch, and the updates pushed anywhere relevant.

Arguably. But I think this is a weak argument.

You seem to think this experiment was performed on the Linux kernel itself. It was not. This research was performed on human beings.

It's irrelevant whether any bugs were ultimately introduced into the kernel. The fact is the researchers deliberately abused the trust of other human beings in order to experiment on them. A ban on further contributions is a very light punishment for such behavior.

How would you feel about researchers delivering known-faulty-under-some-conditions AoA sensors to Boeing, just to see if Boeing's QA process would catch those errors before final assembly?

I would then say that the goal and the choice aren't aligned because "penalizing or lumping everyone together" is exactly the choice made.

It's not the number of people directly affected that will matter, it's the reputational problems of "umn.edu's CS department got the entire UMN system banned from submitting to the Linux kernel and probably some other open source projects."

And anyone without much power to effect change SOL.

I know the kernel doesn't need anyone's contributions anyhow, but as a matter of policy this seems like a bad one.

The ethics board is most likely not at fault here. They were simply lied to, if we take Lu's paper serious. I would just expell the 3 malicious actors here, the 2 students and the Prof who approved it. I don't see any fault in Wang yet.

The damage is not that big. Only 4 committers to linux in the last decade, 2 of them, the students, with malicious backdoors, the Prof not with bad code but bad ethics, and the 4th, the Ass Prof did good patches and already left them.

On the other hand: What obligation do the Linux kernal maintainers have to allow UMN staff and students to contribute to their project?

> Does the ban stay with UMN, even after everyone involved left?

It stays with the university until the university provides a good reason to believe they should not be particularly untrusted.

If they use a different email but someone knows they work at the university?

It's a chain that gets really unpleasant.

It varies a lot. A professor I worked for was previously at a large company in an R&D setting. He dealt with 15-20 different IRB's through various research partnerships, and noted Iowa State (our university) as having the most stringent requirements he had encountered. In other universities, it was pretty simple to submit and get approval without notable changes to the research plan. If they were unsure on something, they would ask a lot of questions.

I worked on a number of studies through undergrad and grad school, mostly involving having people test software. The work to get a study approved was easily 20 hours for a simple "we want to see how well people perform tasks in the custom software we developed. They'll come to the university and use our computer to avoid security concerns about software security bugs". You needed a script of everything you would say, every question you would ask, how the data would be collected, analyzed, and stored securely. Data retention and destruction policies had to be noted. The key linking a person's name and their participant ID had to be stored separately. How would you recruit participants, the exact poster or email you intend to send out. The reading level of the instructions and the aptitude of audience were considered (so academic mumbo jumbo didn't confuse participants).

If you check the box that you'll be deceiving participants, there was another entire section to fill out detailing how they'd be deceived, why it was needed for the study, etc. Because of past unethical experiments in the academic world, there is a lot of scrutiny and you typically have to reveal the deception in a debriefing after the completion of the study.

Once a study was accepted (in practice, a multiple month process), you could make modifications with an order of magnitude less effort. Adding questions that don't involve personal information of the participant is a quick form and an approval some number of days later.

If you remotely thought you'd need IRB approval, you started a conversation with the office and filled out some preliminary paperwork. If it didn't require approval, you'd get documentation stating such. This protects the participants, university, and professor from issues.

--

They took it really seriously. I'm familiar with one study where participants would operate a robot outside. An IRB committee member asked what would happen if a bee stung the participant? If I remember right, the resolution was an epipen and someone trained in how to use it had to be present during the session.

They are probably more familiar with medical research and the types of things that go wrong there. Bad ethics in medical situations is well understood, including psychology. However it is hard to figure out how a mechanical engineer could violate ethics.

Plus, it sets a precedent: if your university condones this kind of "research", you will have to face the consequences too...

It seems fair enough to me. They were curious to see what happens, this happens. Giving them a free pass because they're a university would be artificially skewing the results of the research.

Low trust and negative trust should be fairly obvious costs to messing with a trust model - you could easily argue this is working as intended.

It approved the research, which I don't find objectionable.

The objectionable part is that the group allegedly continued after having been told to stop by the kernel developers.

That's not really how it works. Nobody's out there 'approving' research (well, not seemingly small projects like this), especially at the university level. Professors (all the way down to PhD students!) are usually left to do what they like, unless there are specific ethical concerns that should be put before a review panel. I suppose you could argue that this work should have been brought before the ethics committee, but it probably wasn't, and in CS there isn't a stringent process like there is in e.g. psychology or biology.

Apparently they aren't doing human experiments, it's only processes and such. So they can easily emulate the processes in-house too!

For other research happening in the university. This particular research is trivial anyway, see https://news.ycombinator.com/item?id=26888417

Ascribing a salutary motive to sabotage is just as dangerous as assuming a pernicious motive. Suggesting that people "would" likely follow one course of action or another is also dangerous: it is the oldest form of sophistry, the eikos argument of Corax and Tisias. After all, if publishing research rules out pernicious motives, academia suddenly becomes the best possible cover for espionage and state-sanctioned sabotage designed to undermine security.

The important thing is not to hunt for motives but to identify and quarantine the saboteurs to prevent further sabotage. Complaining to the University's research ethics board might help, because, regardless of intent, sabotage is still sabotage, and that is unethical.

The difference between:

"Dear GK-H: I would like to have my students test the security of the kernel development process. Here is my first stab at a protocol, can we work on this?"

and

"We're going to see if we can introduce bugs into the Linux kernel, and probably tell them afterwards"

is the difference between white-hat and black-hat.

Auditability is at the core of its advantage over closed development.

Submitting bugs is not really testing auditability, which happens over a longer timeframe and involves an order of magnitude more eyeballs.

To adress your first critic: benevolence, and assuming everyone wants the best for the project, is very important in these models, because the resources are limited and dependent on enthusiasm. Blacklisting bad actors (even if they have "good reasons" to be bad) is very well justified.

Seems like a reasonable default assumption to me, until the people repeatedly attempting to sabotage the open source community condescend to -- you know -- stop doing it and then explain wtf they are thinking.

[Edit: they seem to truly love OSS. See child comments. Sorry for my erroneous judgement. It reminded too much of anti-opensource FUD, I'm probably having PTSD of that time...]

I fixed my sentence.

I still think that these professors, either genuinely or by lack of willingness, do not understand the mechanism by which free software warrants its greater quality compared to proprietary ones (which is a fact).

They just remind me the good old days of FUD against open source by Microsoft and its minions...

Fair point.

I'll just leave my comment as it is. The university administration still bears responsibility in the fact that they waived the IRB.

You need to compare what happens with vulnerabilities in OSS vs in proprietary.

A maintainer pakage is just one more open source software (thus also in need of reviews and audits)... which is why some people prefer upstream-source-based distribs, such as Gentoo, Arch when you use git-based AUR packages, or LFS for the hardcore fans.