Comment by MzxgckZtNqX5i
4 years ago
Section IV.A:
> We send the minor patches to the Linux community through email to seek their feedback. Fortunately, there is a time window between the confirmation of a patch and the merging of the patch. Once a maintainer confirmed our patches, e.g., an email reply indicating “looks good”, we immediately notify the maintainers of the introduced UAF and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our correct patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. All the UAF-introducing patches stayed only in the email exchanges, without even becoming a Git commit in Linux branches. Therefore, we ensured that none of our introduced UAF bugs was ever merged into any branch of the Linux kernel, and none of the Linux users would be affected.
It seems that the research in this paper has been done properly.
EDIT: since several comments come to the same point, I paste here an observation.
They answer to these objections as well. Same section:
> Honoring maintainer efforts. The OSS communities are understaffed, and maintainers are mainly volunteers. We respect OSS volunteers and honor their efforts. Unfortunately, this experiment will take certain time of maintainers in reviewing the patches. To minimize the efforts, (1) we make the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we find three real minor issues (i.e., missing an error message, a memory leak, and a refcount bug), and our patches will ultimately contribute to fixing them.
And, coming to ethics:
> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter.
I'm surprised that the IRB determined this to be not human subjects research.
When I fill out the NIH's "is this human research" tool with my understanding of what the study did, it tells me it IS human subjects research, and is not exempt. There was an interaction with humans for the collection of data (observation of behavior), and the subjects haven't prospectively agreed to the intervention, and none of the other very narrow exceptions apply.
https://grants.nih.gov/policy/humansubjects/hs-decision.htm
> It seems that the research in this paper has been done properly.
How is wasting the time of maintainers of one of the most popular open source project "done properly"?
Also, someone correct me if I'm wrong, but I think if you do experiments that involve other humans, you need to have their consent _before_ starting the experiment, otherwise you're breaking a bunch of rules around ethics.
They answer to this objection as well. Same section:
> Honoring maintainer efforts. The OSS communities are understaffed, and maintainers are mainly volunteers. We respect OSS volunteers and honor their efforts. Unfortunately, this experiment will take certain time of maintainers in reviewing the patches. To minimize the efforts, (1) we make the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we find three real minor issues (i.e., missing an error message, a memory leak, and a refcount bug), and our patches will ultimately contribute to fixing them.
And, coming to ethics:
> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter.
> They answer to this objection as well. Same section:
Not sure how that passage justifies wasting the time of these people working on the kernel. Because the issues they pretend to fix are real issues and once their research is done, they also submit the fixes? What about the patches they submitted (like https://lore.kernel.org/linux-nfs/20210407001658.2208535-1-p...) that didn't make any sense and didn't actually change anything?
> And, coming to ethics:
So it seems that they didn't even just mislead the developers of the kernel, but they also misled the IRB board, as they would never approve it without getting consent from the developers since they are experimenting on humans and that requires consent.
Even in the section you put above, they even confess they need to interact with the developers ("this experiment will take certain time of maintainers in reviewing the patches"), so how can they be IRB-exempt?
The closer you look, the more sour this whole thing smells.
> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter.
I was wondering why he banned the whole university and not just these particular researchers. I think your quote is the answer to that. I'm not sure on what basis this exemption was granted.
Here's what the NIH says about it:
Definition of Human Subjects Research
https://grants.nih.gov/policy/humansubjects/research.htm
Decision Tool: Am I Doing Human Subjects Research?
https://grants.nih.gov/policy/humansubjects/hs-decision.htm
And even if they did find some way to justify it under their own rules, some of the research subjects clearly disagree.
1 reply →
Is "we acknowledge that this will waste their time but we're going to do it anyway" really an adequate answer to that objection?
They appear to have told the IRB they weren't experimenting on humans, but that doesn't make sense to me given that the reaction of the maintainers is precisely what they were looking at.
Inasmuch as the IRB marked this as "not human research" they appear to have erred.
Sounds like the IRB may need to update their ethical standards then. Pointing to the IRB exemption doesn't necessarily make it fine, it could just mean the IRB has outdated ethical standards when it comes to research with comp sci implications.
2 replies →
At least one human, GKH, disagrees.
To me, this further emphasizes the idea that Academia has some serious issues. If some academic institution wasted even 10 minutes of my time without my consent, I'd have a bad taste in my mouth about them for a long time. Time is money, and if volunteers believe their time is being wasted, they will cease to be volunteers, which then effects a much larger ecosystem.
Depends on your notion of "properly". IMO "ask for forgiveness instead of permission" is not an acceptable way to experiment on people. The "proper" way to do this would've been to request permission from the higher echelons of Linux devs beforehand, instead of blindly wasting the time of everyone involved just so you can write a research paper.
That's still not asking permission from the actual humans you're experimenting on, i.e. the non-"higher echelons" humans who actually review the patch.
This points to a serious disconnect between research communities and development communities.
I would have reacted the same way Greg did - I don't care what credentials someone has or what their hidden purpose is, if you are intentionally submitting malicious code, I would ban you and shame you.
If particular researchers continue to use methods like this, I think they will find their post-graduate careers limited by the reputation they're already establishing for themselves.
Saying something is ethical because a committee approved it is dangerously tautological (you can't justify any unethical behavior because someone at some time said it was ethical!).
We can independently conclude this kind of research has put open source projects in danger by getting vulnerabilities that could carry serious real world consequences. I could imagine many other ways to carrying out this experiment without the consequences it appears to have had, like perhaps inviting developers to a private repository and keeping the patch from going public, or collaborating with maintainers to set up a more controlled experiment without risks.
This seems by all appearances an unilateral and egoistic behavior without great thought into its real world consequences.
Hopefully researchers learn from it and it doesn't discourage future ethical kernel research.
The goal of ethical research wouldn't be to protect the Linux kernel, it would be to protect the rights and wellbeing of the people being studied.
Even if none of the patches made into the kernel (which doesn't seem to be true, according to other accounts), it's still possible to do permanent damage to the community of kernel maintainers.
Not really done properly: They were testing out the integrity of the system. This includes the process by which they notified the maintainers not to go ahead. What if that step had failed and the maintainers missed that message?
Essentially, the researchers were not in control to stop the experiment if it deviated from expectations. They were relying on the exact system they were testing to trigger its halt.
We also don't know what details they gave the IRB. They may have passed through due to IRB's naivete on this: It had a high human component because it was humans making many decisions in this process. In particular, there was the potential to cause maintainers personal embarrassment or professional censure by letting through a bugged patch. If the researchers even considered this possibility, I doubt the IRB would have approved this experimental protocol if laid out in those terms.
In my admittedly limited interaction with human subjects research approval, I would guess that this would not have been considered a proper setup. For one thing, there was no informed consent from any of the test subjects.
The piss-weak IRB decided that no such thing was necessary, hence no consent was requested. It's impossible not to get cynical about these review boards, their only purpose seems to be to deflect liability.
In their "clarifications" [1], they say:
"In the past several years, we devote most of our time to improving the Linux kernel, and we have found and fixed more than one thousand kernel bugs"
But someone upthread posted that this group has a total of about 280 commits in the kernel tree. That doesn't seem like anywhere near enough to fix more than a thousand bugs.
Also, the clarification then says:
"the extensive bug finding and fixing experience also allowed us to observe issues with the patching process and motivated us to improve it"
And the way you do that is to tell the Linux kernel maintainers about the issues you observed and discuss with them ways to fix them. But of course that's not at all what this group did. So no, I don't agree that this research was done "properly". It shouldn't have been done at all the way it was done.
[1] https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc....
But still, this kind of research puts undue pressure on the kernel maintainers who have to review patches that were not submitted in good faith (where "good faith" = the author of the patch were trying to improve the kernel)
I think that was kind of the point of the research: submitting broken patches to the kernel represents a feasible attack surface which is difficult to mitigate, precisely because kernel maintainers already have such a hard job.
So what's the null hypothesis here? Human maintainers are infallible? Why this even need to be researched?
If something is determined not to be human research, that doesn't automatically make it ethical.
TIL that opensource project maintainers aren't humans.
Something I've expected for years, but have never had evidence... until now.
Or, alternatively, that submitting buggy patches on purpose is not research.
"in all the three cases" is mildly interesting, as 232 commits have been reverted from these three actors. To my reading this means they either have a legitimate history of contributions with three red herrings, or they have a different understanding of the word "all" than I do.
A “simple” change can still require major effort to evaluate. Bogus logic on their part.