← Back to context

Comment by karsinkk

4 years ago

Here's a clarification from the Researchers over at UMN[1].

They claim that none of the Bogus patches were merged to the Stable code line :

>Once any maintainer of the community responds to the email,indicating “looks good”,we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux.

I haven't been able to find out what the 3 patches which the reference are, but the discussions on Greg's UMN Revert patch [2] does indicate that some of the fixes have indeed been merged to Stable and are actually Bogus.

[1] : https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc....

[2] : https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...

The response makes the researchers seem clueless, arrogant, or both - are they really surprised that kernel maintainers would get pissed off at someone deliberately wasting their time?

From the post:

  * Does this project waste certain efforts of maintainers?
  Unfortunately, yes. We would like to sincerely apologize to the maintainers involved in the corresponding patch review process; this work indeed wasted their precious time. We had carefully considered this issue, but could not figure out a better solution in this study. However, to minimize the wasted time, (1) we made the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we tried hard to find three real bugs, and the patches ultimately contributed to fixing them

"Yes, this wastes maintainers time, but we decided we didn't care."

  • Fascinating that the research was judged not to involve human subjects....

    As someone not part of academia, how could this research be judged to not involve people? It _seems_ obvious to me that the entire premise is based around tricking/deceiving the kernel maintainers.

    • Yeah, especially when the researcher begins gaslighting his subjects. He had the gall to call the maintainer's response "disgusting to hear", and then went on to ask for "the benefit of the doubt" after publishing a paper admitting that he decieved them.

      For comparison, imagine that you attented a small conference and unknowingly became a test subject, and when you sit down the chair shocks you with a jolt of electricity. You jump out of your chair and exclaim, "This seat shocked me!" Then the person giving the presentation walks to your seat and sits down and it doesn't shock him (because he's the one holding the button), and he then accuses you of wasting everyone's time. That's essentially what happened here.

    • That was my thinking too, surely their school's IRB would have a field day with this. The question is whether they ran this by their IRB at all. If they did it, there would be implications on the ethics of everything coming out of UMN. If they didn't, then the same for their lab. I know at my school things were quite clear - if your work requires any interaction with any human not in your lab, you need IRB approval. This is literally just a social engineering experiment, so of course IRB should have reviewed it.

      https://research.umn.edu/units/irb

      1 reply →

    • And given the apparent failure of UMN's IRB, banning UMN from contributing to the Linux kernel honestly seems like a correct approach to resolve the underlying issue.

      1 reply →

  • This is an indignant rebuttal, not an apology.

    No one says "wasted their precious time" in a sincere apology. The word 'precious' here is exclusively used for sarcasm in the context of an apology, as it does not represent a specific technical term such as might appear in a gemology apology.

    • Your particular criticism is not fair in my opinion. Both researchers went to undergrad outside the U.S., so they may not speak English as a first language. Therefore, it's not fair to assume to intended that connotation.

      7 replies →

    • I think this may be unintended. It is very hard to formulate a message that essentially says both "we recognize your time is valuable" and "we know we waste your time, but we decided it's not very important" at the same time, without it sounding sarcastic on some level. Inherent contradiction of the message would get through, regardless of the wording chosen.

      2 replies →

  • Or more charitably: "Yes, this spent some maintainers time, but only a small amount and it resulted in bugfixes, which is par for the course of contributing to linux"

  • Indeed! "we could not figure out a better solution in this study".

    There IS a better solution: not to proceed with that "study" at all.

    • Well, or do what an ethical research would do and seek authorization from the board of Linux foundation before doing any (who knows potentially illegal social engineering attacks) on team members.

    • Next study would involve creating fake grant to study researcher review process of discerning which grant is a scam or not

    • Exactly. Since when is "people will sometimes believe lies" a uncertain question that needs experimental review to confirm?

      Maybe that cop convicted yesterday was actually just a UMN researcher investigating the burning scientific question "does cutting off someone's airway for 9 minutes cause death?".

      1 reply →

  • Your honor, I tried to find any solution for testing this new poison without poisoning a bunch of people, but I carefully considered it and I couldn't find any, so I went ahead and secretly poisoned them. Clearly, I am innocent! Though I sincerely apologize for any inconvenience caused.

  • > We had carefully considered this issue, but could not figure out a better solution in this study.

    Couldn't figure out that "not doing it" was an option apparently.

  • > clueless, arrogant, or both

    I'm going to go with "both" here.

    • Had him as a TA, can confirm. Rudest and more arrogant TA I've ever worked with. Outright insulted me for asking questions as a transfer who had never used Linux systems before. Him implying he was ignorant and new is laughable when his whole demeanor was that you're an imbecile for not knowing how these things work.

In the end, the damage has been done and the Linux developers are now going back and removing all patches from any user with a @umn.edu email.

Not sure how the researchers didn't see how this would backfire, but it's a hopeless misuse of their time. I feel really bad for the developers who now have to spend their time fixing shit that shouldn't even be there, just because someone wanted to write a paper and their peers didn't see any problems either. How broken is academia really?

  • This, in of itself, is a finding. The researchers will justify their research with "we were banned which is a possible outcome of this kind of research..." I find this disingenuous. When a community of open source contributors is partially built on trust, then violators can and will be banned.

    The researchers should have approached the maintainers got get buy in, and setup a methodology where a maintainer would not interfere until a code merge was immanent, and just play referee in the mean time.

    • I don’t mind them publishing that result, as long as they make it clear that everyone from the university was banned, even people not affiliated with their research group. Of course anyone can get around that ban just by using a private email address (and the prior paper from the same research group started out using random gmail accounts rather than @umn.edu accounts), but making this point will hopefully prevent anyone from trying the same bad ideas.

  • I feel the same way. People don't understand how it is difficult to be a maintainer. This is very selfish behaviour. Appreciate Greg's strong stance against it.

> I haven't been able to find out what the 3 patches which the reference are, but the discussions on Greg's UMN Revert patch [2] does indicate that some of the fixes have indeed been merged to Stable and are actually Bogus.

That's because those are two separate incidents. The study which resulted in 3 patches was completed some time last year, but this new round of patches is something else.

It's not clear whether the patches are coming from the same professor/group, but it seems like the author of these bogus patches is a Phd student working with the professor who conducted that study last year. So there is at least one connection.

EDIT: also, those 3 patches were supposedly submitted using a fake email address according to the "clarification" document released after the paper was published. So they probably didn't use a @umn.edu email at all.

The main issue here is that it wastes the time of the reviewers and they did not address it in their reply.

It's disrespectful to people who are contributing their personal time while working for free on open source projects.

With more than 60% of all acedemic publications not being reproducible [1], one would think academia has better things to do than wasting other people's time.

[1] https://en.wikipedia.org/wiki/Replication_crisis

I wonder why they didn't just ask in advance. Something like 'we would like to test your review process over the next 6 months and will inform you before a critical patch hits the users', might have been a win-win scenario.

It seems to me like the other patches were submitted in good faith, but that the maintainer no longer trusts them because of the other bad commits.