Comment by yorwba
12 hours ago
Like half of what Schmidhuber is always complaining about is that (except for LSTMs) people aren't standing on the shoulders of his research very much. They try to solve some of the same problems people have always wanted to solve, try some of the same approaches people always tend to try, and then tinker until it works. At no point do they consult Schmidhuber's decade-old papers where he tried something kind of similar but didn't get very impressive results, and hence they also do not think to cite him. Then he comes out of the woodwork to assert priority.
What you're describing is people who fail to follow the most basic principles of academic research. (Check existing academic literature, mention and give credit to prior work.) This would be fine if these people didn't claim to be doing scientific research, didn't boast their academic credentials, didn't publish their finding as original work and didn't demand credit for their work in academia. Of course, they do all of these things. They benefit from a system they're actively denigrating (and in some ways degrading).
To put it more simply, people with academic credentials should not demand acknowledgement of their current intellectual work while denigrating and ridiculing the importance of very similar work done in the past.
It's not just the papers. It's also the students and their students, many of whom ended up working in the top labs today.
You can be influenced downstream by papers you haven't personally read.
Shane Legg was in Schmidhuber's lab at IDSIA before being one of the founders of DeepMind, so he probably read the papers personally and knows what influenced him or not...
Of course, but if you haven't read them you also shouldn't cite them.
And that's where Schmidhuber goes off the rails: publicly shaming published papers into citing you isn't good academic practice. It's bullying.
"if you haven't read them you also shouldn't cite them" -- this is wildly incorrect in an academic context. If I'm using ResNets, I should cite the original ResNet paper, even if I haven't read it. If I'm using Transformers, I should cite the original Transformer paper, even if I haven't read it. If my work is a direct extension of method B, and method B is a direct extension of method A, I should cite the source of A, even if I haven't read it.
You can't claim independence from past work simply because you didn't look directly at it. The job of an academic researcher is to know the landscape of relevant ideas, where they come from, where they're going, and to hopefully contribute a few new good ones.
Citation chains should extend back from your work, along a reasonable line conceptual inheritance, back to a reasonable point of origin. Schmidhuber has different definitions for both of these reasonables than the bulk of the ML research community, to a point that makes him difficult to satisfy.
5 replies →
> Of course, but if you haven't read them you also shouldn't cite them.
But if you build on them you should have read them. I don't know about the specifics and I don't know if Schmidhuber is out of line or not, and citations and impact factors are a terrible mess, but generally speaking, you are responsible for finding and reading and citing any related work that needs to be cited, and if you work on neural networks in an academic context you probably have been forced to read that particular one at some point. Citation obligations don't just disappear because you don't want to do the research.