Comment by Trusteando

A neuron is a dirty, contingent, blind-evolved part, like a screw in a particular machine. It is necessary for this particular machine, but not conceptual, not a feature. The strong activation on top-k images and the change when it is removed just mean that the screw is necessary for this particular machine, not that it's something else. The essay shows neurons are legible, but legibility isn't the same as meaning. Still, given math job prospects, I love screws, they are really not dirty.