Comment by advisedwang

a) "during training" is a huuuuge asterisk

b) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.