← Back to context Comment by kadushka 6 months ago Connections between LLM neurons also change during training. 1 comment kadushka Reply advisedwang 5 months ago a) "during training" is a huuuuge asteriskb) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.
advisedwang 5 months ago a) "during training" is a huuuuge asteriskb) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.
a) "during training" is a huuuuge asterisk
b) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.