Comment by cpldcpu
3 days ago
Thanks! Would be quite interesting to see how this fares compared to mHC.
I noted that LAuReL is cited in the mHC paper, but they refer to it as "expanding the width of the residual stream", which is rather odd.
3 days ago
Thanks! Would be quite interesting to see how this fares compared to mHC.
I noted that LAuReL is cited in the mHC paper, but they refer to it as "expanding the width of the residual stream", which is rather odd.
No comments yet
Contribute on Hacker News ↗