Comment by theschwa
3 days ago
Between the clear writing and the diagrams, this was a great write up. I had actually skipped reading up on mHC as it sounded like it was going to take some time to grok, but this made it immediately approachable. I hope you do more write ups like this in the future.
imho the prose is very ChatGPT unfortunately
> Residual connections are more than a trick to help gradients flow. They’re a conservation law.
> Not a hack, not a trick. A principled constraint that makes the architecture work at scale.
OK, I thought I was reading too much into it but those same sentences also jumped out for me
1 reply →
yes this reads like classic intellectual fellicitatio
[dead]