← Back to context Comment by WithinReason 1 day ago I made an explicit reverse pass (no autodiff), it was 8x faster in Python 3 comments WithinReason Reply hu3 2 hours ago I made an explicit double-reverse pass (no code!), it was 80x faster in my head! bear3r 2 hours ago tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists. love2read 8 hours ago Can you share a link?
bear3r 2 hours ago tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.
I made an explicit double-reverse pass (no code!), it was 80x faster in my head!
tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.
Can you share a link?