Comment by newsy-combi
5 hours ago
The notation itself sucks and anyone taking pride in the ability to parse it is suffering from Stockholm syndrome. It's also one of the most annoying things to look up because search engines ignore the symbols. AI can explain it based on a screenshot now, but then we might as well go to AI for the entire thing and I'm starting to lean in that direction just to spite authors like OP. You're also connecting the syntax to understanding of the concepts which is the cherry on top of the math snobbery in this thread. If the blog had pseudo-code instead of 18th century scribbles, the HN crowd would at least not openly admit to any parts they don't understand, because it's implied that you can code when you post here - not so much that you give a flying f--- about math or really anything theoretical.
> If the blog had pseudo-code instead of 18th century scribbles
E for expectation, blackboard bold, double-bar norms and indexed max/min are all early to mid 20th century notation, making them thoroughly modern. Only summation notation dates since the eighteenth century, and its relative stability over that period is in fact rather a strength and not a weakness. Summation notation is also nearly universal in any quantitative field, and far, far more concise than the equivalent looping C-style pseudocode or functional-style pseudocode.
> You're also connecting the syntax to understanding of the concepts which is the cherry on top of the math snobbery in this thread
Because that is in fact a big part of how it works. Good notation takes a lot of time and thought to develop, and the notation used in the post is far, far more concise and packs far more information into a tiny amount of space than pseudocode or other programming conventions. Much of math is abstraction (compacting complex ideas into specific symbols or concepts, and then coming up with simple rules to allow their manipulation), and is possible in large part only because of good notation like this. See some of Terry Tao's thoughts on the subject below.
The notation here is really also all first-year introductory notation, and incredibly simple. Most CS or computer engineering students should have seen it, and anyone that thinks they can learn anything non-trivial at all about neural networks by avoiding such basic notation is simply not a serious person at all.
EDIT: And it is a mistake to imply pseudocode is generally superior or easier to understand, because pseudocode is often too precise, and implies more structure and relations and implementation details than are actually needed or important. Meaning it actually often communicates the wrong thing compared to good notation.
https://terrytao.wordpress.com/advice-on-writing-papers/use-...
https://mathoverflow.net/questions/366070/what-are-the-benef...