Comment by asdff

The engineers at google did not sit in a blank room and come up with these ideas out of nowhere. They read the literature to figure out what the hell to do. If you look at the Attention is All You Need whitepaper, you will find there are 32 references, like most ~10 page manuscripts.