Comment by janalsncm

Very well said. I also think the goal of crackpots isn’t to create something useful but to have their names next to Maxwell, Einstein, and Hawking. Household names of geniuses. Their individual accomplishments are less important.

> the remaining 1% usually builds something that performs within a measurement error of existing systems

Another point that people miss when they wax poetic about how neural nets are like brains or whatnot: we didn’t pick transformers because they were the most elegant method. We use them because they work really well on the hardware we have. It’s why RNNs fell out of favor, they’re way too slow to train. Transformers made training on the whole internet possible.

Maybe RNNs can be salvaged, but my guess is they’ll be approximately as good as transformers but slower to train.