Comment by joe_the_user

Related: a few years ago, various people were exploring locality sensitive hashing as an alternative to neural networks(SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems [1]).

This never took off. I don't if it didn't work or it was a matter of the "bitter lesson" ideology pushing people to prefer hardware acceleration over smart algorithms. I would note that smart algorithms require hiring (more)smart people and not only can hardware acceleration be cheaper but hardware and data can be more reliably added than smart people, who are short supply.

[1]: https://arxiv.org/abs/1903.03129