Comment by mjburgess

Sure, every computation is equivalent to a lookup table over "predetermined answers". It isn't equivalent if we don't have those answers.

Eg., "what's the US President's telephone number in 2000?" had no answer in 1900.

> If decision trees were truly equivalent to NNs

They are equivalent. And you don't need to precompute answers you don't have. You can take the weights of a NN and encode them as a DT; just as you can also transform a NN to just be k-nearest-neighbors.

The reason we dont do that is prediction efficiency.

Also, of course, such functions are also basically impossible to train as a practical matter. That bares little on their equivalence.

All ML models are expressible as k-nearest-neighbors -- this is useful information because it demystifies the process. Countless papers end with "and we dont know why!" -- where the "why" is obvious if you reformulate the model.

ML is just ranking a historical dataset of size N, by similarity to some X, selecting up to N examples from it, weighting each by W and then taking an average.