Comment by calf

The difference with your FIR matrix and a deep net matrix is the latter generalized to trillions of parameters i.e. a trillion dimensional space to approximate general, computable functions. The machine learning ideas are fundamentally the same at a high level, just the architecture (nonlinearity whereas in engineering we prefer analyzing linear systems) specific details and tricks and such.

In fact at least at Berkeley a current professor teaching deep learning is a signals expert (Prof Sahai), his lectures are on YouTube.