Comment by kadushka
15 days ago
Theory is still needed if you want to understand things like variational inference (which is in turn needed to understand things like diffusion models). It’s just like physics - you need math theories to understand things like quantum mechanics, because otherwise it might not make sense.
I think machine learning research is more like engineering, where you do need some math, but you don't need a physics degree. You don't need to understand everything first to discover that some engineering solutions work and others don't. And most abstract theories likely wouldn't have helped you anyway because they are not sufficiently concrete to apply to what you are doing in practice.
To make some progress in ML you might not need a lot of theory, but to understand why things work – you absolutely do. Moreover, the DL field as a whole desperately needs theories explaining what’s going on in these large models.