Comment by pishpash
18 hours ago
There is no point in that because the loss function itself is already an approximation. No one knows what is the exact loss function for any given non-trivial real-world task.
18 hours ago
There is no point in that because the loss function itself is already an approximation. No one knows what is the exact loss function for any given non-trivial real-world task.
Sure, that's kind of my point.
There is an optimal set of weights that minimizes the loss function for a given set of training data, but we cannot find it.
Granted, even if we could, it might just be overfitting.
I mean, things humans defined can be pretty clear. Like your electricity rate. Natural systems less so. Not pretending no complexity in human made things, but at least some models can be fully specified.