Comment by tomrod

There is some art to it, especially in model architecture choices.

But your smarter data scientists will try to get the best information density on test data to approximate generalization. MDL, as championed here, or AIC/BIC if working with more established and acceptable methods.