Comment by srean

True.

It so happens that one gets the best generalization error bounds when one combines PAC with Bayesian ideas -- the PAC-Bayesian bounds.