Comment by cyanydeez

except, once you get real analysis on open models, you're likely going to see the same insecurities everywhere because the models have blind spots. The same blind spots. The reason distillation works so well is because of the shared nature of their training corpus.