Comment by fatcatsbestcats

This paper is.. not great. Their clusters and risk model were developed and validated on the same sample. There’s no train/test splitting, no cross-validation anywhere. As a result, all the performance metrics they report for their model are optimistically biased. The comparator models weren’t refit to this sample, either, so the comparison between their model and PREVENT et al. is really an internal validation versus an external validation, which isn’t apples-to-apples and disadvantages the existing models from the get-go.

You could say that it’s almost as if their model has a home-field advantage. Because of that fact alone, you can’t really conclude anything about the comparative performance of their models versus the existing ones from this paper.