Comment by fwip
3 days ago
To carry on this analogy - do test prep workbooks make you meaningfully more competent in general, or is it benchmaxing? (Versus studying textbooks for a similar time, of course.)
3 days ago
To carry on this analogy - do test prep workbooks make you meaningfully more competent in general, or is it benchmaxing? (Versus studying textbooks for a similar time, of course.)
All machine learning is benchmaxxing. It's a big problem.
It all comes down to how aligned the benchmarks are with what you want the model to do. If it's close, good performance. If it's not, unpredictable performance.