Comment by hansworst
1 year ago
Overfitting on test data absolutely does mean that the model would perform better in benchmarks than it would in real life use cases.
1 year ago
Overfitting on test data absolutely does mean that the model would perform better in benchmarks than it would in real life use cases.
I think you're talking about something different from what sigmoid10 was talking about. sigmoid10 said "manipulate behaviour during those benchmarks". I interpreted that to mean the compiler detects if a benchmark is going on and alters its behavior only then. So this wouldn't impact real life use cases.