← Back to context Comment by pupppet 3 hours ago I don’t understand how there isn’t a website out there tracking this stuff already. 3 comments pupppet Reply copperx 44 minutes ago https://modelregression.com/ bpavuk 3 hours ago how the hell do we even track that? and before someone says...—"Benchmarks!"...I'll tell that they can be gamed so easily, and they are on a consistent basis. solarkraft 2 hours ago Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?
bpavuk 3 hours ago how the hell do we even track that? and before someone says...—"Benchmarks!"...I'll tell that they can be gamed so easily, and they are on a consistent basis. solarkraft 2 hours ago Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?
solarkraft 2 hours ago Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?
https://modelregression.com/
how the hell do we even track that? and before someone says...
—"Benchmarks!"
...I'll tell that they can be gamed so easily, and they are on a consistent basis.
Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?