← Back to context

Comment by pupppet

4 hours ago

I don’t understand how there isn’t a website out there tracking this stuff already.

how the hell do we even track that? and before someone says...

—"Benchmarks!"

...I'll tell that they can be gamed so easily, and they are on a consistent basis.

  • Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?