← Back to context

Comment by solarkraft

3 hours ago

Is anybody tracking these quality changes? All I've seen so far are accusations (quite a few at this point) but not really any actual data.

4 comments

solarkraft

Reply
pupppet  3 hours ago

I don’t understand how there isn’t a website out there tracking this stuff already.

  • bpavuk  2 hours ago

    how the hell do we even track that? and before someone says...

    —"Benchmarks!"

    ...I'll tell that they can be gamed so easily, and they are on a consistent basis.

    • solarkraft  1 hour ago

      Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?