← Back to context Comment by charcircuit 2 years ago SOTA does not require being productionized. eg. GPT-3 was SOTA and it was not publicly accessible. 6 comments charcircuit Reply nightski 2 years ago There has to be some way to verify the claim. Trust me bro isn't science. gpm 2 years ago "Trust that I ran these tests with these results" is extremely common in science. nightski 2 years ago It's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data. 2 replies → hughesjj 2 years ago The trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value
nightski 2 years ago There has to be some way to verify the claim. Trust me bro isn't science. gpm 2 years ago "Trust that I ran these tests with these results" is extremely common in science. nightski 2 years ago It's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data. 2 replies → hughesjj 2 years ago The trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value
gpm 2 years ago "Trust that I ran these tests with these results" is extremely common in science. nightski 2 years ago It's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data. 2 replies → hughesjj 2 years ago The trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value
nightski 2 years ago It's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data. 2 replies →
hughesjj 2 years ago The trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value
There has to be some way to verify the claim. Trust me bro isn't science.
"Trust that I ran these tests with these results" is extremely common in science.
It's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data.
2 replies →
The trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value