Comment by eightysixfour
2 days ago
You don’t see the value of independent replication of findings?
The agent didn’t have access to the code, although they acknowledge it could theoretically be in the training set, even then the original code wouldn’t conform to the structure of the test.
> even then the original code wouldn’t conform to the structure of the test.
yeah, this part should be central in this work: how well those tests are built, do they actually are catching data leakage, how this is measured, etc.