Comment by jsheard
2 months ago
> So the "Verified" part of "SWE Bench Verified" means.. not "Verified" at all.
Seems on-brand for an LLM-related thing to claim that it has verified something without actually checking.
2 months ago
> So the "Verified" part of "SWE Bench Verified" means.. not "Verified" at all.
Seems on-brand for an LLM-related thing to claim that it has verified something without actually checking.
that was my exact thought. how fitting
Verified has a completely different meaning for this, it's that the questions have verified valid solutions.