← Back to context Comment by paxys 2 days ago Makes sense why they released an entire study yesterday discrediting SWE-bench Pro. 1 comment paxys Reply osti 2 days ago And they'd be right, it's an almost saturated benchmark where even some subpar open source models score very well on. And most models are clustered within a small range so it really doesn't tell you much.
osti 2 days ago And they'd be right, it's an almost saturated benchmark where even some subpar open source models score very well on. And most models are clustered within a small range so it really doesn't tell you much.
And they'd be right, it's an almost saturated benchmark where even some subpar open source models score very well on. And most models are clustered within a small range so it really doesn't tell you much.