Comment by ACCount37

And this proves what exactly? That any task can be solved by pure chance at pass@k, with k blowing out to infinity as the solution space grows?

We know that. The value of intelligence is being able to outperform that random chance.

LLMs already outperform a typewriter monkey, a keyboard cat, and a non-insignificant amount of humans on a very diverse range of tasks.