Comment by famouswaffles
4 hours ago
>Adding a path finding algorithm and environment transform tools to a supposed "AGI", sure does seem like cheating to me.
You would need all that if you, a human wanted any chance of solving this benchmark in the format LLMs are given. The funny thing about this benchmark is that we don't even know how solvable it is, because the baseline is tested with radically different inputs.
>I guess you really want to love the current SOTA LLMs. It's a shame they're dumb af.
I guess you really don't want to think critically. Yeah good day lol.
Really tired of you making up stuff about this. The baseline and entire benchmark evaluation is clearly defined, with a statistically sound number of participants for the baseline using the same consistent deterministic environments to perform evaluation. The fact you don't like where the "human performance" line was drawn or how the scale is derived is not the same as the benchmark being tested with "radically different inputs". Clearly you would rather hype AI than critically advance it. So I won't waste time with someone who is clearly not posting in good faith.
Byebye now.
Humans and LLMs are not seeing the benchmark in the same format. What's made up about that ? Can you solve this in the JSON format ?
Look man, don't reply if you don't want to.