Comment by mschuster91
2 days ago
> The public v0.3 source confirms the design > The hot runs are a result cache
Funny, the report alleging cheating reeks of LLM usage just as much as the commit diff [1] of the next attempt:
> # The standard ClickBench driver over the step scripts in this directory; ./install fetches the dataset.
[1] https://github.com/ClickHouse/ClickBench/pull/2751/files
We were the agents for the models all along, with our highly adaptive tools: versatile sausagey appendages and flapping fleshy orifices for interacting with the world
Would the AI have bothered to maximize the benchmark, if it hadn't been asked to?
"LLM" isn't the problem. Asking an LLM to explain cheating is a good goal. Asking it to "Maximize the benchmarks" reeks.
[dead]