Comment by nojito
5 days ago
>Nobody was watching it?
Agent loops are you give it a task and it tries its best to finish it.
The only results are success and failure. If it's a success you go through the logs to see the actions it took and if it's a failure you do the same thing.
Why would you look at it in realtime when the whole point of agentic work is to get them to run autonomously as long as possible?
My friend, they’re evaluating a new model on a benchmark, not asking Claude Code refactor their GitHub repo. Every single metric is measured so they can brag about it later; how many tokens, how long, how many function calls, ratio of thinking/response tokens. It’s literally a trillion dollar company evaluating their latest model, they’re absolutely studying it.
Do you work at a lab?
Yes evals and training runs are high stakes. But these places and people are also under enormous pressures. They are building as fast as they can. Researchers may have multiple eval runs going on while they work on other things. And it is rarely a single latest model, there are often multiple candidate models training with different recipes, each regularly yielding a new checkpoint for testing.
Some labs are more rigorous than others, but often the "final" model is picked from a handful less than a week before launch.
Yes ideally the world's leading AI companies would be far more careful in evaluating what could be the world's most powerful AI. But this isn't really the state of the industry today. And it is hard to justify being more careful when it means your competitor can go to market faster than you.
I’m not holding them to a standard of “being careful”, I’m assuming they’re interested in the metrics they’re evaluating. I mentioned in another comment that each task in the benchmark takes ~90 minutes, depending on the model being tested. OpenAI says to answer one question it found two zero-days and performed a series of privilege escalations and moved across their network before hacking HF. Was that in 90 minutes, flying under the radar, or was there zero metrics being noticed for however long it took to do all that? No chart on a wall showing them the actual metrics they’re evaluating? Okay, then how about anomaly detection of any form? Taking days to perform a 90 minute task or using millions of tokens and multiple instance hand-offs on a question that other instances completed for far less time/tokens?
Consider any metric they would necessarily be tracking while it performs the evaluation. Assume they’re total assholes and only care about their marketing materials. Then narrow it down to those metrics that wouldn’t be off the scales while it discovered two zero-days in pursuit of implementing one known vulnerability. Was it the first run and they had no idea what to expect? Or did nobody notice that earlier runs completed in ### thousand tokens, but this run is off past ## million on one question, having to be compressed and handed off to new instances due to context limits, while it was in pursuit of publishing two new CVEs?
Assume they’re belligerent fools running a million benchmarks in parallel with nobody watching: there’s nothing to make them check or automatically pull the plug when it’s stuck in a loop, or did they set the threshold to 48 trillion tokens per question?
Those issues are what bugs me about it.
3 replies →